Dwipal Desai
Dwipal Desai
  • Blog
  • About
  1. Home
  2. Posts
  3. Relearning the Bitter Lesson... while scaling AI products

Relearning the Bitter Lesson... while scaling AI products

Oct 22, 2026 Dwipal Desai

Rich Sutton’s The Bitter Lesson is one of my favorite essays on AI. The main insight from 70+ years of research: general methods that scale with compute beat clever, human-designed knowledge.

Wearable sensor signals from Scaling Wearable Foundation Models

I keep relearning it the hard way on my last few projects:

  • Health - we used lots of specific classifiers for each insight, surrounded by tons of scaffolding. Lately, the direction of travel is large, multi-modal models that give better insights.

  • Agent context and memory - we went from complex graph databases, rigid schemas and rich taxonomies to a much simpler structure with a fraction of the complexity.

  • Playground - we spent tremendous effort on the scaffolding and archetypes, and it all got thrown away when the new model landed.

We’ve seen this across the industry - including Boris Cherny, who created Claude Code and talks about scaffolding as temporary duct tape until the model catches up.

High quality, artisanal, hand-built stuff brings a lot of joy, which makes this lesson even more bitter.

Recent Posts

  • Why didn't AI replace radiologists? Nov 10, 2026
  • Relearning the Bitter Lesson... while scaling AI products Oct 22, 2026
  • Launching Google Playground - Create > Play > Share! Oct 7, 2026
  • Understanding PAUSD Budget Dynamics Mar 3, 2026
  • Who is reading all the stuff AI is writing? Mar 1, 2026

Tags

  • Finance
  • Travel
  • Culture
  • Mobile Phones
  • Business Strategy
  • Entrepreneurship
  • Gadgets
  • Music
  • Photography
  • Productivity
  • Startups
  • Apple
  • Consumerism
  • Design
  • Life Hacks
  • Personal
  • Tech History
  • Tech Trends
  • YouTube
  • Analytics
← Launching Google Playground - Create > Play > Share! Why didn't AI replace radiologists? →
© 2026 Dwipal Desai.
"Be curious, not judgemental."