Rich Sutton’s The Bitter Lesson is one of my favorite essays on AI. The main insight from 70+ years of research: general methods that scale with compute beat clever, human-designed knowledge.
I keep relearning it the hard way on my last few projects:
Health - we used lots of specific classifiers for each insight, surrounded by tons of scaffolding. Lately, the direction of travel is large, multi-modal models that give better insights.
Agent context and memory - we went from complex graph databases, rigid schemas and rich taxonomies to a much simpler structure with a fraction of the complexity.
Playground - we spent tremendous effort on the scaffolding and archetypes, and it all got thrown away when the new model landed.
We’ve seen this across the industry - including Boris Cherny, who created Claude Code and talks about scaffolding as temporary duct tape until the model catches up.
High quality, artisanal, hand-built stuff brings a lot of joy, which makes this lesson even more bitter.
