AI
How AI models work inside, where the cost goes, how to read the numbers people publish, and how to check what agents build.
Series
Series · 4 lessons
Attention and long context
Why AI models pay more and more to read long text, the ways people cut that cost, how a tree of summaries searches a context, and how to judge the claims.
Start the series- Attention: why long text costs AI models so muchClick a word to see where attention looks back, then see why doubling the length of a text means four times the work.
- Ways to stop AI models rereading everythingCompare a sliding window, indexers and a tree on the same context, and see which forgets and which only moves the cost elsewhere.
- Searching a context with a tree: how ZHA worksWatch a question walk down a tree level by level, tune top-p, try random data, then run a small version in about 130 lines.
- 400 times faster: reading the ZHA results honestlySeparate what was measured from what was projected in the ZHA paper, see why deep trees lose attention, and take a checklist to the next paper.