Dolittle
An engraved green parrot speaks into a brass horn as glowing marks become handwriting in an open notebook.

Thinking text & internal activations

Dolittle

Two views of a thinking model.
Room to explore what they tell us.

By Andrew Hunter

I read Anthropic’s paper on natural language autoencoders and thought it would be a fun experiment to build: put a model’s generated thinking beside interpretations of its internal activations, and make both easy to explore.

Start with the idea

From the workbench

All writing

One captured moment

The same point. A different description.

In this reference run, a key has moved from a drawer to a coat pocket. The model’s thinking and the NLA’s description take us to different places.

Target · thinking text

The coat is hanging by the door, so the key is in the coat pocket, which is part of the coat.

NLA · activation interpretation

the entire response resolves the key's location, final sentence restating the answer for the key's position tied to Rosa being inside the suitcase in Mexico City.

Exact excerpts from the Qwen3 8B reference run · 16 October 2026 · block 24, position 426. Shortened excerpts, not a live demo.

We can check that the transcript says “coat pocket.” Establishing what the activation represents takes more than comparing those words.

How to read a disagreement

The build

First, a pair we can run.

Qwen3 8B and its NLA pair have run on a 24 GB RTX 3090 Ti. Next comes the C++/CUDA engine and a browser interface for inspecting a generation. Explanations and explorations will accompany the work; milestones will mark capabilities we can use.

Reference runs complete. NVIDIA first; Metal planned next. AMD support is undecided. The engine and live interface are still to come.

About the project