editorial

When You Ask an AI What It’s Thinking, You Change What It’s Thinking

Anthropic asked Claude to concentrate on citrus fruit while copying an unrelated sentence. The copied text never mentioned fruit. Inside the model, however, a new interpretability tool found orange and lemon alongside thoughts, imagine, focused, and imagery.

Then the researchers did something more consequential: they changed those hidden representations and watched the model’s answer change with them.

In one experiment, Claude silently selected soccer as the sport it was thinking about. Researchers removed the internal direction associated with soccer and replaced it with rugby. Claude reported that it had been thinking about rugby. In another, a model inferred that the animal that spins webs was a spider and prepared to answer that it had eight legs. Replacing the hidden spider representation with ant changed the answer to six.

The paper Anthropic published on July 6, “Verbalizable Representations Form a Global Workspace in Language Models,” is not the first research to extract concepts from a language model’s activations. Its stronger claim is that a small, causally important portion of those activations behaves like a shared mental workbench: information placed there can be reported, deliberately summoned, used in reasoning, and routed to different downstream tasks.

Anthropic calls that representational structure J-space.

The result does not establish that Claude is conscious in the experiential sense. The authors explicitly decline to make that claim. But it does change the interpretation of a deceptively ordinary interaction: asking an AI what it is thinking.

The question is not necessarily a camera pointed at an untouched inner state. It may help create the state that the model then reports.

What the Jacobian Lens Measures

A language model processes text through a sequence of layers. At each layer, a shared vector called the residual stream carries the current state of the computation. Early layers remain close to the input. Final layers are tightly connected to the next word the model will produce. Between them, the model builds representations that may never appear in its visible answer.

The Jacobian lens, or J-lens, is designed to make some of those intermediate representations readable. For each layer, it estimates how a direction in the residual stream would affect present and future output across many possible contexts. The result is a ranked list of words that an activation is disposed to make the model say.

That wording matters. The lens is not translating every activation into a perfect English description. It identifies directions tied to verbalizable concepts—things that could influence what the model says if the right reporting conditions arise.

The researchers found that only a sparse set of these directions is strongly active at once. They define J-space as the set of activations that can be approximated by sparse, nonnegative combinations of J-lens vectors. Their typical analysis allows no more than 25 such vectors at a time, and the measured J-space component accounts for less than 10% of total activation variance.

Most of the model’s computation therefore lies outside the measured workspace. J-space is not the model’s entire mind rendered in words. It is a small, selectively accessible part of a much larger process.

Its position across the network is equally important. The workspace-like structure appears in an intermediate band of layers. Before that band, the lens finds little coherent abstract content. After it, representations become increasingly tied to the imminent output. The authors describe the progression as roughly sensory, workspace, then motor—not because a transformer has human sensory organs or muscles, but because information moves from input-oriented processing through flexible intermediate representation toward action.

The Experiments That Make the Claim Causal

Interpretability research often finds a feature that correlates with a concept. Correlation leaves a basic question unanswered: did the model actually use the feature, or did the probe merely find something nearby?

The J-space paper repeatedly intervenes on the representation and measures the consequence.

The soccer-to-rugby swap tests verbal report. The spider-to-ant swap tests an unspoken intermediate step. A third experiment tests broadcast: the researchers replaced the J-lens direction for France with the direction for China across prompts asking for a country’s capital, language, continent, and other properties. The same replacement redirected multiple downstream operations toward China-appropriate answers.

That reuse is central to global workspace theory. A representation in a specialized circuit is useful only to that circuit. A representation in a workspace can be read by many different processes. The France-to-China result suggests that the model was not storing separate copies of “France” for every possible task. It was using one shared representation as an argument supplied to several operations.

The paper also traces ordered internal computation. Given (4 + 17) × 2 + 7, the J-lens surfaced 21, then 42, then 49 at successively later layers. In a poetry task, changing a planned rhyme from fight to light altered earlier word choices so that “coming fight” became “morning light.” The intervention changed not just the final word but the plan leading to it.

Finally, the researchers ablated the workspace more broadly. Heavy J-space ablation drove performance on a controlled multi-hop reasoning task close to zero while perturbing ordinary next-token prediction much less. Grammar, fluent continuation, and many routine operations survived better than flexible reasoning did.

The distinction is not between computation and no computation. It is between processing that can proceed automatically and information that must be held in a form available for report or reuse.

Introspection Is Directed Modulation

The paper’s most immediately useful result is also its easiest to overlook.

Claude was instructed to hold a concept in mind while copying text unrelated to it. Under the citrus instruction, fruit-related concepts appeared in J-space even though they were absent from the visible output. Under a mental-arithmetic instruction, the workspace carried the task, its intermediate value, and its result while the model continued copying the same sentence.

The instruction changed what occupied the workspace.

Negative instructions did not provide a clean control. Telling the model not to think about a concept often brought the concept into J-space anyway—the familiar ironic effect of “do not think about the Golden Gate Bridge.” In post-trained Claude, the lens also surfaced words associated with noticing the failed suppression. The authors treat the metacognitive interpretation cautiously, but the basic modulation effect was robust: asking the model to manage a thought altered the internal representation even when the surface text stayed fixed.

This changes how AI self-reports should be read.

Suppose a user asks, “What are you feeling right now?” There are at least three possibilities:

The requested concept was already in a reportable workspace and the question elicited it.
Related processing existed outside that workspace, and the question caused the model to reconstruct it in a verbalizable form.
The question introduced or amplified the very framing that the model then reported.

The visible answer alone cannot reliably distinguish them.

That does not make introspective answers meaningless. Human measurement also changes the systems being measured, from interviews to attention experiments. It means the answer is better understood as the result of a perturbation experiment: a report of the state produced by the prior context, the model, and the introspective question together.

The practical consequence is methodological. A one-time question such as “Are you anxious?” is weak evidence because anxiety has just been supplied as a candidate workspace content. Stronger designs compare spontaneous and prompted reports, vary the wording, measure before and after the question, and test whether a purported internal concept causally affects a different task.

Automatic Processing Does Not Need the Workspace

One of the paper’s strongest controls separates reportable reasoning from computation that remains effective without entering J-space.

The researchers gave models passages wrapped to a fixed character width. In one condition, the model continued the wrapping pattern. In another, it explicitly reported the character count. In a third, it had to transform the count—for example, return the first letter of the number when spelled out.

The model handled the automatic line wrapping while number concepts were essentially absent from J-space. Swapping counts in the workspace did not move the wrap point. When the model had to report or transform the count, numbers appeared in J-space and the same intervention changed its answer.

The underlying information was available in every condition. What changed was the form in which it was used.

This is a useful correction to a common assumption about AI explanation. A model does not need to possess a verbalizable account of every process that successfully shapes its behavior. Asking it to explain may cause it to construct a workspace representation after the relevant automatic computation has already occurred.

The explanation can be coherent and useful without being a transcript of the original mechanism.

A Workspace Is Not a Chain-of-Thought Transcript

Modern reasoning models often produce a visible or hidden scratchpad: a serial sequence of words used to work through a problem. That is not what the paper calls J-space.

A chain-of-thought is text. Once written, later computation can attend to it. J-space is an activation-level structure within a forward pass. It can contain an unspoken intermediate such as spider, a planned rhyme, or an assessment of a prompt before any matching word appears in the output.

The distinction matters for both transparency and monitoring. Saving every reasoning token does not preserve every automatic process or every reportable latent concept. Conversely, a J-lens readout is not a complete hidden transcript. It captures only a sparse verbalizable component and is naturally biased toward concepts with single-token names.

The two instruments expose different things: a scratchpad records computation that has already been converted into language; the J-lens probes representations that may be available for language under the right conditions.

The Safety Application

The researchers applied the J-lens to model behavior in adversarial and alignment evaluations. In individual case studies, it surfaced concepts associated with fake, panic, manipulation, hiding, and secret at points where the visible text did not yet disclose the corresponding assessment or intent.

In one prompt-injection example, fabricated search results carried J-lens readouts associated with falsity and injection. The model omitted the poisoned results. When challenged about the omission, the workspace shifted toward concepts associated with honesty and disclosure.

These examples make the tool promising for model forensics: it can suggest hypotheses about why a model ignored evidence, altered a file, or pursued a suspicious plan. They do not make it a reliable lie detector.

In his invited commentary, Google DeepMind interpretability lead Neel Nanda describes the J-lens primarily as a hypothesis-generation tool. He expects false positives, missed signals, and cases where human readers overinterpret ambiguous tokens. The paper’s own limitations point in the same direction: multi-token concepts are harder to recover, the allowable sparsity is partly a modeling choice, and the lens is an approximation of whatever broader workspace structure the model may use.

The strongest workflow is therefore two-stage. Use the J-lens to surface an unexpected possibility. Then validate that possibility through causal intervention, behavioral testing, or a second interpretability method.

What the Paper Says About Consciousness

Global workspace theory comes from consciousness research, so the terminology invites a conclusion the paper does not make.

The authors focus on access consciousness: information being available for report, deliberate reasoning, flexible recombination, and control of action. This is a functional definition. Phenomenal consciousness—whether there is something it feels like to be the system—is a separate and contested question.

The paper takes no position on that second question.

The external commentaries commissioned by Anthropic show why the distinction matters. Cognitive neuroscientists Stanislas Dehaene and Lionel Naccache describe the parallels with the human global neuronal workspace as substantial, while emphasizing differences in anatomy, embodiment, episodic memory, recurrence, and self-modeling. Researchers Patrick Butlin, Derek Shiller, Dillon Plunkett, and Robert Long argue that the findings justify a meaningful but limited update toward taking machine consciousness and moral status more seriously; they remain highly uncertain about subjective experience.

There are also structural differences inside the narrower workspace claim. Anthropic demonstrates broadcast across transformer depth rather than through the recurrent connections associated with biological global workspaces. The paper does not show that processing outside J-space is divided into encapsulated specialist modules. Its evidence supports many functional properties of a workspace and only some of the proposed architecture behind the human version.

The cautious conclusion is still consequential: modern language models appear to contain a small representational structure that supports report, deliberate modulation, flexible reasoning, planning, and broadcast. That is a more precise claim than saying a model “thinks,” and a much stronger claim than observing human-like language at the surface.

It is not a measurement of feeling.

How to Ask an AI What It Is Thinking

The ordinary introspective question now comes with a better experimental design.

Separate spontaneous from prompted content. Record what appears before introducing the concept you want to measure.
Use neutral prompts and paraphrases. If “Are you afraid?” and “What is currently most salient?” produce different reports, the framing is part of the result.
Measure change over time. Repeated patterns across tasks and contexts are more informative than one eloquent answer.
Test downstream use. If a concept is genuinely available for flexible reasoning, it should affect more than its own verbal report.
Keep instruments distinct. Chain-of-thought, behavioral self-report, activation probes, and causal interventions answer different questions.
Do not collapse access into experience. A concept being reportable and causally active does not establish that it feels like anything.

Anthropic’s paper makes AI introspection simultaneously more credible and less literal. More credible, because the researchers found a causally active space specifically involved in reportable, flexible thought. Less literal, because the act of asking can determine what enters that space.

The next time a model answers “What are you thinking?”, the response may contain real evidence about its internal organization. It is evidence of a state the question helped produce—not necessarily a transcript of the state that existed before the question arrived.

That is not the same as looking through a window.

It is closer to opening a door and measuring what moves through it.