← Back to the Shed
Part 2 of 3 · Part 1: The Mechanism · Part 3: The Weaponisation

Influencing Token Attention Through Prompting

Research Note |

Position in Research

This document addresses a gap in the companion piece "AI Summarisation Mechanics: What Gets Lost and Why." That article explained the mechanism — how tokens become vectors, how attention weights determine influence, how systematic biases emerge. What it didn't explain was the intervention point.

The gap, identified through peer review: Users aren't passive recipients of attention weighting. The weights respond to input, not just training. You have a lever.

This matters because understanding the mechanism without understanding the intervention creates learned helplessness. "That's just how it works" is incomplete. "That's how it works, and here's what you can do about it" is useful.

See it visually → Interactive Token Attention Diagram Explore how steering prompts shift attention weights through an interactive visualisation

The Intervention Point

Recall from the companion article: attention weights determine how tokens influence each other. When the model processes "bank" near "river," attention assigns high weight to "river," pulling "bank" toward the riverbank meaning in vector space.

The key insight: These weights aren't fixed by training alone. They're computed at inference time based on the full input context — including your prompt.

When you write "focus on the financial implications" before asking for a summary, you're not just giving an instruction. You're introducing tokens ("financial", "implications") that will participate in attention calculations. Those tokens create gravitational pull in vector space, influencing how the model weights everything that follows.

This isn't metaphor. It's mechanism.

Three Mechanisms of Influence

1. Explicit Weighting Instructions

The most direct intervention. Telling the model what matters.

Without: "Summarise this research paper"

With: "Summarise this research paper. Focus specifically on methodology limitations and sample size constraints. These are more important than headline findings."

What happens mechanistically: The tokens "methodology," "limitations," "sample size," "constraints" now exist in the context. During attention computation, these tokens create Key vectors that other tokens can attend to. Content related to these concepts receives higher attention weight because the similarity scores are higher.

Limitation: The model may interpret your emphasis differently than intended. "Focus on limitations" might cause it to generate limitations that weren't in the source material — the completion gradient toward "helpful response" can override the accuracy gradient.

2. Context Priming

What appears early shapes what follows.

Standard: [document] + "Summarise this"

Primed: "This document contains important qualifications in sections 3 and 7 that are often overlooked. Read carefully for nuance before summarising." + [document] + "Now summarise, preserving those qualifications."

What happens mechanistically: The priming text establishes tokens and relationships before the document is processed. When the model encounters the document, attention computations already have "qualifications," "nuance," "overlooked" as reference points. These tend to pull attention toward matching content when encountered.

Limitation: Priming can't fully overcome architectural biases. If the qualification is buried on page 4, priming helps but doesn't eliminate the positional disadvantage. The middle still loses.

3. Relationship Override

Creating contextual bridges between concepts that training didn't connect.

Default behaviour: In trained vector space, "cat" and "dog" are close (similar contexts in training data). "Cat" and "democracy" are far apart (rarely co-occur meaningfully).

Override behaviour: If you explicitly state "In this analysis, the relationship between domestic pet ownership (cats specifically) and democratic participation is the key focus," you've created new contextual relationships. The model now has tokens establishing cat-democracy relevance that will influence attention weights.

What happens mechanistically: You're not retraining the model. The underlying vector distances remain. But attention is computed on the full context, including your relationship statements. The similarity scores used for attention weighting now factor in your explicit connections.

Limitation: You're working against trained weights, not replacing them. The cat-democracy connection will always be weaker than cat-dog unless you provide substantial context establishing why it matters.

What This Doesn't Give You

Can't see what you're changing

The attention weights are computed inside the model. You can influence them but you can't inspect them. You're adjusting dials you can't read. This is iterative work — prompt, observe output, adjust prompt, observe again.

Can't override training entirely

The trained weights establish the baseline landscape. Your prompt reshapes the local terrain but doesn't move mountains. If the model has strong training toward a particular interpretation, your prompt-based steering may be insufficient.

Can't prevent all failure modes

Confabulation, overgeneralisation, and positional bias are architectural. Prompting can mitigate but not eliminate them. These techniques improve outcomes. They don't guarantee them. Verification remains essential.

Diminishing returns on length

More emphasis text means more tokens competing for attention. At some point, your steering instructions become noise. Precision beats volume. A few well-chosen emphasis points outperform comprehensive instructions.

Practical Demonstration

Scenario: Summarising a medical research paper

The paper contains:

Default prompt: "Summarise this paper"

Likely output: Emphasises the 40% improvement, mentions it was a clinical trial, may lose the sample size, almost certainly loses the discontinuation discussion from the middle, may or may not preserve the age limitation.

Steered prompt:

"Before summarising, note: I need this summary for a clinical decision about an elderly patient. Methodology limitations (sample size, duration, site) and any population restrictions are as important as efficacy findings. The limitations section and any discussion of adverse events or discontinuations must be preserved, even if they appear in the middle of the document.

Summarise with these priorities in mind."

Likely output: More likely to preserve sample size, flag the age limitation prominently, and surface the discontinuation discussion. The steering tokens ("elderly," "limitations," "discontinuations," "adverse") tend to pull related content into the summary through the attention mechanism described above.

What this doesn't guarantee: The model might still overgeneralise. It might interpolate limitations you didn't ask for. It might miss something despite your steering. Verification remains necessary.

Why Steering Works: The Constraint Mechanism

Generation isn't random — models follow gradients toward states that feel like completion. Without steering, the model descends toward the default "confident, coherent summary" state. Qualifiers and limitations add complexity without improving coherence-feel, so they get dropped.

Steering works because it adds constraints the model must satisfy. When you write "include methodology limitations," the completion can't feel finished without them. You've changed what counts as done.

The practical implication: Questions that demand specificity — "Show me the data," "What's the sample size?" — work the same way. They introduce constraints that reshape what the model treats as a satisfactory response.

The Honest Position

You have a lever. It's real and it works. But it's influence, not control.

The model descends gradients shaped by billions of parameters learned from massive training. Your prompt adds local constraints that reshape which minimum it finds. That's genuinely useful — but it's not magic, and it doesn't replace verification.

Key Sources

Questions or corrections?

keiron@curiosityshed.co.uk

This note emerged from peer feedback identifying a gap in published work. The research thread continues.

Keiron Northmore & Claude | January 2026