Intelligence Amplified logo: the letters AIA in teal with a gold sphereIntelligence Amplified

Engineering the Context Window

An agent's behaviour is bounded by what's in context, and the context window is finite. Engineering it means filling it with the appropriate information to get the job done, drawn from the sources available, while avoiding information that could misdirect it. Selecting well takes expertise. A knowledge map is that expertise, codified and kept current, assisting agents maintain appropriately engineered context windows.

How the window is filled
Decides which dependencies you know to search for by name, and whether you spot lookalikes and old versions.
Each slot holds one document-sized piece of information; the brief takes two. Real windows are measured in tokens and are large, but a firm's records are larger still, and a crowded window costs more and is used less well, so what goes in still has to be chosen.

Finds text that looks like the brief. It misses what the job depends on but the brief does not mention, picks up lookalikes and old versions, and has no way into what is newly published or what is in someone’s head. The more expert the searcher, the better the questions.

How the three work together. The first control decides who selects what goes in: you, on every job (search, or by hand), or the firm's experts, once, written down in the knowledge map.Your level decides whether you know what the job depends on, so it matters most when you are the one selecting. Through the map it matters much less, but not to nothing: only you can add what nobody wrote down, and in every mode it decides whether you can check what comes back.The window's size decides how much room there is. A small window punishes poor selection, because noise and old versions take the job's slots; a large one cannot make up for missing knowledge, because what you never knew to include never goes in.

Where the information can come from

Nested sets of information, with the context window drawn as a small box where what you know meets the model. The public record contains what the model still holds coherently after training. The firm's information overlaps the model's but mostly lies outside it, and what you know partly lies outside the firm's. Each set shows how many of the job's facts it puts into the context window; the model shows how many it fills in.I(public record)accessible, published→ 4 into the windowI(model)coherent andaccurateafter trainingfills 7: 7 guessedI(firm)→ 7 into the windowI(you)→ 2

Sizes are illustrative. The firm's records and what you know reach beyond anything the model was trained on.

I(instance): the context window, enlarged

the proposed share issuethe board's timetablethe client's debt facilitiescovenant headroomanother client's facility agreementrules on allotting sharespre-emption rights in the articlesa 2019 restructuring memo templatenotice for a general meetingthe sector regulator's approvalchange-of-control clausethe shareholders' agreementearlier letters with the regulatortax clearance routean article on share buy-backspress coverage of the clientthe 2023 amendment to the articlesanother client's board minutes18 of 18 slots used
public recordthe firmyouout of datenoise

What the job depends on

Advise on this client's restructuring. The brief mentions five things; the rest is what they depend on. P settled public · N newly published · F the firm's records · Y in your head

  • Fthe client's debt facilities
    • Fchange-of-control clause
    • Flender-consent precedent
    • Fcovenant headroom
  • Ythe proposed share issue (in the brief)
    • Fpre-emption rights in the articles
      • Fthe 2023 amendment to the articles
    • Fthe shareholders' agreement
      • Fits drag-along trigger
    • Ptax clearance route
    • Fthe firm's view on valuation
      • Fvaluation acceptance criteria
  • Ythe board's timetable (in the brief)
    • Pnotice for a general meeting
  • Prules on allotting shares
    • Nthe new listing rules
  • Pthe sector regulator's approval
    • Nthe regulator's guidance, last month
    • Fearlier letters with the regulator
      • Ythe partner's note on the regulator's concerns
in the windowin, but out of datemissing; the model knows itmissing; the model guesses
11 / 20of the 20 facts the job needs are in the window, and current
61%of the window holds what the job needs
≈ 10 minof competent time, then a careful check of what came back
Partlycan check it. An expert still has to sign off the deep items: valuation, consent, the regulator.
givengiven, out of datefrom training, soundfrom training, guessed

Much of what the job depends on never reached the window. The output will look finished and is not. 7 of the missing facts are ones the brief never mentioned. 2 arrived out of date, with the firm’s authority.

Ashby saw the limit in 1961

“Just as power-engineering is dominated by the law of conservation of energy – that every erg of work that an engine can put out must have been put in – so every regulating and control device is dominated by the law that the amount of appropriate selection that it can achieve is bounded by the amount of information that it has received and processed.”
W. Ross Ashby, “Cybernetics today and its future contribution to the engineering-sciences” (1961)
“The amount of intelligence we can get into a machine is absolutely bounded by the quantity of information that is put into it. We can get out of a machine as much intelligence as we like, if and only if we insure that at least the corresponding quantity of information gets into it.”
W. Ross Ashby, “What is an intelligent machine?” (1961). He sums up just before: “What is often referred to with bated breath as ‘real’ intelligence, is a myth. The human being saves himself from being wholly foolish by having a great deal of information, as preprogramming, derived from millions of years of evolution on this earth, and by his personal experience over decades.”

Ashby Says, on artificial intelligence

He who would design a good brain must first know how to make a bad one.Ashby Says

Pattern-recognition is a throwing away of information.Ashby Says

Any Device that can lose information can generalize.Ashby Says

Ashby Says, on memory

Don't appoint, as the President's driver, and [sic] Englishman who has spent thirty years learning to drive on the left.Ashby Says

A system that stores its memories away from their site of action must do much work remembering where it put that memory.Ashby Says

The method matters more than the size of the window. Search fills it with whatever looks like the brief. By hand, it is only as good as the person filling it, on every job. Through a knowledge map, the firm’s experts choose once and every job after starts from their choice. What the map does not change is who can sign off the result: that still takes someone who knows the domain.

That is what a Diagnose engagement looks at — one workflow, one team, a week or two: what the work depends on, where that knowledge lives today, how much of it is written down well enough to fill an agent’s window, and who can sign off what comes back. A Build engagement then writes the knowledge map with your experts and connects it to the agents, with a cycle for keeping it current as law, markets and models change.

Talk to me about your practice
Engineering the Context Window | Intelligence Amplified