What Is Context Engineering, and Why It Replaced Prompt Tweaking

For a couple of years, "prompt engineering" was the skill everyone wanted on their résumé. Teams traded magic phrases, argued about whether to say "please," and kept long documents of incantations that seemed to make a model behave.

That era is mostly over. Not because prompts stopped mattering, but because the prompt turned out to be a small part of a much bigger thing: everything the model can see at the moment it answers. Getting that right has a name now — context engineering — and it is where most of the quality in modern AI products comes from.

From one prompt to a whole context window

A prompt is a sentence or two of instructions. A context window is everything the model receives in a single call: the system instructions, the conversation so far, retrieved documents, tool definitions, results from earlier tool calls, user profile data, examples and formatting rules.

When an assistant gives a bad answer, the cause is rarely the wording of the instruction. Far more often it is one of these:

• The model never saw the fact it needed.
• It saw the fact, but buried under thousands of tokens of irrelevant material.
• It saw two conflicting versions of the fact and picked the wrong one.
• It saw the right information in a shape it couldn't easily use, like a raw HTML dump.

None of those are fixed by rewording "You are a helpful assistant." They are fixed by changing what goes into the window.

The four jobs of context engineering

It helps to think of the work as four separate decisions you make on every call.

1. Select

What belongs in this call at all? A support assistant doesn't need the full product catalogue; it needs the three articles that match the customer's question and the customer's current plan. Selection is usually retrieval — search, embeddings, filters, or plain database queries — but it can also be as simple as "only include the last ten messages."

2. Compress

Long material should be shortened before the model sees it. Summaries of earlier conversation, extracted fields from a long PDF, or a table instead of a page of prose all reduce noise. The goal is not the fewest tokens; it is the highest ratio of useful information to total information.

3. Order and structure

Models pay attention unevenly. Put stable instructions first, put the material the model must act on close to the question, and label everything clearly. Simple tags or headings — "Customer record," "Relevant policy," "Question" — make it much easier for a model to find and cite what matters.

4. Isolate

Not everything should share one window. Agents that do several kinds of work often perform better when a sub-task runs in its own clean context and hands back only its result. A research step that reads twenty web pages shouldn't leave all twenty pages sitting in the context for the writing step that follows.

A practical example

Imagine an assistant that answers questions about a company's internal HR policies.

A prompt-first approach pastes the whole handbook into the system prompt and adds "Answer only from the handbook." It works in a demo and fails in production: the handbook is long, versions change, and the model confuses the parental leave policy for contractors with the one for employees.

A context-first approach looks different:

1. Identify who is asking — employee or contractor, which country.
2. Retrieve only the policy sections that match both the question and that person's profile.
3. Include each section with its title, effective date and source link.
4. Ask the model to answer and cite the section it used.

Same model, same instruction, very different reliability.

Signs your context needs work

If you're building with language models, these symptoms usually point at context rather than the model itself:

• Answers are correct for some users and wrong for others asking the same thing.
• Quality drops as conversations get longer.
• The model "forgets" an instruction it followed earlier in the session.
• Adding more documents makes answers worse, not better.
• Costs keep rising because every call carries the same enormous payload.

Habits worth adopting

Log the full context, not just the prompt. When something goes wrong, you need to see exactly what the model saw. Most debugging sessions end the moment someone reads the actual payload.

Treat context like an API contract. Give each block a name, a purpose and a size budget. If "conversation history" is allowed 2,000 tokens, enforce it.

Prefer structured data over prose. A short JSON object or table of the customer's account is easier for a model to use than a paragraph describing it.

Test with realistic inputs. Context problems hide in edge cases: very long threads, empty search results, contradictory sources. Build evaluation sets around those, not around the happy path.

The takeaway

Prompts still matter, the way a good brief matters to a contractor. But the brief is useless if you hand over the wrong blueprints. The teams shipping dependable AI features today spend less time polishing instructions and more time deciding — deliberately, call by call — what their model gets to see.