From the moment generative AI made headlines, prompt engineering was the most hyped skill to generate user-intended outputs from LLMs. Prominent leaders from the AI and technology space promoted prompt engineering as superior to traditional coding. Prompt engineering focuses on making the user’s intent clear. It involves efficient framing of user inputs: assigning roles to the LLM, structuring instructions, providing examples, and defining the output format to extract the desired outcome.

Over the past years in the space of Generative AI, LLMs have grown more capable than before to a point where prompt engineering alone is not sufficient to extract the desired outcome. Though LLMs can work with huge chunks of information for a longer period of time, they still have limitations that require context engineering to efficiently generate high-quality outputs. Let’s deep dive on what context engineering is and why it matters for agentic coding.

Foundational Concepts: Tokens and Context Windows

Before moving to context engineering, it’s necessary to understand the following concepts associated with LLMs.

• Tokens: Tokens are the basic units of information available to an LLM at any time including system prompts, user prompts, files, message history, MCP, user-generated output, etc.
• Context Window: Every LLM has a limited context window—i.e., the maximum number of tokens it can process in a single request.

A simple analogy to illustrate the above would be a bottle of water. LLMs vary in context window size as bottles do. Their capacity is limited and can contain only a limited amount of tokens. As bottles cannot hold water beyond their capacity, LLMs cannot function if accumulated tokens exceed the context window.

Understanding tokens, token accumulation, and the limited context windows of LLMs are foundational for grasping context engineering.

Why Context Engineering Matters?

Based on research done across various types of LLMs, a common observation among their behaviour is “context rot”. Context rot is the deterioration in output quality as context space usage increases due to token accumulation. Quality declines because LLMs have limited attention focus.

LLMs’ limited attention focus is similar to the human mind. With less information, the human brain can efficiently focus on the intended task. But with huge chunks of information (consider reading an email versus a 100-page PDF document), it is difficult to recall pieces of information, make connections, and come up with a logical conclusion. Context rot may cause a model to hallucinate because it loses access to relevant information buried in bloated context.

Above limitations can be better managed with context engineering. Context engineering is handpicking the minimal set of information (or tokens) required for the agents to get the desired outcome aligned with user expectations. By curating only relevant information—neither vague nor bloated—the agent can produce quality output. Let’s move on to further understand how context engineering can help with agentic coding.

What Do Coding Assistants Load Into Context by Default?

In terms of agentic coding with coding assistants such as GitHub Copilot or Claude Code, the agent preloads files from the code repository such as copilot-instructions.md and Claude.md into the context window. Apart from the instruction files, MCP tool definitions are also loaded into the context. GitHub Copilot goes a step further to load high-level project context such as project metadata and folder structure.

The reason behind adding instruction files is to guide the agent in the right direction aligned with the project’s architecture decisions, coding standards, and rules. As it’s getting loaded by default, it’s crucial to carefully design the content inside the instruction files. The instructions should be concise and to the point to get quality output at reduced token usage.

Why Default Context Isn’t Enough?

The instruction file is supposed to contain important information about the project such as coding standards and quality practices. But it doesn’t have detailed information necessary for every task that may arise. Tasks of different natures may require information from external sources such as files, MCP tools, and the web. Hence, it is important to curate the right context for the agent to get quality output.

Strategies for Efficient Agentic Coding

1. Context Consciousness

Context consciousness: Being aware of whether the content loaded in context is relevant to the task at hand. This ensures the agent sees only the high-signal information which is valuable for generating quality output. A common mistake is using the same context window for two or more unrelated tasks.

In this case, subsequent tasks carry the burden of previous context accumulation, which dilutes the agent’s attention focus, resulting in subpar output. Clear the session or start a new one if the new task is unrelated to the previous task.

2. Explicitness

Explicitness: Stating known information to the agent upfront, such as which tools to use and which files to edit. This helps the agent by pointing it in the right direction, avoiding trial and error and reaching the expected outcome sooner without repeated inferences.

Few examples:

• Requesting or attaching the GitHub MCP server for reviewing a PR
• Tagging the .gitignore file to ignore build output in git changes

3. Efficiently Handling Long-Running Tasks

The following strategies can help effectively manage context for longer tasks:

Context Compaction: Coding agents provide options to compact the conversation history. As accumulated context may include tool call responses, user-generated output, and message history that may not be relevant for subsequent steps, compacting discards excessive verbosity while retaining a condensed summary of important decisions and open items.

Execute a Phased Plan: If a task requires multiple steps and validations (for example, a codebase migration to a newer version), create a multi-phased plan. Write the plan into a plan.md file in the repository. Also create progress.md and memory.md files to track progress and record important decisions. With this approach, the plan can be executed across multiple context windows in a phased manner.

Implications of Poorly Managed Context

Improper context management can result in:

• Quality degradation
• Hallucination
• Token consumption, which in turn results in cost explosion

Conclusion

As enterprises are rapidly adopting AI for coding and building applications, coding assistants have become essential for all engineers. With increased adoption of agentic coding, the following concerns have become prevalent:

• Low-quality code
• High cost

Context engineering can help deliver high-quality code at optimized cost. Therefore, developers must master context engineering for scalable agentic coding adoption across enterprises.