This guide is for the moment before that debate takes over. It covers the major patterns used in production agentic systems, when each one earns its place, and the trade-offs involved. The examples use class-based Python — code you would actually review in a pull request, not demo-grade snippets.

We will cover:

1. Pattern 1: Single-Agent (Month 1)

2. Pattern 2: Multi-Agent Sequential (Month 3)

3. Pattern 3: Multi-Agent Parallel (Month 5)

4. Pattern 4: Loop (Month 6)

5. Pattern 5: Review and Critique

6. Pattern 6: Iterative Refinement

7. Pattern 7: Coordinator (Month 9)

8. Pattern 8: Hierarchical Task Decomposition (Month 11)

9. Pattern 9: Swarm (Month 12)

10. Pattern 10: ReAct (Reason and Act) (Month 13)

11. Pattern 11: Human-in-the-Loop (Month 14)

12. Pattern 12: Plan-and-Execute (Month 15)

13. Pattern 13: Reflexion (Month 16)

14. Pattern 14: Custom Logic (Month 18)

15. Pattern 15: Event-Driven Agent

We will follow a single company: an e-commerce platform called Vend. Their AI platform team starts with one support agent and, over eighteen months, evolves into a system spanning multiple patterns. Each shift happened for a reason. Understanding the reason behind the decision matters more than memorizing the pattern itself.

Before You Pick a Pattern

A task justifies an agent when at least one of these is true:

• A single model call cannot produce a reliable result.

• The model must choose between tools or data sources at runtime.

• The task requires planning, validation, or iterative refinement.

• The workflow contains uncertainty that cannot be expressed through static business logic.

A task usually does not justify an agent when the input-to-output path is predictable.

Tasks like summarization, classification, templated generation, and simple extraction are often faster, cheaper, and more reliable as direct model calls.

Wrapping them in an agent typically adds latency, failure points, and unnecessary complexity without delivering meaningful value.

Shared Infrastructure

All examples use the following base structure. In production you would add retries, request IDs, structured logging, rate-limit handling, budget tracking, policy checks, and durable state.

import json
import os
import asyncio
from dataclasses import dataclass, field
from typing import Any
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
MODEL = "gpt-4o"

@dataclass
class AgentResult:
    content: str
    tool_calls: list[dict] = field(default_factory=list)
    metadata: dict = field(default_factory=dict)

class BaseLLMAgent:
    def __init__(self, model: str = MODEL):
        self.client = client
        self.model = model

    def _text(self, system: str, user: str) -> str:
        response = self.client.responses.create(
            model=self.model,
            input=[
                {"role": "system", "content": system},
                {"role": "user", "content": user},
            ],
        )
        return response.output_text

    def _structured(self, system: str, user: str, schema: dict[str, Any]) -> dict[str, Any]:
        response = self.client.responses.create(
            model=self.model,
            input=[
                {"role": "system", "content": system},
                {"role": "user", "content": user},
            ],
            text={"format": {"type": "json_schema", "name": "output", "schema": schema, "strict": True}},
        )
        return json.loads(response.output_text)

Pattern 1: Single-Agent [Month 1]

The AI team gets its first task: let customers ask questions about their orders without waiting for a human agent. The workflow is straightforward — look up an order, check shipping status, create a support ticket if needed. Three tools, one clear responsibility

This is where most agentic systems should begin: one model, one system prompt, and a bounded set of tools. The model decides which tool to invoke, observes the result, and continues until it has enough context to respond. Simpler systems are easier to reason about, debug, and improve.

import json
from openai import OpenAI
from dataclasses import dataclass

@dataclass
class OrderContext:
    order_id: str
    status: str
    carrier: str
    eta: str

@dataclass
class SupportTicket:
    ticket_id: str
    order_id: str
    reason: str

class CustomerSupportAgent(BaseLLMAgent):
    """Single-agent that handles order support by choosing its own tools."""

    SYSTEM_PROMPT = (
        "You are a customer support agent. Use tools when order data is needed. "
        "Do not invent order status. If the issue cannot be resolved from tools, create a ticket."
    )

    TOOLS = [
        {
            "type": "function",
            "name": "get_order",
            "description": "Fetch order status by order ID.",
            "parameters": {
                "type": "object",
                "properties": {"order_id": {"type": "string"}},
                "required": ["order_id"],
                "additionalProperties": False,
            },
        },
        {
            "type": "function",
            "name": "create_support_ticket",
            "description": "Create a support ticket when the issue cannot be answered from order data.",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {"type": "string"},
                    "reason": {"type": "string"},
                },
                "required": ["order_id", "reason"],
                "additionalProperties": False,
            },
        },
    ]

    def _get_order(self, order_id: str) -> dict:
        # Replace with real DB/API call
        return {"order_id": order_id, "status": "shipped", "carrier": "DHL", "eta": "2026-06-01"}

    def _create_support_ticket(self, order_id: str, reason: str) -> dict:
        return {"ticket_id": "SUP-1042", "order_id": order_id, "reason": reason}

    def _dispatch_tool(self, name: str, args: dict) -> dict:
        if name == "get_order":
            return self._get_order(**args)
        if name == "create_support_ticket":
            return self._create_support_ticket(**args)
        return {"error": f"Unknown tool: {name}"}

    def run(self, user_message: str, max_turns: int = 4) -> str:
        messages = [
            {"role": "system", "content": self.SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ]

        for _ in range(max_turns):
            response = self.client.responses.create(
                model=self.model, input=messages, tools=self.TOOLS
            )
            tool_calls = [item for item in response.output if item.type == "function_call"]

            if not tool_calls:
                return response.output_text

            messages += response.output
            for call in tool_calls:
                result = self._dispatch_tool(call.name, json.loads(call.arguments))
                messages.append({
                    "type": "function_call_output",
                    "call_id": call.call_id,
                    "output": json.dumps(result),
                })

        return "I could not complete the request safely within the tool budget."

When to Use and When it Breaks:

Pattern 2: Multi-Agent Sequential [Month 3]

Legal operations receives vendor contracts, and the review process is well-defined: extract obligations, identify risk clauses, and draft a summary for the procurement manager. The sequence never changes, and each step produces exactly what the next step requires.

The sequential pattern runs specialized agents in a fixed order. Each agent has a narrow prompt and a clear input/output contract. Orchestration is deterministic — reasoning happens within each stage, not in the routing between agents, making the process predictable and auditable.

from dataclasses import dataclass

@dataclass
class ContractAnalysis:
    obligations: str
    risks: str
    review_packet: str

class ContractIntakePipeline(BaseLLMAgent):
    """Sequential pipeline: extract → assess → draft."""

    def _extract_obligations(self, contract_text: str) -> str:
        return self._text(
            "Extract concrete obligations from vendor contracts. "
            "Return only obligations with dates, parties, or deliverables.",
            contract_text,
        )

    def _assess_risks(self, obligations: str) -> str:
        return self._text(
            "You are a contract risk reviewer. Identify operational, financial, and compliance risks. "
            "Do not provide legal advice.",
            obligations,
        )

    def _draft_review_packet(self, contract_text: str, obligations: str, risks: str) -> str:
        return self._text(
            "Prepare a concise contract review packet for a procurement manager. "
            "Include obligations, risks, and questions for counsel.",
            f"Contract:\\\\n{contract_text}\\\\n\\\\nObligations:\\\\n{obligations}\\\\n\\\\nRisks:\\\\n{risks}",
        )

    def run(self, contract_text: str) -> ContractAnalysis:
        obligations = self._extract_obligations(contract_text)
        risks = self._assess_risks(obligations)
        packet = self._draft_review_packet(contract_text, obligations, risks)
        return ContractAnalysis(obligations=obligations, risks=risks, review_packet=packet)

When to Use and When it Breaks:

Pattern 3: Multi-Agent Parallel [Month 5]

The on-call team gets a midnight alert: checkout is failing. Three signals need immediate investigation — application logs, infrastructure metrics, and recent deployments. These are genuinely independent sources of information. Running them sequentially wastes the first few minutes that matter most during an incident.

The parallel pattern distributes independent subtasks across multiple agents simultaneously and combines their outputs into a single view. When tasks do not depend on one another, parallelism reduces latency and speeds up decision-making under pressure.

import asyncio
from dataclasses import dataclass

@dataclass
class IncidentReport:
    log_analysis: str
    metric_analysis: str
    deployment_analysis: str
    synthesis: str

class IncidentTriageAgent(BaseLLMAgent):
    """Parallelizes three specialist agents then synthesizes a root-cause hypothesis."""

    async def _ask_specialist(self, role_prompt: str, incident: str) -> str:
        return await asyncio.to_thread(self._text, role_prompt, incident)

    async def run(self, incident_description: str) -> IncidentReport:
        log_prompt = "You inspect application logs. Identify error patterns with timestamps and request IDs."
        metric_prompt = "You inspect metrics. Identify saturation, latency spikes, and traffic anomalies."
        deploy_prompt = "You inspect recent releases. Flag risky changes that align with the reported symptoms."

        log_view, metric_view, deploy_view = await asyncio.gather(
            self._ask_specialist(log_prompt, incident_description),
            self._ask_specialist(metric_prompt, incident_description),
            self._ask_specialist(deploy_prompt, incident_description),
        )

        synthesis = self._text(
            "You are the incident commander. Synthesize evidence from three specialist views. "
            "State confidence level and the single safest next action.",
            f"Logs:\\\\n{log_view}\\\\n\\\\nMetrics:\\\\n{metric_view}\\\\n\\\\nDeployments:\\\\n{deploy_view}",
        )

        return IncidentReport(
            log_analysis=log_view,
            metric_analysis=metric_view,
            deployment_analysis=deploy_view,
            synthesis=synthesis,
        )

When to Use and When it Breaks:

Pattern 4: Loop [Month 6]

The data engineering team ingests CSV extracts from third-party suppliers, where data quality is often inconsistent. They need an agent that can profile the data, propose a cleaning strategy, validate it against quality checks, and retry when needed.

The loop pattern repeats a sequence of agent steps until an exit condition is satisfied. The challenge is not the loop itself — it is defining a reliable stopping condition. Without one, you end up with runaway costs, unpredictable behavior, and no guarantee of termination.

import json
from dataclasses import dataclass

@dataclass
class CleaningResult:
    plan: str
    rounds_taken: int
    passed: bool

class DataQualityRepairAgent(BaseLLMAgent):
    """Iteratively proposes and judges cleaning plans until quality passes or budget is exhausted."""

    QUALITY_SCHEMA = {
        "type": "object",
        "properties": {
            "passes": {"type": "boolean"},
            "reason": {"type": "string"},
            "remaining_issues": {"type": "array", "items": {"type": "string"}},
        },
        "required": ["passes", "reason", "remaining_issues"],
        "additionalProperties": False,
    }

    def _profile(self, sample_rows: str) -> str:
        return self._text(
            "Profile tabular data. Identify missing values, inconsistent formats, "
            "suspicious categories, and likely schema issues.",
            sample_rows,
        )

    def _propose_plan(self, profile: str, previous_plan: str | None) -> str:
        return self._text(
            "Propose a safe pandas cleaning plan. Prefer reversible transformations. "
            "Do not drop columns unless explicitly justified.",
            f"Profile:\\\\n{profile}\\\\n\\\\nPrevious attempt:\\\\n{previous_plan or 'None'}",
        )

    def _judge(self, profile: str, plan: str) -> dict:
        return self._structured(
            "Judge whether the cleaning plan addresses the data profile without unsafe assumptions.",
            f"Profile:\\\\n{profile}\\\\n\\\\nPlan:\\\\n{plan}",
            self.QUALITY_SCHEMA,
        )

    def run(self, sample_rows: str, max_rounds: int = 3) -> CleaningResult:
        profile = self._profile(sample_rows)
        plan = None

        for round_num in range(1, max_rounds + 1):
            plan = self._propose_plan(profile, plan)
            verdict = self._judge(profile, plan)

            if verdict["passes"]:
                return CleaningResult(plan=plan, rounds_taken=round_num, passed=True)

            # Append reviewer feedback to the profile so the next proposal is informed
            profile += f"\\\\nReviewer feedback (round {round_num}):\\\\n{json.dumps(verdict)}"

        return CleaningResult(plan=plan, rounds_taken=max_rounds, passed=False)

When to Use and When it Breaks:

Note: This is a simplified example of using LLM-as-a-Judge to review and approve another model’s response.

In this example, the flow uses a loop with a boolean approval flag. In practice, there are many variations. Instead of a binary decision, the judge can return a score, or provide structured feedback that the generator uses to improve the next iteration. The design space here is broad, so I am intentionally keeping the implementation lightweight.

✅ Pattern 5: Review and Critique → A judge agent reviews the output, critiques it, and provides actionable feedback on what should improve.

✅ Pattern 6: Iterative Refinement → The system runs in a feedback loop with a score threshold. Until the response crosses the required quality bar, the generator continues refining the output.

Pattern 7: Coordinator [Month 9]

Support has expanded into five distinct departments: billing, technical support, account management, shipping, and fraud. A single agent trying to handle all of them ends up relying on a 3,000-word system prompt, eventually struggling with edge cases and conflicting instructions. At that point, the issue is no longer prompt quality — different request types genuinely require different context, tools, and decision-making logic.

The coordinator pattern introduces a central routing agent that directs incoming requests to specialized agents. Unlike sequential or parallel workflows, the execution path is determined dynamically at runtime, based on the nature of the request. This keeps individual agents focused, reduces prompt complexity, and improves reliability as the system grows.

from dataclasses import dataclass

@dataclass
class RoutingDecision:
    specialist: str
    reason: str

class SupportCoordinator(BaseLLMAgent):
    """Routes support messages to the right specialist agent."""

    ROUTING_SCHEMA = {
        "type": "object",
        "properties": {
            "specialist": {
                "type": "string",
                "enum": ["billing", "technical", "account", "shipping", "fraud"],
            },
            "reason": {"type": "string"},
        },
        "required": ["specialist", "reason"],
        "additionalProperties": False,
    }

    SPECIALISTS: dict[str, str] = {
        "billing": "You handle billing support. Request invoice IDs when needed. Never quote prices from memory.",
        "technical": "You handle technical troubleshooting. Ask for logs and environment details when needed.",
        "account": "You handle account access issues. Never request passwords or raw secrets.",
        "shipping": "You handle delivery and tracking questions. Escalate customs issues to a human agent.",
        "fraud": "You handle suspected fraud reports. Collect evidence carefully. Never accuse the customer.",
    }

    def _route(self, message: str) -> RoutingDecision:
        result = self._structured(
            "Route this support request to exactly one specialist based on the primary issue.",
            message,
            self.ROUTING_SCHEMA,
        )
        return RoutingDecision(**result)

    def _run_specialist(self, specialist_prompt: str, message: str) -> str:
        return self._text(specialist_prompt, message)

    def run(self, message: str) -> str:
        decision = self._route(message)
        specialist_prompt = self.SPECIALISTS[decision.specialist]
        return self._run_specialist(specialist_prompt, message)

When to Use and When it Breaks:

Pattern 8: Hierarchical Task Decomposition [Month 11]

Strategy asks the AI platform team: “Which three countries should we expand into next year?” This is not a typical support task — it is a multi-dimensional research problem. No single agent can answer it reliably. The question needs to be broken down into competitive analysis, regulatory constraints, logistics feasibility, and market sizing, each requiring specialized context and reasoning.

The hierarchical decomposition pattern uses a root agent to break a complex objective into smaller, well-defined subgoals, which are then delegated to specialized worker agents. Once the work is complete, the root agent synthesizes the outputs into a coherent recommendation. This pattern works best when the problem is too broad for a single reasoning pass but can be decomposed into independent areas of expertise.

import json
from dataclasses import dataclass

@dataclass
class Workstream:
    name: str
    deliverable: str
    questions: list[str]

@dataclass
class ResearchBrief:
    goal: str
    workstreams: list[Workstream]
    reports: list[str]
    synthesis: str

class MarketExpansionResearchAgent(BaseLLMAgent):
    """Decomposes a research goal into workstreams, executes each, and synthesizes results."""

    PLAN_SCHEMA = {
        "type": "object",
        "properties": {
            "workstreams": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "name": {"type": "string"},
                        "deliverable": {"type": "string"},
                        "questions": {"type": "array", "items": {"type": "string"}},
                    },
                    "required": ["name", "deliverable", "questions"],
                    "additionalProperties": False,
                },
            }
        },
        "required": ["workstreams"],
        "additionalProperties": False,
    }

    def _plan(self, goal: str) -> list[Workstream]:
        result = self._structured(
            "Break a market expansion research goal into 3 to 5 independent workstreams. "
            "Each workstream must have a clear deliverable and 2-4 focused questions.",
            goal,
            self.PLAN_SCHEMA,
        )
        return [Workstream(**ws) for ws in result["workstreams"]]

    def _execute_workstream(self, ws: Workstream) -> str:
        return self._text(
            "You are a specialist research agent. Answer only the assigned workstream. "
            "List assumptions and evidence gaps separately. Be specific.",
            json.dumps({"name": ws.name, "deliverable": ws.deliverable, "questions": ws.questions}),
        )

    def _synthesize(self, goal: str, reports: list[str]) -> str:
        combined = "\\\\n\\\\n".join(
            f"--- Workstream {i+1} ---\\\\n{r}" for i, r in enumerate(reports)
        )
        return self._text(
            "Synthesize specialist research reports into an executive brief. "
            "Highlight conflicts between workstreams, unsupported assumptions, and missing evidence.",
            f"Goal: {goal}\\\\n\\\\nWorkstream reports:\\\\n{combined}",
        )

    def run(self, goal: str) -> ResearchBrief:
        workstreams = self._plan(goal)
        reports = [self._execute_workstream(ws) for ws in workstreams]
        synthesis = self._synthesize(goal, reports)
        return ResearchBrief(goal=goal, workstreams=workstreams, reports=reports, synthesis=synthesis)

When to Use and When it Breaks:

Pattern 9: Swarm [Month 12]

The product team faces a contentious question: should they launch a subscription tier? Research suggests customers want it. Engineering estimates six months of billing infrastructure work. Finance questions whether the unit economics make sense, while Support expects a spike in customer confusion and ticket volume. Every perspective is valid, but the team needs a way to surface the trade-offs before leadership makes a decision.

The swarm pattern allows multiple specialized agents to contribute to a shared working context. Each agent brings a different perspective, challenges assumptions, and adds supporting evidence. Once enough signal has accumulated, a facilitator agent synthesizes the discussion into a recommendation. This pattern works best when the goal is not a single “correct” answer, but a well-reasoned decision shaped by competing viewpoints.

from dataclasses import dataclass

@dataclass
class DebateResult:
    question: str
    transcript: str
    recommendation: str

class ProductDecisionSwarm(BaseLLMAgent):
    """Runs a structured debate across specialist perspectives before synthesizing a recommendation."""

    ROLES: dict[str, str] = {
        "research": "You represent user research. Cite evidence, customer segments, and unmet needs. Challenge unsupported claims.",
        "engineering": "You represent engineering. Assess feasibility, complexity, risks, and maintenance burden.",
        "finance": "You represent finance. Evaluate pricing, margin, cost, and opportunity cost.",
        "support": "You represent customer support. Assess operational burden, edge cases, and customer confusion risks.",
    }

    def run(self, question: str, rounds: int = 2) -> DebateResult:
        transcript = f"Decision question: {question}\\\\n"

        for round_no in range(1, rounds + 1):
            for name, role_prompt in self.ROLES.items():
                contribution = self._text(
                    role_prompt + " Respond to the current transcript. "
                    "Add new evidence or challenge a specific weak claim. Be concise and specific.",
                    transcript,
                )
                transcript += f"\\\\n[{name}, round {round_no}]\\\\n{contribution}\\\\n"

        recommendation = self._text(
            "You are a neutral decision facilitator. Review the full debate. "
            "Summarize the strongest arguments, identify unresolved risks, and state a final recommendation. "
            "Do not add new claims — only synthesize what is already in the transcript.",
            transcript,
        )

        return DebateResult(question=question, transcript=transcript, recommendation=recommendation)

When to Use and When it Breaks:

Pattern 10: ReAct (Reason and Act) [Month 13]

The DevOps team builds an internal troubleshooting assistant. When an engineer reports that “the queue processor seems stuck,” the assistant should search documentation, check service health, correlate symptoms, and only then suggest a fix. The investigation path cannot be predefined — it depends entirely on what the system uncovers along the way.

The ReAct pattern alternates between reasoning and action: deciding what to investigate, invoking a tool, observing the result, and determining whether there is enough evidence to proceed or answer. The most valuable output is not the hidden reasoning itself — it is the action trace, which makes the investigation observable, debuggable, and easier to trust in production.

import json
from dataclasses import dataclass

@dataclass
class TroubleshootingResult:
    answer: str
    action_trace: list[dict]

class KnowledgeBaseTroubleshootingAgent(BaseLLMAgent):
    """Iteratively calls tools to gather evidence before stating a root cause."""

    SYSTEM_PROMPT = (
        "You are a troubleshooting agent. Use tools to gather evidence. "
        "Return the likely cause, your confidence level, and the next safe action. "
        "Do not speculate beyond what the tools returned."
    )

    TOOLS = [
        {
            "type": "function",
            "name": "search_docs",
            "description": "Search internal troubleshooting documentation by keyword.",
            "parameters": {
                "type": "object",
                "properties": {"query": {"type": "string"}},
                "required": ["query"],
                "additionalProperties": False,
            },
        },
        {
            "type": "function",
            "name": "get_service_status",
            "description": "Get current operational status and recent incidents for a service.",
            "parameters": {
                "type": "object",
                "properties": {"service": {"type": "string"}},
                "required": ["service"],
                "additionalProperties": False,
            },
        },
    ]

    def _search_docs(self, query: str) -> dict:
        return {"matches": ["Restart workers after QUEUE_URL change", "Check IAM permission sqs:SendMessage"]}

    def _get_service_status(self, service: str) -> dict:
        return {"service": service, "status": "degraded", "since": "2026-05-21T09:20:00Z"}

    def _dispatch_tool(self, name: str, args: dict) -> dict:
        if name == "search_docs":
            return self._search_docs(**args)
        if name == "get_service_status":
            return self._get_service_status(**args)
        return {"error": f"Unknown tool: {name}"}

    def run(self, problem: str, max_turns: int = 5) -> TroubleshootingResult:
        messages = [
            {"role": "system", "content": self.SYSTEM_PROMPT},
            {"role": "user", "content": problem},
        ]
        action_trace = []

        for _ in range(max_turns):
            response = self.client.responses.create(
                model=self.model, input=messages, tools=self.TOOLS
            )
            calls = [item for item in response.output if item.type == "function_call"]

            if not calls:
                return TroubleshootingResult(answer=response.output_text, action_trace=action_trace)

            messages += response.output
            for call in calls:
                args = json.loads(call.arguments)
                result = self._dispatch_tool(call.name, args)
                action_trace.append({"tool": call.name, "args": args, "result": result})
                messages.append({
                    "type": "function_call_output",
                    "call_id": call.call_id,
                    "output": json.dumps(result),
                })

        return TroubleshootingResult(
            answer="Investigation limit reached without sufficient evidence.",
            action_trace=action_trace,
        )

When to Use and When it Breaks:

Pattern 11: Human-in-the-Loop [Month 14]

The finance team wants to automate refund approvals, but only for low-risk, clear-cut cases. Requests involving high amounts, fraud signals, or policy exceptions still require human approval. The agent’s role is to investigate the case, gather context, and recommend an action — the final decision remains with a person.

This pattern is not just a UI workflow — it is an architectural decision. You need durable state, reviewer assignment, approval logs, timeout handling, escalation paths, and safe recovery mechanisms. The agent’s responsibility is to prepare evidence and hand off context, not pause indefinitely waiting for a human response.

import json
from dataclasses import dataclass

@dataclass
class HumanApproval:
    approved: bool
    reviewer: str
    comment: str

@dataclass
class RefundOutcome:
    recommendation: str
    approval: HumanApproval
    customer_response: str

class RefundApprovalAgent(BaseLLMAgent):
    """Investigates refund request, prepares recommendation, pauses for human approval."""

    def _lookup_context(self, order_id: str) -> dict:
        # Replace with real payment service call
        return {
            "order_id": order_id,
            "amount": 420.00,
            "days_since_purchase": 12,
            "prior_refunds": 0,
            "fraud_score": 0.02,
        }

    def _build_recommendation(self, customer_message: str, context: dict) -> str:
        return self._text(
            "You are a refund operations agent. Recommend approve, deny, or request more information. "
            "Cite specific policy facts. Do not approve if fraud score exceeds 0.1.",
            f"Customer message:\\\\n{customer_message}\\\\n\\\\nOrder context:\\\\n{json.dumps(context)}",
        )

    def _request_human_approval(self, recommendation: str) -> HumanApproval:
        # In production: create a review task in the ops queue (Slack, Jira, internal tool).
        # Block until approved or timed out. This stub simulates a reviewer response.
        return HumanApproval(
            approved=False,
            reviewer="ops-manager",
            comment="Ask customer for photos of the damaged item first.",
        )

    def _draft_customer_response(self, recommendation: str, approval: HumanApproval) -> str:
        review_summary = json.dumps({"approved": approval.approved, "comment": approval.comment})
        return self._text(
            "Draft a customer-safe response based on the manager's review decision. "
            "Do not claim the refund is approved unless it is. Be empathetic but accurate.",
            f"Agent recommendation:\\\\n{recommendation}\\\\n\\\\nHuman review:\\\\n{review_summary}",
        )

    def run(self, order_id: str, customer_message: str) -> RefundOutcome:
        context = self._lookup_context(order_id)
        recommendation = self._build_recommendation(customer_message, context)
        approval = self._request_human_approval(recommendation)
        customer_response = self._draft_customer_response(recommendation, approval)
        return RefundOutcome(
            recommendation=recommendation,
            approval=approval,
            customer_response=customer_response,
        )

When to Use and When it Breaks:

Pattern 12: Plan-and-Execute [Month 15]

The DevOps team wants an agent that can handle multi-step infrastructure operations from a natural language request: “Resize the worker fleet from 10 to 20 instances, verify the queue drains, and update the runbook.” The objective is clear, but the exact execution path depends on what the system discovers at each step.

The Plan-and-Execute pattern separates planning from execution. A planner agent first reasons about the strategy and produces a structured task plan before any action is taken. This plan can be reviewed, modified, or approved by humans. An executor agent then carries out the steps one at a time, adapting as needed during execution.

This differs from hierarchical decomposition, where tasks are recursively broken into subgoals. Here, planning is a one-time decomposition into an execution plan. It also differs from ReAct — the entire plan is visible upfront, rather than emerging incrementally through action and observation.

import json
from dataclasses import dataclass

@dataclass
class ExecutionStep:
    step_id: int
    action: str
    tool: str
    parameters: dict
    expected_outcome: str

@dataclass
class ExecutionPlan:
    goal: str
    steps: list[ExecutionStep]

@dataclass
class ExecutionResult:
    plan: ExecutionPlan
    step_results: list[dict]
    summary: str

class PlanAndExecuteAgent(BaseLLMAgent):
    """Generates a structured plan before taking any actions, then executes step by step."""

    PLAN_SCHEMA = {
        "type": "object",
        "properties": {
            "steps": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "step_id": {"type": "integer"},
                        "action": {"type": "string"},
                        "tool": {"type": "string", "enum": ["resize_fleet", "check_queue_depth", "update_runbook"]},
                        "parameters": {"type": "object", "additionalProperties": {"type": "string"}},
                        "expected_outcome": {"type": "string"},
                    },
                    "required": ["step_id", "action", "tool", "parameters", "expected_outcome"],
                    "additionalProperties": False,
                },
            }
        },
        "required": ["steps"],
        "additionalProperties": False,
    }

    def _plan(self, goal: str) -> ExecutionPlan:
        result = self._structured(
            "Create a step-by-step execution plan for the given infrastructure task. "
            "Order steps by dependency. Use only available tools.",
            goal,
            self.PLAN_SCHEMA,
        )
        steps = [ExecutionStep(**s) for s in result["steps"]]
        return ExecutionPlan(goal=goal, steps=steps)

    def _execute_step(self, step: ExecutionStep) -> dict:
        # Real implementation would dispatch to infrastructure APIs
        return {"step_id": step.step_id, "status": "completed", "output": f"Executed: {step.action}"}

    def _summarize(self, plan: ExecutionPlan, results: list[dict]) -> str:
        return self._text(
            "Summarize the execution results. Flag any steps that did not complete as expected.",
            f"Goal: {plan.goal}\\\\n\\\\nResults:\\\\n{json.dumps(results, indent=2)}",
        )

    def run(self, goal: str) -> ExecutionResult:
        plan = self._plan(goal)
        step_results = []

        for step in plan.steps:
            result = self._execute_step(step)
            step_results.append(result)
            # In production: halt here if the step failed; surface for human review
            if result.get("status") != "completed":
                break

        summary = self._summarize(plan, step_results)
        return ExecutionResult(plan=plan, step_results=step_results, summary=summary)

When to Use and When it Breaks:

Pattern 13: Reflexion [Month 16]

The data team asks the AI platform to generate Python scripts for one-off data transformations. The first attempt often compiles successfully but breaks at runtime, usually on edge cases or unexpected inputs. Instead of escalating failures to a human, the team wants the agent to analyze the error, understand what went wrong, and retry with a better approach.

The Reflexion pattern gives agents the ability to learn from their own failures. Unlike a standard loop pattern, where retries are driven by an external judge, the agent evaluates its own output, reflects on the failure, and records observations about what went wrong. That memory then directly influences the next attempt, improving performance over time rather than repeating the same mistake.

import json
from dataclasses import dataclass

@dataclass
class ReflexionMemory:
    attempt: int
    output: str
    error: str | None
    reflection: str

@dataclass
class ReflexionResult:
    final_output: str
    memory: list[ReflexionMemory]
    succeeded: bool

class ReflexionCodingAgent(BaseLLMAgent):
    """Generates code, simulates execution, reflects on failures, and retries with accumulated memory."""

    REFLECTION_SCHEMA = {
        "type": "object",
        "properties": {
            "succeeded": {"type": "boolean"},
            "error_analysis": {"type": "string"},
            "what_to_fix": {"type": "string"},
        },
        "required": ["succeeded", "error_analysis", "what_to_fix"],
        "additionalProperties": False,
    }

    def _generate(self, task: str, memory: list[ReflexionMemory]) -> str:
        memory_context = "\\\\n\\\\n".join(
            f"Attempt {m.attempt}: FAILED\\\\nError: {m.error}\\\\nReflection: {m.reflection}"
            for m in memory
            if m.error
        )
        return self._text(
            "Write a Python function for the given data transformation task. "
            "Handle edge cases explicitly. Learn from past failures.",
            f"Task:\\\\n{task}\\\\n\\\\nPast failures:\\\\n{memory_context or 'None yet'}",
        )

    def _execute_and_observe(self, code: str) -> tuple[str | None, str | None]:
        """Returns (output, error). Replace with sandbox execution in production."""
        try:
            # Simulate execution — in production use a sandbox (e.g., subprocess, E2B)
            exec_globals: dict = {}
            exec(compile(code, "<agent>", "exec"), exec_globals)
            return "Executed successfully", None
        except Exception as e:
            return None, str(e)

    def _reflect(self, code: str, error: str) -> str:
        result = self._structured(
            "Analyze why this code failed. Identify the root cause and what should change in the next attempt.",
            f"Code:\\\\n{code}\\\\n\\\\nError:\\\\n{error}",
            self.REFLECTION_SCHEMA,
        )
        return result["what_to_fix"]

    def run(self, task: str, max_attempts: int = 3) -> ReflexionResult:
        memory: list[ReflexionMemory] = []

        for attempt in range(1, max_attempts + 1):
            code = self._generate(task, memory)
            output, error = self._execute_and_observe(code)

            if error is None:
                memory.append(ReflexionMemory(attempt=attempt, output=code, error=None, reflection="Succeeded"))
                return ReflexionResult(final_output=code, memory=memory, succeeded=True)

            reflection = self._reflect(code, error)
            memory.append(ReflexionMemory(attempt=attempt, output=code, error=error, reflection=reflection))

        return ReflexionResult(final_output=memory[-1].output, memory=memory, succeeded=False)

When to Use and When it Breaks:

Pattern 14: Custom Logic [Month 18]

The refund workflow has outgrown every clean architectural pattern. A high-value damaged-goods refund follows a different path than a standard return. Fraud signals route cases to specialists. Some checks can run in parallel, others require human approval, and certain situations call for store credit instead of a cash refund. At this stage, the business process contains real branching logic, and those branches often carry legal or financial consequences.

The custom logic pattern combines deterministic orchestration with agentic behavior. The challenge is deciding what belongs in code and what belongs in the model. Eligibility rules, permission checks, payment actions, and safety constraints should remain deterministic and auditable. Meanwhile, tasks involving interpretation, drafting, routing recommendations, and exception handling are often where the model adds the most value.

import asyncio
import json
from dataclasses import dataclass

@dataclass
class PurchaseVerification:
    order_id: str
    purchased_by_user: bool
    amount: float

@dataclass
class RefundPolicy:
    within_window: bool
    damaged_item_reported: bool
    fraud_flag: bool
    requires_manual_review: bool

@dataclass
class WorkflowResult:
    path: str
    output: str

class RefundOrchestrator(BaseLLMAgent):
    """Custom workflow: parallel checks → deterministic routing → agentic response drafting."""

    async def _verify_purchase(self, order_id: str) -> PurchaseVerification:
        # Replace with real payment service call
        return PurchaseVerification(order_id=order_id, purchased_by_user=True, amount=89.99)

    async def _evaluate_policy(self, order_id: str) -> RefundPolicy:
        # Replace with real policy service call
        return RefundPolicy(
            within_window=False,
            damaged_item_reported=True,
            fraud_flag=False,
            requires_manual_review=True,
        )

    def _draft_manual_review_packet(self, context: dict) -> str:
        return self._text(
            "Prepare a manual review packet for operations. "
            "Include the customer claim, policy status, decision points, and missing evidence.",
            json.dumps(context),
        )

    def _draft_refund_confirmation(self, context: dict) -> str:
        return self._text(
            "Draft an internal refund processing note. Include only confirmed facts from context.",
            json.dumps(context),
        )

    def _draft_store_credit_offer(self, context: dict) -> str:
        return self._text(
            "Draft a customer response offering store credit. "
            "Explain the policy reason clearly without being dismissive.",
            json.dumps(context),
        )

    async def run(self, order_id: str, customer_message: str) -> WorkflowResult:
        # Run eligibility checks in parallel — they are independent
        purchase, policy = await asyncio.gather(
            self._verify_purchase(order_id),
            self._evaluate_policy(order_id),
        )

        context = {
            "customer_message": customer_message,
            "purchase": {"purchased_by_user": purchase.purchased_by_user, "amount": purchase.amount},
            "policy": {
                "within_window": policy.within_window,
                "damaged_item_reported": policy.damaged_item_reported,
                "fraud_flag": policy.fraud_flag,
            },
        }

        # Deterministic routing — these rules must not be delegated to the model
        if not purchase.purchased_by_user:
            return WorkflowResult(path="denied", output="Refund denied: requester is not the purchaser.")

        if policy.fraud_flag:
            return WorkflowResult(path="fraud_hold", output="Refund held: fraud flag active. Routing to fraud team.")

        if policy.requires_manual_review:
            return WorkflowResult(path="manual_review", output=self._draft_manual_review_packet(context))

        if policy.within_window:
            return WorkflowResult(path="approved", output=self._draft_refund_confirmation(context))

        return WorkflowResult(path="store_credit", output=self._draft_store_credit_offer(context))

When to Use and When it Breaks:

Pattern 15: Event-Driven Agent

The fraud team realizes their biggest challenge is not routing, but timing. By the time a support ticket highlights suspicious activity, the opportunity to block or intervene on the transaction has often already passed. They do not need agents that wait for instructions — they need agents that react the moment something important happens.

Most patterns discussed so far are request-driven: they begin when a user or system explicitly asks for something. The event-driven pattern works differently. It is asynchronous and reactive, subscribing to event streams such as a Kafka topic, webhook, or database change feed. When a predefined condition is triggered, the agent wakes up and acts autonomously, without waiting for a human request.

import asyncio
import json
from dataclasses import dataclass
from datetime import datetime

@dataclass
class FraudEvent:
    event_id: str
    user_id: str
    amount: float
    merchant: str
    timestamp: str
    risk_signal: str  # e.g., "velocity_spike", "new_device", "geo_anomaly"

@dataclass
class FraudDecision:
    event_id: str
    action: str  # "block" | "flag" | "allow"
    reasoning: str
    triggered_at: str

class FraudDetectionAgent(BaseLLMAgent):
    """
    Reacts to real-time fraud events. In production, subscribe this to a Kafka consumer
    or webhook endpoint rather than calling run() directly.
    """

    DECISION_SCHEMA = {
        "type": "object",
        "properties": {
            "action": {"type": "string", "enum": ["block", "flag", "allow"]},
            "reasoning": {"type": "string"},
        },
        "required": ["action", "reasoning"],
        "additionalProperties": False,
    }

    async def _enrich_context(self, event: FraudEvent) -> dict:
        """Fetch user history and merchant reputation — replace with real API calls."""
        return {
            "user_prior_disputes": 0,
            "merchant_risk_tier": "medium",
            "transaction_count_last_hour": 7,
        }

    def _decide(self, event: FraudEvent, context: dict) -> dict:
        return self._structured(
            "You are a fraud risk analyst. Evaluate the transaction event and enriched context. "
            "Only block when evidence is strong. Flag when uncertain. Allow when risk is low.",
            f"Event:\\\\n{json.dumps(event.__dict__)}\\\\n\\\\nContext:\\\\n{json.dumps(context)}",
            self.DECISION_SCHEMA,
        )

    async def handle_event(self, event: FraudEvent) -> FraudDecision:
        context = await self._enrich_context(event)
        decision = self._decide(event, context)
        return FraudDecision(
            event_id=event.event_id,
            action=decision["action"],
            reasoning=decision["reasoning"],
            triggered_at=datetime.utcnow().isoformat(),
        )

    async def run_consumer_loop(self, event_queue: asyncio.Queue) -> None:
        """Consume events from an async queue — wire to Kafka/webhook in production."""
        while True:
            event = await event_queue.get()
            decision = await self.handle_event(event)
            # Publish decision back to event bus, audit log, or action service
            print(f"[{decision.triggered_at}] {decision.event_id}: {decision.action} — {decision.reasoning}")
            event_queue.task_done()

When to Use and When it Breaks:

Pattern Selection Summary

Ten Rules for Production Agentic Systems

1. Start with the smallest pattern that works. A single agent with good tool contracts beats a multi-agent architecture with weak ones.

2. Give every tool a narrow schema and an honest description. Tool descriptions are not documentation — they are the model’s only source of truth about what the tool does. Write them as you would write a contract.

3. Cap iterations, tool calls, and total spend per request. An agent without budget limits is a liability waiting to appear in a bill review.

4. Log the action trace. Model request ID, selected tool, arguments, output summary, and final decision. Without this, incident investigation is guesswork.

5. Keep irreversible actions behind deterministic checks or human approval. A model should never be the only gate between a user input and a database write, a money movement, or a production change.

6. Evaluate agents with real failure cases, not just happy-path prompts. An agent that handles the happy path correctly is a prototype. An agent that handles the edge cases correctly is a product.

7. Separate prompts by responsibility before the system prompt becomes unreadable. When you catch yourself adding “but do not do X when Y” to a system prompt, that is a sign the agent is doing two jobs.

8. Treat multi-agent systems as distributed systems. Partial failure, timeouts, retries, ownership, and observability are not optional. They are the cost of autonomy.

9. Model review is not a substitute for deterministic validation. Use critics, judges, and reviewers to improve quality. Use tests, policy engines, permission checks, and database constraints to enforce correctness.

10. Prefer the simpler pattern. Not because simpler is always better — but because the complexity budget you save on the orchestration layer can be spent on the thing that actually matters: better tools, better prompts, and better evaluation.

The Right Shape

Agent design patterns are not maturity levels. A swarm is not more sophisticated than a single-agent system if the task only needs one reliable tool-using agent. A hierarchical decomposition is not better than a sequential pipeline if the business process is already stable. Plan-and-Execute is not an upgrade from ReAct if the plan will be stale before the third step.

The right pattern is the one that matches the shape of uncertainty in the work:

• Uncertainty in tool choice → Single-Agent or ReAct

• Uncertainty in routing → Coordinator

• Uncertainty in quality → Review & Critique or Iterative Refinement

• Uncertainty in execution path → Plan-and-Execute or ReAct

• Uncertainty in self-correction → Reflexion or Loop

• Uncertainty in business risk → Human-in-the-Loop or Custom Logic

• Uncertainty in problem structure → Hierarchical Decomposition or Swarm

• Uncertainty in timing (cannot wait for a user request) → Event-Driven Agent

• Uncertainty in output quality at scale → LLM-as-Judge

The most reliable production systems are usually not the most autonomous ones. They are the ones that put autonomy exactly where it creates value and constrain it everywhere else.

References

• Google Cloud Architecture Center, “Choose a design pattern for your agentic AI system”: https://cloud.google.com/architecture/choose-design-pattern-agentic-ai-system

• Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models” (ICLR 2023)

• Shinn et al., “Reflexion: Language Agents with Verbal Reinforcement Learning” (NeurIPS 2023)

• Chase, “Plan-and-Solve Prompting” and LangGraph plan-and-execute documentation

• Anthropic, “Building Effective AI Agents”: https://www.anthropic.com/research/building-effective-agents

• OpenAI API documentation, Responses API: https://platform.openai.com/docs/api-reference/responses

• OpenAI API documentation, Function calling guide: https://platform.openai.com/docs/guides/function-calling

Every time you type claude in your terminal, it starts with zero context. At this point, this is probably the millionth article saying the same thing.

But if you’ve already read that a hundred times, here’s the part most people skip: adding the right context.

One thing I strongly prefer is keeping it crisp. Claude has been trained on an immense amount of data. If you’re stuffing your context with things it already knows, you’re just wasting tokens.

Here’s what I’ve added to my root claude.md file:

# Global Context & Instructions

## My Role
I am a Senior AI Engineer and Full Stack Software Engineer specializing in Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) frameworks, and AI agent orchestration. My daily work bridges high-performance backend engineering and advanced AI research. 
---

## Preferences
When assisting me, please adhere to the following interaction and coding preferences:

**CONVERSATIONAL CHATBOT**
- When I am interacting with claude, make sure the response is very crisp. Always support markdown, table. Limit the output tokens are limited as possible, unless ask to elaborate more.

**PLANNING**
- Always use internet to verify things if you are working on tools like AWS, Bazel, Cloud Architecture.

**CODING PREFERENCES**

- Default to `Python 3.12`. Always use standard coding practices, design pattern, class based approach
- Add crisp and relevant comments in the code, do not add noise.

**CONTENT GENERATION** 
- When helping with Medium articles or YouTube content, provide SEO-optimized descriptions, engaging thumbnails/hooks, and highly structured outlines. Maintain a highly technical but accessible tone for Medium drafts.

**DEBUGGING**
- While debugging cloud related issue, always read latest aws documentation and use aws mcp to fetch details from cloud.
**SKILL USAGE**
- Wherever possible use skills
- Never use skills from `Atlasian`

First, yes, I’ve added a few repetitive things.

But later I optimized this file to store my personal coding preferences — the tools, MCPs, and workflows I want Claude to use.

The goal here is to optimize for how I code, not to create a perfectly organized, company-wide standard.

Subscribe now

2. Researching and Learning Like a Pro…

Whenever I want to research or learn something new, the old approach was pretty straightforward — read 2–3 articles and then get hands-on.

Now, you can effectively read 50+ articles and condense them into a single, well-structured document.

Here’s my workflow:

• First, I go to Gemini and use Deep Research mode to generate a research canvas. Then I export it to a Google Doc.

For example, I recently wanted to learn about LLM evaluation techniques. Gemini generated an 18-page document for me:

It did make things a little complex and added some irrelevant details here and there. But overall, I’d say I got a pretty solid understanding of the topic I wanted to learn.

Over time, I started trying one more thing: giving the exact same prompt to different models.

• DeepSeek

• Claude

• Gemini

• ChatGPT

• Grok

You’ll notice that each one gives noticeably different outputs. Some explain things better, some go deeper, and some are just easier to follow depending on the topic.

These are the free (or mostly free) alternatives I regularly use when I’m trying to learn something new.

Subscribe now

3. MCP Servers:

I don’t know why people keep saying MCPs are dead. To me, they’re definitely not.

I’m a developer. On any given day, I’m writing code, managing AWS infrastructure, maintaining JIRA tickets, replying on Slack, and juggling everything else developers usually end up doing.

So if I want to connect all of these things into a single ecosystem, why wouldn’t I use MCPs?

The context switching becomes dramatically lower. Instead of jumping across ten tabs and tools, I can run a single command and work across multiple systems from one place.

Recently, I was debugging an issue in AWS, and I plugged in an AWS MCP. It genuinely made the process smoother.

It read the logs, fetched the required ARNs, and when I passed an ARN, it already understood what resource I was referring to and how it was configured. That level of context awareness removes a lot of friction when debugging.

Recently, while setting up a new AWS CDK stack, I needed multiple ARNs and endpoints from AWS. Instead of hopping across different AWS console pages, I pulled everything I needed with a single line:

Install your self: AWS MCP, highly recommended

4. Skills:

This is actually a pretty powerful concept.

Heavy models like Opus often cost more because of the extra thinking tokens they use. But what if the response you want can be achieved through a predictable chain of thought — one you can define upfront? In that case, the response becomes faster, cheaper, and far more consistent.

That’s where skills become useful.

You don’t always have to write skills from scratch either. There are plenty of community-built ones available on GitHub. Some examples I’ve tried:

• senior-pm-prompt

• emilkowalski/skill

• aayushostwal/nexus

• superpowers

You can define in your root claude.md file which skills should always be loaded and what behaviors Claude should follow.

One important thing to keep in mind: skills are not free from a context perspective. By default, Claude reserves only a small portion of the context window for them. So you can’t just keep adding skills endlessly — there’s a tradeoff between capability and available context.

Subscribe now

5. Persistent Memory of Agents and Git Worktrees:

Since I work in a startup environment, it’s pretty common for me to juggle multiple tasks in the same repo.

That creates an annoying problem: either you keep your terminal session alive when switching tasks, or you end up rebuilding the entire context when you come back to an older one.

It’s frustrating — and honestly, expensive from a token perspective.

I ended up using two approaches to manage persistent memory for my agent.

The first one is a /save-context command that writes the current context to a file on disk. That way, when I start a new session, I can simply reload it instead of gathering everything again.

# Dump the current session context to a file so it can be resumed later.

Please create a context snapshot file that will let me resume this session from scratch.

## Steps

1. Determine a filename: `context-snapshot-<YYYY-MM-DD-HHMM>.md` using the current date and time.

2. Write the file to the project root (or current working directory if no project root is found). The file should contain the following sections:

---

### Section 1 — Session Goal
A 2–3 sentence summary of the overall goal or task we have been working on in this session.

### Section 2 — Current Status
What has been completed so far and what the current state is. Be specific: mention files created/modified, decisions made, and what was the last action taken.

### Section 3 — Pending Tasks
A prioritised numbered list of things still left to do to complete the goal.

### Section 4 — Key Decisions & Context
Any important decisions, constraints, or background context that the next Claude session must know to continue correctly. Include things like: chosen libraries, architectural decisions, naming conventions, credentials locations, environment details, or anything non-obvious.

### Section 5 — Exact Resume Prompt
Write a ready-to-paste prompt (inside a markdown code block) that the user can drop at the start of a fresh Claude Code session to resume seamlessly. It should:
- Reference the snapshot file by name
- Ask Claude to read the file first
- State the immediate next step clearly

---

3. After writing the file, print the absolute path to the file and a one-line confirmation message.

Recently, I started using agentmemory — a dedicated repo for maintaining context across sessions.

Instead of relying on temporary terminal history or rebuilding context every time, I keep the important details in one place and let the agent pick things up from there.6.

6. Working on multiple Github Repo:

If you work across multiple GitHub repos, you already know that every repo has its own nuances.

Different coding patterns, different deployment flows, different tooling, and sometimes even different engineering philosophies.

That’s exactly why you should stop treating every repository the same.

You can define repo-specific rules, skills, and MCP servers for each project. This isn’t a groundbreaking idea — chances are, you’ve already read about it before.

But the real question is: have you actually implemented it?

In practice, most people don’t.

If you’re working in a monorepo, this becomes even more useful. You can add a project-scoped claude.md file for different services or modules so Claude gets the right context much faster.

Instead of re-explaining architecture, conventions, or dependencies every single time, you can simply reference the relevant project context with something like this:

# payments-service/claude.md

- This service handles payment orchestration and Stripe webhooks. 
- Prefer using Nexus for codebase navigation and AWS MCP for infra/log debugging. 
- Follow existing async patterns, avoid introducing sync calls, and reuse shared payment utilities before adding new abstractions.

# frontend/claude.md

- This is a Next.js frontend app. Always follow the existing component structure and design system. 
- Use shared UI components first, prefer server components where possible, and avoid adding new dependencies unless necessary.

At the end of the day, I don’t think agentic AI is about replacing engineers.

It’s about reducing friction.

Less context switching. Less repeating yourself. Less manually digging through dashboards, logs, docs, and codebases.

Most people are still using these tools like smarter chatbots. But the real unlock happens when you start building systems around them — persistent memory, MCPs, repo-specific context, skills, and workflows that match how you actually work.

None of this is perfect yet. Half of the time, I’m still experimenting.

But one thing is clear: the developers who learn how to orchestrate these tools will move much faster than the ones treating AI like autocomplete.

You open Claude Code. You start typing. And almost immediately you find yourself explaining — again — that your project uses App Router, not Pages Router. That you run pnpm, not npm That you want typed components only, please.

Claude doesn’t know. It can’t. Every session is a blank slate. You’re repeating yourself to one of the most capable tools you’ve ever used — and that’s just wasteful.

CLAUDE.md is the fix. One markdown file. Loaded automatically. Every single time.

Think of it like this: CLAUDE.md is onboarding documentation for your AI teammate — the briefing you’d give a brilliant new engineer on their first day, so they stop asking obvious questions and start shipping.

What is CLAUDE.md, exactly?

CLAUDE.md is a plain markdown file that Claude Code reads at the start of every session. It becomes part of Claude’s context automatically — you don’t reference it, attach it, or paste it. It’s just there.

You can use it to document anything Claude can’t infer from your code alone: build commands, test workflows, project structure, naming conventions, architectural decisions, gotchas, and domain-specific terminology.

There’s no required format. Keep it short. Keep it human-readable. Every line should answer the question: what would Claude get wrong without this?

The file hierarchy

CLAUDE.md works in layers. More specific files override more general ones.

The project CLAUDE.md is where most of your leverage lives. Start there.

Getting started in 30 seconds

Don’t write it from scratch. Use the /init command. Claude Code will scan your project — package files, config, directory structure — and generate a starter CLAUDE.md tailored to what it finds.

# in your project directory
/init

Then prune it ruthlessly. The generated file often includes things Claude already knows. Delete anything that isn’t genuinely project-specific. Shorter is better — every line competes for attention with your actual task.

What actually belongs in it

The single highest-impact instruction you can add: tell Claude how to verify its own work.

# CLAUDE.md:
# Verification
After making code changes, always run:
1. `pnpm lint`   — catch style issues
2. `pnpm test`   — verify nothing broke
3. `pnpm build`  — check for type errors

When Claude knows how to check its own work, it catches mistakes before you ever see them. This one pattern measurably improves output quality across every task.

Subscribe now

What to leave out

This is just as important. Overloaded CLAUDE.md files actively hurt performance — when there are too many instructions, Claude doesn’t just ignore the new ones. It starts ignoring all of them uniformly.

Frontier models can reliably follow roughly 150–200 instructions. Claude Code’s system prompt already uses ~50. That leaves you around 100–150 before things quietly start degrading.

• ❌ Code style rules — use a linter. LLMs are slow and expensive compared to ESLint.

• ❌ Things Claude already knows — TypeScript syntax, common frameworks, async/await.

• ❌ Credentials — use environment variables; reference them by name only.

• ❌ Rapidly-changing info — use session prompts for volatile context.

• ❌ Full documentation — link to docs, don’t reproduce them.

Scaling for bigger projects

When a single file gets unwieldy, split it. Create a .claude/rules/ directory and drop focused rule files inside:

your-project/
├── CLAUDE.md          # core project context
└── .claude/
    └── rules/
        ├── testing.md    # testing conventions
        ├── api-rules.md  # backend-specific rules
        └── security.md   # security requirements

Files in .claude/rules/ with YAML frontmatter specifying paths: are lazy-loaded — they only enter context when Claude touches matching files. This is powerful: you can have detailed rules for your API layer that never clutter a UI-only task.

Five rules for a great CLAUDE.md

1. Every line must earn its place. Ask: would Claude make a mistake without this? If no — delete it.

2. Use /init to generate, then prune. Deleting is always faster than writing from scratch.

3. Always include your verification commands. This single instruction has the highest return of anything you can add.

4. Keep it human-readable. If a new team member couldn’t scan it in two minutes, it’s too long.

5. Treat it as a living document. Add lines when Claude makes a mistake that a clear instruction would have prevented.

The bottom line

CLAUDE.md is one of the highest-leverage things you can configure in your entire development environment. A good one transforms Claude Code from a capable but context-blind assistant into something that actually knows your project — your commands, your architecture, your quirks.

Run /init. Delete the noise. Add your verification steps. Commit it. Then watch how much less you repeat yourself.

But what if the map is actually a distraction? A recent, rigorous study from ETH Zurich and DeepMind has turned this assumption on its head, and the results are a wake-up call for anyone who has been meticulously curating these files.

The Great AGENTS.md Experiment

The paper, titled “Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?” by Thibaud Gloaguen, Niels Mündler, Mark Müller, Veselin Raychev, and Martin Vechev, set out to answer a simple question that no one had yet rigorously tested: Do these context files actually help AI coding agents complete real-world tasks?

To find out, the researchers built a new benchmark called AGENTbench, consisting of 138 real-world software engineering tasks from 12 Python repositories that already use developer-written context files. They complemented this with the established SWE-bench Lite benchmark, which has 300 tasks from popular repositories, where they used LLMs to generate the context files.

They then set four popular coding agents loose on these tasks in three different modes: with no context file at all, with an LLM-generated context file, and with a human-written one. The results were nothing short of counterintuitive.

Subscribe now

The Surprising Results: A Marginal Affair

The researchers’ findings directly contradict the prevailing “more context is better” wisdom. Here’s what they discovered:

• LLM-Generated Files Hurt Performance: On average, automatically generated AGENTS.md files decreased task success rates by a staggering 3%. Providing an AI with auto-generated instructions was worse than providing none at all.

• Human-Written Files Are... Just Okay: Developer-written context files provided only a marginal improvement, increasing success rates by an average of 4%. This is a very small gain for what is often a significant documentation effort.

  

• A Hefty Price Tag for Small Gains: Even when they “helped,” context files came with a significant cost. They consistently increased the inference cost of completing a task by over 20%. The agents required more steps, more tokens, and more thinking time to achieve barely better (or even worse) results.

Why Does More Context Hurt Performance?

This is the critical question. If the agents are following the instructions, why aren’t they doing better? The study’s analysis of agent behavior traces provides some fascinating answers.

1. Agents Became Overzealous Rule-Followers

The good news is that the coding agents are excellent instruction-followers. The bad news is that this becomes a problem. When a context file instructs an agent to “thoroughly test all changes” or “use uv for package management,” the agent complies, even when it’s unnecessary for the simple bug fix at hand. This leads to:

• Increased, Often Redundant, Testing: Agents ran more tests than needed, wasting time and compute.

• Unnecessary Exploration: They navigated through more files, performed more searches, and used more repository-specific tooling than required for the task.

The result is a well-intentioned agent that spends a lot of its energy on meta-tasks, losing sight of the core goal: resolving the issue efficiently.'

2. Context Files Are a Poor Man’s Codebase Overview

One of the primary promises of a context file is to provide a useful architectural overview, helping the agent quickly find the right files. The study found this promise to be largely unfulfilled. Agents with access to context files did not discover relevant files any faster than those without them. It turns out that the agents’ built-in ability to explore and search a codebase is already quite good, and a static, often high-level map doesn’t add much value.

3. The “No Documentation” Trap

Interestingly, the researchers tested a scenario where all other documentation was removed, leaving the context file as the sole source of information. In this extreme case, LLM-generated context files did consistently improve performance, even outperforming human-written documentation. This explains why some developers anecdotally report success—their repositories likely had little to no documentation to begin with, so any documentation was a massive help. This is a crucial insight: the effectiveness of a context file is inversely proportional to the quality of your project’s existing documentation.

A Better Way Forward: The Principle of Minimal Instruction

So, should you throw your AGENTS.md file in the trash? Not necessarily, but the way you create them needs a fundamental rethink. The study’s core recommendation is a principle of minimalism.

“We therefore suggest omitting LLM-generated context files for the time being, contrary to agent developers’ recommendations, and including only minimal requirements (e.g., specific tooling to use with this repository).”

Here are some actionable takeaways based on the research:

• Stop Auto-Generating: For now, the evidence strongly suggests you should ditch the /init command and other automated tools for creating context files. The risk of them causing harm far outweighs the negligible benefit. Always hand-write your AGENTS.md based on real experience with the codebase.

• Start with an Empty File: A zero-length AGENTS.md is better than a verbose one. Build it iteratively, only adding instructions when you observe the agent consistently struggling with a specific, non-obvious aspect of your project.

• Focus on the “What,” Not the “How”: Your instructions should clarify what the agent needs to accomplish, not a step-by-step guide on how to do it. For example, “Use uv for package management” is a good, concise instruction. A paragraph on how to structure a loop is redundant and potentially harmful.

• Invest in Good Conventional Documentation: The study shows that agents are very capable of understanding standard documentation. A well-maintained README.md, clear code comments, and logical directory structures are likely a far better investment of your time than a dedicated, highly-curated agent file.

Conclusion

The “Evaluating AGENTS.md” paper is a much-needed reality check in the fast-moving world of AI coding assistants. It reminds us that the intuitive solution is not always the right one. While the idea of a dedicated instruction file for our AI partners is elegant in theory, in practice, it’s a tool that is often misused, leading to distracting, costly, and self-sabotaging behavior.

The path forward is not about providing more context, but better, more precise context. The most effective AGENTS.md file you can have is likely the one that is almost empty, containing only the few critical details that your coding agent cannot discover on its own. Let’s stop building ornate maps that confuse the tourist and instead just point them in the right direction. I highly recommend reading the full paper for a deeper dive into the data. You can also check out the open-source code for their AGENTbench framework.

Sources:

[1] https://arxiv.org/html/2602.11988v1

However, as your team creates highly effective, context-aware “Claude Skills” (custom prompts, Model Context Protocol (MCP) servers, and project-specific instructions), a new challenge emerges:

How do you scale and share these AI capabilities across dozens or hundreds of repositories?

Without a centralized strategy, teams end up with fragmented prompt libraries, duplicated effort, and inconsistent AI outputs.

In this guide, we will explore three proven strategies for distributing and maintaining global Claude Skills across your organization to achieve seamless AI workflow automation and build powerful shared AI tooling.

1. Introduction: The Need for Shared AI Tooling

A “Claude Skill” in a modern engineering context typically consists of:

• Custom Instructions: ./skills/skill.md files that dictate how the AI interacts with your specific codebase.

• Prompt Templates: Standardized markdown files containing architectural guidelines, testing standards, or PR review checklists.

• MCP Servers: Scripts and configurations that allow Claude to interact with your internal APIs, databases, or cloud infrastructure.

When you centralize these skills, you ensure that every developer — whether they are on the frontend, backend, or DevOps team — has access to the most optimized, up-to-date AI context. This reduces hallucination, enforces coding standards automatically, and accelerates onboarding.

The Centralized AI Tooling Architecture

Method 1: Git Submodules (Most Common Approach)

For many engineering teams, the simplest way to share code is through standard Git mechanisms. Git Submodules allow you to keep a dedicated repository of Claude Skills and embed it into any project.

Repository Structure

First, create a centralized repository named org-claude-skills:

org-claude-skills/
├── skills/
│   ├── code-review.md
│   ├── write-tests.md
├── mcp-servers/
│   ├── internal-wiki-search/
│   └── jira-integration/
├── global-instructions.md
└── README.md

Setup Steps & Example Commands

To add these skills to a target application repository, a developer simply adds the skills repo as a submodule.

# Navigate to your application repository
cd my-node-backend
# Add the centralized skills repository as a submodule
git submodule add https://github.com/your-org/org-claude-skills.git .claude-skills
# Initialize and update
git submodule update --init --recursive

• Pros: No new infrastructure required; strictly tied to Git versions.

• Cons: Submodules can be cumbersome for developers unfamiliar with updating them.

Method 2: Install Skills via Script (Recommended for AI Teams)

For teams that want to automate the distribution of AI developer productivity tools without the overhead of Git submodules, an installation script is the perfect middle ground. This allows you to fetch the latest prompt templates and configurations dynamically.

Automation Workflow

Scripting Workflow

1. Host your Claude Skills in a central repo or an S3 bucket.

2. Provide developers with a simple curl command.

3. The script downloads the relevant skills into a .ai/ or .claude/ directory in their local project.

Install Script Example (install-skills.sh)

#!/bin/bash
# install-skills.sh - Automate shared AI tooling distribution
SKILLS_REPO_URL=https://raw.githubusercontent.com/your-org/org-claude-skills/main)
TARGET_DIR=”.claude”
echo “Installing Global Claude Skills...”
mkdir -p $TARGET_DIR/prompts
# Download global coding standards
curl -s “$SKILLS_REPO_URL/prompts/frontend-standards.md” -o “$TARGET_DIR/prompts/frontend-standards.md”
curl -s “$SKILLS_REPO_URL/prompts/testing-guidelines.md” -o “$TARGET_DIR/prompts/testing-guidelines.md”
# Download MCP Config template
curl -s “$SKILLS_REPO_URL/mcp-config.json” -o “$TARGET_DIR/mcp-config.json”
echo “Claude Skills successfully installed in $TARGET_DIR/”

CI/CD Usage

You can integrate this script into your CI/CD pipeline (e.g., GitHub Actions) to ensure that automated AI code reviewers always have the absolute latest prompt templates before analyzing a PR.

Method 3: NPM / Python Package (Best for Large Organizations)

For enterprise-scale platform engineering teams, packaging Claude Skills as reusable libraries via native package managers (NPM, PyPI) provides the ultimate level of version control and seamless integration.

Package Distribution Workflow

Package Structure

If your engineering org uses Node.js, you can create a package named @myorg/claude-skills.

claude-skills-package/
├── package.json
├── index.js           # Exports paths to prompts and MCP tool definitions
├── prompts/
│   └── standard-react-component.md
└── bin/
    └── setup-claude.js # CLI tool to scaffold local AI configs

Publishing Workflow & Install Example

Publish the package to your private NPM registry:

npm publish --access restricted

Developers can now install the skills just like any other dependency:

npm install @myorg/claude-skills --save-dev

You can then provide an executable binary in the package to scaffold the AI context locally:

# Scaffolds the .claude.json or MCP settings using the package data
npx setup-claude-skills --type frontend

• Pros: Excellent version control, semantic releases, and a familiar developer workflow.

• Cons: Language-specific (NPM for JS, PyPI for Python), which might be an issue in polyglot organizations.

Comparison Table

Best Practices

Regardless of the distribution method you choose, adhere to these best practices when scaling AI developer productivity:

• Versioning is Critical: AI models change, and so do effective prompting strategies. Always version your skills. If a prompt breaks due to a new Claude model update, you need the ability to roll back.

• Documentation for AI (and Humans): Write self-describing metadata for your skills. Use XML tags (e.g., <skill_description>) inside your prompt files so the AI understands when to use a specific skill.

• Security & Secrets: Never hardcode API keys or internal secrets inside shared Claude Skills. Use environment variable placeholders (e.g., {{JIRA_API_KEY}}) and rely on local .env files or secure secret managers during runtime execution.

Conclusion

Adding global Claude Skills across multiple repositories transforms how your organization leverages generative AI.

• If you are a small team, start with Git Submodules.

• If you manage a polyglot environment with diverse languages, use an Install Script.

• If you are a Platform Engineering team supporting hundreds of engineers, invest in Package Managers to distribute AI context as version-controlled code.

By standardizing your AI tooling, you ensure that every prompt is highly engineered, secure, and ready to accelerate your software development lifecycle.

Hopefully, this saves you from the headache of managing 50 different .claude folders.

How are you handling skill distribution across your teams?

If you’ve found a cleaner way to sync these—or if you’ve run into any symlink quirks—drop a comment below. I’d love to see how others are tackling this!

Thank you :)

So, why distinguish between them? Because while their “DNA” (Markdown) is similar, their “Biology” (how the AI pursues them) is entirely different.

1. The Skill: The “Atomic” Tool

A Skill is a discrete, bounded capability. Think of it as a specialized tool in a digital Swiss Army knife, designed to perform one specific task with high reliability.

• Claude’s Perspective: When Claude sees a skill, it views it as a contract. This contract stipulates: “If I provide input X in this exact format, I am guaranteed to get output Y.” It is a technical handshake where the AI relinquishes creative control in favor of procedural accuracy.

• The Scope: A skill doesn’t think about the big picture or the ultimate “why” of a user’s request. For example, a fetch_github_repo skill doesn’t care if you are fetching the repo to debug a crash, audit security, or simply learn the syntax; its only responsibility is to navigate the API authentication, handle pagination, and return the raw code string. It is a servant of the process, not the goal.

• The Content: In a skills.md file, you focus on parameters, constraints, and schemas. The language here is often clinical and precise, defining the exact boundaries of what the tool can and cannot do. You are defining the mechanics of an action.

The Analogy: A Skill is a Hammer. It is excellent at hitting nails, but it has no opinion on whether you are building a bookshelf or a birdhouse, and it certainly won’t decide to start swinging on its own.

2. The Agent: The “Reasoning” Orchestrator

An Agent is an autonomous entity designed to achieve a high-level goal by managing its own thoughts, planning its own trajectory, and executing actions.

• Claude’s Perspective: When Claude acts as an agent, it moves from “Execution Mode” to “Reasoning Mode.” It adopts a persona and maintains a stateful memory of the task’s progress. Most importantly, it possesses the authority to decide which skills to use, when to use them, and — crucially — when to stop because the goal has been met or the path is blocked.

• The Scope: An agent is goal-oriented and context-aware. It looks at a prompt like “Migrate this app to AWS,” realizes that this isn’t a single step, and begins a mental breakdown of the problem. It identifies that it must first analyze the local environment, then provision cloud resources, and finally deploy the code. It manages the transitions between these phases.

• The Content: In an agents.md file, you focus on Persona, Logic, and Chain of Thought. The instructions here are strategic, telling the AI how to think when it encounters an error or how to prioritize competing interests. You are defining the intent and the judgment of the entity.

The Analogy: An Agent is the Carpenter. They know how to use the hammer, the saw, and the drill. They look at a blueprint, manage the timeline, and adjust their technique if they hit a knot in the wood. They turn a collection of movements into a finished home.

Subscribe now

3. If They Are Interchangeable, Why Exist as Two Entities?

Technically, you could write a single massive Markdown file that combines everything. However, the separation into Agents and Skills is an architectural choice for Scale, Precision, and Context Management.

1. Context Window Efficiency: If you put every instruction into the primary Agent, the “Context Window” (the AI’s active memory) gets cluttered with irrelevant details. By keeping Skills separate, the Agent only “loads” the specific skill it needs at that exact moment. This prevents “Instruction Dilution,” where the AI starts ignoring rules because there are too many of them.

2. Modular Scalability: You can write a “Security Auditor” Agent and a “DevOps Engineer” Agent. Both might need the same read_file skill. By separating them, you don’t have to duplicate the technical code for reading files in both places. You define the Capability once (Skill) and the Governance twice (Agents).

3. Deterministic vs. Creative: Skills are meant to be Deterministic (predictable). You want a calculator to always return 4 when you add 2+2. Agents are meant to be Creative (adaptive). You want an agent to pivot if a deployment fails. Keeping them separate ensures that your tools remain reliable while your logic remains flexible.

4. Parallel Agent Processing: The Speed Boost

One of the most powerful reasons to separate these entities is the ability to run Parallel Agent Processing.

In a traditional “Sequential” setup, one Agent does everything one step at a time. If it needs to research five topics, it does them one by one. In a Multi-Agent architecture, the “Lead Agent” can delegate work to several “Sub-Agents” simultaneously.

• How it works: A high-level goal like “Audit this 50-file repository for security flaws” is broken down. The Lead Agent identifies 10 key modules and spawns 10 Sub-Agents at once.

• The Latency Benefit: If one file takes 30 seconds to audit, a sequential agent takes 5 minutes (30s x 10). A parallel system can finish the entire job in roughly 40 seconds (including coordination time).

• Cross-Checking: Parallel agents can also act as “Reviewers.” While Agent A generates a solution, Agent B (running in parallel) can begin analyzing the draft for security risks, effectively “vibe-checking” the work in real-time before it’s even presented to the user.

Summary Table

Conclusion

If you are building an AI system, don’t just build a library of skills. Without an Agentic layer to manage them, your AI is just a powerful engine without a driver. Conversely, an Agent without Skills is a brilliant strategist with no hands.

To build truly “smart” systems, define your Skills for precision and your Agents for purpose. By separating the how from the why, you enable modular growth and high-speed parallel processing that a monolithic “all-in-one” prompt could never achieve.

This is a 5-article series on Context Engineering:

1. ✅ Prompt Engineering

2. ️➡️ Memory and Token Management

3. AI Grounding Strategies and Advanced RAG

4. LLM Evaluation and Ensemble Methods

These series are completely organic and contain a lot of examples from my experiences. I want to refer to all those engineers who are starting their AI journey.

Token and memory management are critical aspects of context engineering. But before we dive into them, it’s important to understand how an LLM actually generates a response.

Let’s walk through a simple example.

Input: 
Explain how airplanes fly

LLM Output: 
Airplanes fly because of lift created by the wings moving through air.

Here is a detailed diagram that demonstrates what happens when you make an API call.

• The input prompt is first converted into tokens — essentially a sequence of integers.

• These integers are then transformed into vectors, known as embeddings.

• The embeddings are passed into the LLM, where the model performs inference.

• As the model processes them, it produces a probabilistic distribution over the next possible tokens.

• The most likely token is selected, converted back into text, and appended to the response.

• This process repeats step by step, allowing the model to generate a continuous sequence of text.

Now let’s dive deeper into each step and see what’s actually happening under the hood.

Tokenization:

Language models do not read text the way humans do. Before a model can process a sentence, the text is first tokenized, and each token is mapped to a discrete numeric ID.

One straightforward way to do this is to assign a unique ID to each character or word.

Character-level tokenization, however, produces very long sequences. Even a short sentence turns into a large array of tokens, which makes computation inefficient and expensive.

Word-level tokenization has the opposite problem. The vocabulary grows extremely large because every unique word needs its own ID. Managing such a massive vocabulary quickly becomes a bottleneck.

To balance these trade-offs, modern language models use subword tokenization. Instead of relying strictly on characters or full words, this method splits rare or unfamiliar words into smaller, meaningful pieces while keeping common words as single tokens. This approach keeps the vocabulary manageable while preventing sequences from becoming unnecessarily long.

If you want to see how many tokens your prompt consumes, you can use the code snippet below:

import tiktoken
# Background highlight colors
HIGHLIGHTS = [
    “\033[41m”,  # red background
    “\033[42m”,  # green background
    “\033[43m”,  # yellow background
    “\033[44m”,  # blue background
    “\033[45m”,  # magenta background
    “\033[46m”,  # cyan background
]
RESET = “\033[0m”

def highlight_tokens(text: str, model: str = “gpt-4o”):
    try:
        encoding = tiktoken.encoding_for_model(model)
    except KeyError:
        encoding = tiktoken.get_encoding(”cl100k_base”)
    tokens = encoding.encode(text)
    print(f”Total tokens: {len(tokens)}\n”)
    for i, token in enumerate(tokens):
        piece = encoding.decode([token])
        highlight = HIGHLIGHTS[i % len(HIGHLIGHTS)]
        print(f”{highlight}{piece}{RESET}”, end=”“)
    print(”\n”)

prompt = “Hello, explain me about tokenization”

highlight_tokens(prompt)

The output looks like:

It’s evident now that one word can correspond to multiple tokens if required.

But what happens if the prompt contains spelling mistakes?
Consider the sentence: “Hello, explain me about toeknizatoin.”

Since “toeknizatoin” is not a known word, the tokenizer cannot match it to familiar subwords. Instead, it splits the word into multiple smaller fragments. This increases the total number of tokens compared to the correctly spelled word “tokenization.”

Even small spelling mistakes can therefore increase token usage. To keep prompts efficient, ensure they are spelled correctly and clearly written.

Embedding

Computers cannot process words directly — text must first be converted into numbers. But if that’s the case, why aren’t tokens alone sufficient?

Each token is mapped to an integer ID, but integers by themselves carry no semantic meaning. The numbers 42 and 913 don’t inherently tell the model anything about the relationship between the words they represent.

This is where embeddings come in.

An embedding is a dense vector representation of a token or piece of text that captures its semantic meaning. For example:

vector_dog → [0.91, -0.22, 0.55, 0.10]
vector_cat → [0.88, -0.25, 0.59, 0.09]
vector_car → [-0.44, 0.77, -0.12, 0.63]

In high-dimensional space, semantically similar words tend to appear closer together.

vector_dog • vector_cat → high similarity (dot product)
vector_dog • vector_car → low similarity (dot product)

This allows the model to understand relationships between words — something that simple token IDs cannot capture.

Inferencing

LLMs are stateless. When an application communicates with an LLM, the model retains no memory of past conversations. To generate a coherent response, every API call must include the relevant context in the prompt.

This requirement introduces a significant engineering constraint driven by the Transformer architecture.

Self-Attention Matrix

Transformers rely on a self-attention mechanism to determine how each token relates to every other token in the sequence. Because of this, every token must attend to every other token.

As a result, adding more tokens increases the computational complexity quadratically — O(n²).

For example, consider the following sentence (assuming one word equals one token):

Input = [AI, will, change, world]

Attention Matrix Looks like:

Each cell represents how much one token attends to another token.

This means the model computes 4 × 4 = 16 attention relationships for just four tokens. As the number of tokens grows, the number of computations grows rapidly, which is why token and memory management become critical in LLM applications.

Since we have computation constraints, “Context Window” comes into the picture to avoid system degradation.

Context Window and Token Limits

Context window is the maximum number of tokens that LLM can hold in the working memory. Hence, this acts as a hard boundary for input and generated tokens.

Context limits of common LLMs:

Prompt Tokens vs. Completion Tokens

Now, what does a 128K token limit actually mean?

Suppose your input prompt — including the role, task, constraints, and output structure — already consumes 120K tokens. In that case, the LLM has only 8K tokens remaining to generate the response before it reaches the context limit and memory is exhausted.

Context Rot

Most modern LLMs now support much larger context windows. However, relying on extremely large prompts is often an architectural anti-pattern. Large prompts tend to suffer from a phenomenon known as context rot.

The attention mechanism in transformers typically shows high precision at the very beginning of the prompt (the primacy effect, where system instructions usually live) and at the very end of the prompt (the recency effect, where the latest user query appears).

As a result, information placed in the middle of a long prompt gradually loses influence during generation. This degradation of attention to mid-prompt information is what we refer to as Context Rot.

Memory Types in LLM Applications

Since LLMs are inherently stateless, what we call “memory” is not built into the model itself — it’s a system-level abstraction created and managed by the surrounding software architecture.

Short-Term Memory (Conversation History):
This is session-scoped memory that captures the ongoing interaction. Typically, it is maintained as an ordered array of messages representing the back-and-forth between the user and the system.

Long-Term Memory (Semantic & Factual Store):
This layer enables persistence beyond a single session. It stores structured or unstructured information such as user preferences, facts, and organizational knowledge, allowing the system to retain context over time without endlessly expanding the prompt.

Working Memory:
Working memory consists of the immediate tokens being processed during a single inference. It is constrained by the model’s context window and functions similarly to RAM — holding only what is necessary for generating the current response.

This is where the LLM process chain of thoughts.

Together, these layers of memory operate in coordination to produce context-aware, personalized responses for the user.

For AI Engineers, understanding memory in LLM systems is critical because the model itself does not retain any state between requests. This means that everything we call “memory” must be explicitly designed and managed at the system level.

Now, since we know about tokens and memory with regard to LLMs. Let’s talk about strategies to manage them.

Token Management Strategies

To keep prompts within the model’s context window, you need a few guardrails at the code level. The exact approach depends on the use case, but there are a couple of patterns that show up again and again.

Deterministic Token Counting:

Before sending any payload to an LLM, you should programmatically count the tokens to forecast costs and prevent context window overflow. Sample Code:

import tiktoken
class TokenManager:
    “”“Manages token counting and truncation using deterministic BPE encoding.”“”
    
    def __init__(self, model_name: str = “gpt-4o”):
        # Load the exact BPE encoding used by the target model
        self.encoding = tiktoken.encoding_for_model(model_name)
    
    def count_tokens(self, text: str) -> int:
        “”“Returns the exact number of tokens in a string.”“”
        return len(self.encoding.encode(text))
        
    def truncate_to_budget(self, text: str, max_tokens: int) -> str:
        “”“Truncates a string to fit within a specific token limit.”“”
        tokens = self.encoding.encode(text)
        if len(tokens) <= max_tokens:
            return text
        # Slice the token array and decode back to a string
        truncated_tokens = tokens[:max_tokens]
        return self.encoding.decode(truncated_tokens)
# Usage Example
manager = TokenManager()
budgeted_context = manager.truncate_to_budget(”A massive document payload...”, max_tokens=2000)

Sliding Window

This is the simplest way to manage short-term conversational memory. You keep a fixed number of recent messages, and as new ones come in, the oldest ones are dropped from the prompt.

When to use:
Works well for short, task-driven interactions where older context stops being useful pretty quickly.

Downside:
Anything important mentioned early on — like a user’s name or a key constraint — gets lost once it falls out of the window. This is often referred to as “hard amnesia.”

Summarization

To deal with that loss of context, you can introduce a summary layer. Instead of keeping the full conversation, you compress older messages into a concise summary.

The prompt then typically looks like:
System Prompt + Summary + Last N messages (verbatim)

When to use:
Useful for longer conversations — think support bots, assistants, or any system where maintaining continuity matters but you can’t afford to send the entire history every time.

Below is an example of how you might combine both approaches in practice.

class MemoryBufferManager:
    “”“Maintains conversation history, enforcing token limits via sliding window and summarization.”“”
    
    def __init__(self, max_history_tokens: int = 4000):
        self.messages =
        self.summary = “”
        self.max_history_tokens = max_history_tokens
    def add_message(self, role: str, content: str):
        “”“Appends a new message and ensures the budget is maintained.”“”
        self.messages.append({”role”: role, “content”: content})
        self._enforce_budget()
    def _enforce_budget(self):
        “”“Continually evicts oldest messages until under budget.”“”
        while self._calculate_total_tokens() > self.max_history_tokens and len(self.messages) > 1:
            evicted_msg = self.messages.pop(0)
            self._update_summary_async(evicted_msg) 
    def _update_summary_async(self, evicted_message: dict):
        “”“
        In a production environment, this triggers an asynchronous LLM call 
        to merge the evicted_message into self.summary, preventing pipeline blocking.
        “”“
        # Pseudo-logic: self.summary = async_llm_call(f”Summarize: {self.summary} + {evicted_message}”)
        pass
        
    def get_prompt_context(self) -> list:
        “”“Assembles the final context payload for the LLM.”“”
        context =
        if self.summary:
            context.append({”role”: “system”, “content”: f”Prior Conversation Summary: {self.summary}”})
        context.extend(self.messages)
        return context

Semantic Chunking for RAG

When you bring long-term memory into prompts using RAG, large documents can’t be sent as-is. They need to be broken down into smaller chunks first.

A common way to do this is recursive character chunking.

Instead of splitting text randomly, this method follows a hierarchy of separators — like paragraphs, then lines, then words. It typically uses something like ["\n\n", "\n", " ", ""] to break the text step by step.

The idea is simple: keep related content together for as long as possible, so each chunk still makes sense on its own.

from typing import List

class RecursiveTextSplitter:
    “”“Splits text hierarchically to preserve semantic boundaries.”“”
    
    def __init__(self, chunk_size: int):
        self.chunk_size = chunk_size
        # Hierarchy of semantic separators: Paragraph -> Sentence -> Word -> Character
        self.separators = [”\n\n”, “\n”, “. “, “ “, “”]
    def split_text(self, text: str) -> List[str]:
        return self._split_recursive(text, self.separators)
    def _split_recursive(self, text: str, separators: List[str]) -> List[str]:
        # Failsafe for characters if no separators remain
        if not separators:
            return [text[:self.chunk_size]] # Hard slice
        separator = separators
        next_separators = separators[1:]
        
        # Split by current highest-order separator
        splits = text.split(separator)
        
        final_chunks =
        current_chunk = “”
        for split in splits:
            # Reconstruct the text piece with its separator
            piece = split if not current_chunk else separator + split
            
            # If combining the piece keeps us under budget, aggregate it
            if self.token_manager.count_tokens(current_chunk + piece) <= self.chunk_size:
                current_chunk += piece
            else:
                if current_chunk:
                    final_chunks.append(current_chunk)
                # If the individual split itself is too large, recurse deeper
                if self.token_manager.count_tokens(split) > self.chunk_size:
                    final_chunks.extend(self._split_recursive(split, next_separators))
                    current_chunk = “”
                else:
                    current_chunk = split
                    
        if current_chunk:
            final_chunks.append(current_chunk)
            
        return final_chunks

Summary:

Building LLM applications isn’t just about prompts — it’s about disciplined system design. Tokens, memory, and context are your real constraints, and how you manage them defines success.

I’d be curious to know what techniques you’re using in production and what challenges you’ve encountered along the way.

About the Series

This is a 5-article series on Context Engineering:

1. ➡️ Prompt Engineering

2. ✅ Memory and Token Management

3. AI Grounding Strategies and Advanced RAG

4. LLM Evaluation and Ensemble Methods

These series are completely organic and contain a lot of examples from my experiences. I want to refer to all those engineers who are starting their AI journey.

In this article, I have gathered 7 techniques for prompt engineering.

1. Chain of Thought Prompting

2. Role-based Prompting

3. Few-Shot Prompting

4. Constraint Prompting + Negative Prompting

5. Output Formatting Prompting

6. Critic + Generator Pattern

7. HTML-based Prompting

Chain of Thought Prompting (CoT)

CoT is a technique where LLM uses the intermediate steps to generate a comprehensive context. Instead of mapping a prompt directly to an output Prompt -> Answer, CoT inserts a reasoning pathway Prompt -> Reasoning Steps -> Answer

Why CoT?

LLMs are nothing but next token predictors. When you ask them to answer directly, they tend to do all the computation in a single call.

Whereas in CoT, you provide them with more computational space. Also, for the step they are working on, all other steps act as a context.

The AI reads its own reasoning, which heavily biases its next predictions toward the correct logical path.

CoT using ChatGPT Free Tier Model

While most of the models do support step thinking for simple tasks, for your use case, you can define a comprehensive Chain of Thoughts.

Benefits of Using CoT:

• Massive Accuracy Boost: On complex logic, math, and reasoning benchmarks, CoT has been shown to increase AI accuracy by 20% to over 50%.

• Explainability: You are no longer dealing with a “black box.” You can read the AI’s “mind” and see exactly how it arrived at its conclusion.

• Debuggability: If the AI gets the answer wrong, you can look at the chain of thought, find the exact step where the logic failed, and correct your prompt or provide a better example.

While you can use CoT for complex tasks like Analysis, Debugging, PR Review, and Strategic Planning, you can avoid this for simple tasks, such as summarizations.

Role-Based Prompting

This method is where you define a persona or a role for your LLM above the prompt.

To do Role-based prompting, you need to define S.O.A.P:

To move from “Basic” to “Advanced,” you need to go beyond just naming a job. Use the framework:

• Specificity: Add constraints or “vibe” (e.g., “Use data-driven arguments and avoid fluff”).

• Objective: State the goal clearly.

• Persona: Define the title (e.g., “You are a Senior Data Scientist”).

• Audience: Tell the AI who it is talking to (e.g., “Speaking to a board of directors”).

Role-Based Prompting

Why?

When we specify the role, we narrow the probabilistic distribution of word selection. It sets the proper context, filtering, tone calibration, and depth of reasoning.

Here are some small examples that you can use:

Methods for Role-Based Prompting.

Few-Shot Prompting

It is a method where we give some examples in the prompt itself. These examples, known as “shots,” act as a template for the AI to follow.

Few-Shot Prompting Example

Why?

When you are using the Few-Shot Method, you help LLMs to:

• Recognize Patterns: Few-Shot prompting can help more accurately than a 500-word prompt. It helps AI to mimic rather than just calculate.

• Consistent Output: Since we have given a template, the output is also streamlined.

Constraint Prompting

Here, we define clear boundaries, rules, or limitations for LLMs.

Constraint Prompting

Why?

• Precision: It eliminates “AI fluff” and hallucinations.

• Efficiency: You save time on editing because the first draft is already formatted for your needs.

In constraint prompting, you can also define what things you don’t want. That’s called Negative Prompting.

Negative Prompting is used for:

• Anatomy & Quality Control: To avoid the “uncanny valley” (extra fingers, fused limbs, distorted faces).

• Aesthetic Direction: To remove styles you dislike, such as “cartoonish” or “high contrast.”

Negative Prompting

Output Formatting Prompting

In this method, you clearly define the output structure. Instead of a model determining the structure, you provide a mold for output.

Output Formatting Prompting

Why:

• Automation Readiness: If you are building an app, the output structure matters when it comes to reliability and scalability.

• Consistency: You get a consistent output.

Critic + Generator Pattern

In this pattern, we use two distinct roles or agents that work together to achieve a goal in a single prompt.

• Generator: Which creates…

• Critic: Which evaluates…

Critic + Generator Pattern

Why?

• Reduced Hallucinations: The Critic can catch factual errors that the Generator made.

• Higher Reasoning: By forcing the AI to “think twice,” you simulate another layer of reasoning which single-pass prompt often lacks.

HTML-based Prompting

Using HTML tags in prompts can improve how LLMs understand structure and context.

While models don’t actually “render HTML,” they are trained on a huge amount of HTML from the web, so they naturally understand these patterns. Here are the main benefits.

Why?

• LLM understands which text belongs to which Instruction, which reduces instruction mixing.

• Improves prompt readability

• Helps in giving nested instructions as it understands HTML code.

Ensembling Prompting Techniques: Pro Tip

When you’re designing a prompt for you production grade application, it’s very obvious to use a combination of the above techniques.

And this helps you to not only get correct output for the first time but also the 1000th time.

Here is a sample prompt template on how an engineer can apply all these techniques together:

<prompt>

<!-- Role Based Prompting -->
<role>
Senior Backend Engineer
</role>

<objective>
Design a scalable user authentication system.
</objective>

<!-- Chain of Thoughts (CoT) -->
<instruction id="1">
<title>Architecture Exploration</title>

<tasks>
1. Explore 3 architecture options
2. Compare the architectures based on scalability, security, and operational complexity
3. Select the best design
4. Provide a system architecture diagram
5. Provide sample API endpoints
</tasks>

</instruction>

<!-- Critic + Generator Pattern -->
<instruction id="2">
<title>Code Review</title>

Act as a senior code reviewer and critique the authentication implementation.

Check for:
- Password leaks
- Secure password hashing
- Token security
- Session hijacking protection
- Rate limiting
- Handling of authentication edge cases
</instruction>

<instruction id="3">
<title>Refinement</title>

Improve the implementation based on the review feedback and produce the updated version.
</instruction>

<!-- Few Shot Prompting -->
<examples>

<example>
<title>System Diagram</title>

[Client] -> [API Gateway / Middleware] -> [Redis (Atomic Counter)] -> [App Service]

</example>

<example>
<title>API Endpoint</title>

GET /v1/resource  
Returns **429 Too Many Requests** if the bucket is empty.

</example>

</examples>

<!-- Constraint Prompting -->
<constraints>

- Must support 1M users
- Use PostgreSQL as the primary database
- Use Redis for session storage
- Must support horizontal scaling

</constraints>

<!-- Output Formatting Prompting -->
<output_format>

Return the response in **Structured Markdown** with the following sections:

1. Architecture Options
2. Architecture Comparison Table
3. Selected Architecture
4. System Diagram
5. API Endpoints
6. Security Review
7. Improved Implementation

</output_format>

</prompt>

That’s the end of the article. Hope you have found valuable insights. The goal is no longer just to give the model the right answer — it is to build a system that knows exactly which part of its “world” to look at to find that answer itself.

Do share what different techniques you are using for writing optimal prompts.

Stay tuned for Part 2, where we move from static prompts to dynamic, self-evolving context architectures.

Today, engineering managers aren’t looking for “code monkeys” to mechanically translate Jira tickets into syntax. We are looking for Product Engineers. We want holistic developers who understand business value, user experience, and how to leverage AI to multiply their output.

Before I write a single line of code today, I have to ask: “How does this feature impact our MRR? Is the cloud compute cost of this microservice justified by user engagement?”

If you want to be highly competitive in the current market, you don’t need to memorize every syntax quirk of every new framework. Instead, you must deeply understand the underlying concepts, the architectural trade-offs, and what happens when your code actually hits production at scale. You need a “T-shaped” skill set: deep expertise in one specific area, supported by a broad, working knowledge across the rest of the stack.

Here is my pragmatic, battle-tested roadmap for the 2026 software engineer, complete with how the big players are actually using this stuff in the real world.

1. The AI-Augmented Developer (The New Baseline)

Let’s be real: writing every single line of boilerplate code from scratch isn’t a badge of honor anymore — it’s an inefficient use of company time. You must know how to build with AI (to speed up your workflow 5x) and build features using AI (to deliver smarter products).

Core Concepts in the Wild:

• Retrieval-Augmented Generation (RAG): Foundational LLMs are useless for proprietary company knowledge. RAG is how we fetch relevant private data from our databases and inject it into the LLM’s prompt before it generates an answer. It’s vastly cheaper than fine-tuning and kills hallucinations.

• Agentic Workflows: We’ve moved beyond chatbots. Agents are autonomous systems given a goal and tools (like a Python interpreter or database access)

• Real-World Example: Look at Meta. They utilize multi-agent systems internally where “Planner Agents” delegate tasks to “Coder Agents” to automatically review PRs, find security vulnerabilities, and suggest architectural fixes before a human even looks at the code.

• Vector Embeddings: Transforming unstructured data (text, images) into mathematical vectors to search by meaning.

• Real-World Example: Netflix relies heavily on high-dimensional vector embeddings for its personalization engines. When you watch a movie, you aren’t just matching tags; your profile is mathematically clustered near similar content in a latent space, allowing their systems to surface exactly what you want to watch next.

• Token Economics: AI API usage is billed by the “token”. A senior engineer knows when to use a fast, cheap model (like Claude Haiku) for basic routing, and when to bring out the heavyweight models for reasoning.

The Toolkit:

• AI IDEs: Cursor is currently dominating our workflows. GitHub Copilot remains the enterprise standard.

• Orchestration: LangChain or LlamaIndex to build robust RAG pipelines.

2. Backend Engineering & System Architecture

The backend isn’t just about simple CRUD endpoints anymore. It’s about high-performance microservices and event-driven systems capable of handling asynchronous, unpredictable AI workloads.

Core Concepts in the Wild:

• API Architectures (REST vs GraphQL vs gRPC):

• Real-World Example: Netflix is the poster child for microservices and gRPC. When you hit “play,” that request doesn’t go to a monolith; it fans out via gRPC (which uses ultra-fast, tightly compressed Protocol Buffers) to dozens of internal microservices to verify billing, fetch DRM licenses, and locate the nearest CDN node in milliseconds.

• Event-Driven Architecture & Eventual Consistency: Moving away from synchronous requests where Service A waits idly for Service B.

• Real-World Example: LinkedIn literally invented Apache Kafka because their traditional databases couldn’t handle the sheer volume of profile views, messages, and feed updates. By moving to a Publisher/Subscriber (Pub/Sub) model, systems communicate asynchronously. It guarantees eventual consistency — your connection count might take a few seconds to update across the globe, but the system never goes down.

• Idempotency: An operation that produces the same result no matter how many times it’s executed. If a user’s network drops and they mash “Submit Payment” five times, your backend must recognize the duplicate request ID and charge them only once.

The Toolkit:

• Languages: Python (mandatory for AI/Data), Go (for high-performance concurrent cloud services), Node.js/TypeScript, and increasingly, Rust.

• Brokers & Caching: Kafka, RabbitMQ, and Redis.

3. Databases (Traditional & Modern)

Data is the foundation. The volume of unstructured data generated by AI has changed how we store information, but relational integrity is as crucial as ever.

Core Concepts in the Wild:

• CAP Theorem vs. ACID Compliance: You can only guarantee two out of three: Consistency, Availability, and Partition tolerance

• Real-World Example: Think about Amazon on Prime Day. They use DynamoDB (NoSQL) for the shopping cart. Why? Because Availability is everything. If the database partition fails, Amazon would rather risk a momentary inconsistency (a slight delay in updating inventory numbers) than tell a user “Error: Cannot add to cart.” They chose Availability over strict Consistency to save millions in revenue per minute.

• Database Denormalization: While we learn 3rd Normal Form in school, at scale, we often deliberately denormalize (store duplicate data together) to avoid massive, database-locking JOIN operations.

• Real-World Example: Meta built TAO, a distributed data store, specifically to cache and serve their massively denormalized social graph because traditional SQL joins simply couldn’t scale to billions of daily active users fetching complex feeds.

The Toolkit:

• Relational: PostgreSQL is the absolute “Swiss Army Knife” of the industry right now.

• Vector DBs: pgvector, Pinecone, or Milvus for housing billions of AI embeddings.

• ORMs: Prisma or Drizzle for end-to-end type safety in TypeScript environments.

Subscribe now

4. Frontend & Web Development

The exhaustive “framework wars” of the 2010s have largely settled. Today, frontend engineering is obsessed with perceived performance, extreme accessibility, and shipping zero unnecessary JavaScript.

Core Concepts in the Wild:

• Rendering Strategies (CSR, SSR, SSG, RSC): > Real-World Example: Netflix famously removed client-side React from their landing pages. By shifting to Server-Side Rendering (SSR) and vanilla JS for their marketing pages, they drastically cut the Time-to-Interactive (TTI) for users on slow mobile connections.

• State Management Paradigm Shift: We’ve moved away from massive global stores (like legacy Redux) toward simpler local state, treating server data purely as a “cache.”

• Real-World Example: Look at how Meta rebuilt the modern Facebook.com web app. They relied heavily on Relay (GraphQL) to fetch exactly the data a component needs — nothing more, nothing less — drastically reducing the payload size sent to the user’s browser.

• Accessibility (a11y): Ensuring your application is natively usable by people with visual, auditory, or motor disabilities. This is no longer an afterthought; it is a strict legal requirement.

The Toolkit:

• The Non-Negotiable: TypeScript. Writing vanilla JS in enterprise apps is basically professional negligence today.

• Frameworks: React and Next.js (App Router) dominate. Vue/Nuxt is the strongest alternative.

• Styling & Build: Tailwind CSS and Vite.

5. Cloud, DevOps & Security (Infrastructure)

The historical wall between “developer” and “operations” is gone. The modern paradigm is “you build it, you run it.”

Core Concepts in the Wild:

• Deployment Strategies: Pushing updates without taking the system down

• Real-World Example: Amazon executes thousands of deployments a day without anyone noticing, using Canary Rollouts. They deploy a new feature to just 1% of users, monitor their observability dashboards for error spikes, and slowly ramp it up to 100% if metrics stay green.

• Observability (Logs, Metrics, Traces): Knowing exactly why your app is failing on the inside.

• Chaos Engineering: > Real-World Example: Netflix pioneered “Chaos Monkey.” They literally run scripts that randomly kill production servers during business hours. Why? To force their engineers to build systems so resilient and decoupled that the destruction of a random node doesn’t impact the end user’s streaming experience.

• Infrastructure as Code (IaC): Writing declarative code (Terraform) to provision servers instead of clicking through a console.

The Toolkit:

• Containers: Docker and Kubernetes (K8s).

• Cloud & CI/CD: AWS/GCP/Azure, provisioned via Terraform, automated through GitHub Actions.

• Observability: Datadog or Prometheus/Grafana.

Final Thoughts

Don’t try to learn all of this in a weekend. The best engineers learn pragmatically. Build a real application, run into a scaling bottleneck, and then learn the tool that solves it.

The tech stack will change again by 2030, but an engineer who understands system trade-offs, business value, and how to adapt will always be in demand. Keep building.