The LLM tech stack in 2026: every layer, the real options, and how to choose
The eight layers of a production LLM app in late 2026, from models and serving to MCP, evals, guardrails and gateways, with current options and how to pick.
Articles / tag
18 articles.
The eight layers of a production LLM app in late 2026, from models and serving to MCP, evals, guardrails and gateways, with current options and how to pick.
Build RAG where documents and questions are in different languages. Real retrieval and token numbers for Hindi, Hinglish and Tamil, plus answer-language checks.
When graph retrieval beats vector search, how entity extraction and community summaries work, a runnable networkx pipeline, and what indexing costs.
Single-agent architectures compared with diagrams, model calls and latency per task, a decision table and runnable code running two patterns on one task.
Design an AI support agent end to end: scale and cost math, intent routing, RAG, refund tools with permission tiers, human handoff, guardrails, evals, rollout.
Five RAG architectures side by side, with diagrams, latency and cost estimates, failure modes, how to evaluate each, and when to upgrade.
Supervisor, pipeline, hierarchical, fan-out, handoff, critic loop and blackboard, each with a diagram, LLM-call cost, failure modes and how to test it.
Where MCP, A2A and ANP each sit in one agent system, with current spec versions, real JSON messages, security, latency, a decision table and runnable Python.
Where LLM guardrails sit, what each layer costs in latency and money, prompt injection, PII, tool authorization, tracing with OpenTelemetry and alerts.
How to design memory for an AI agent: what to store, when to write it, how to retrieve it, how to handle conflicts and deletion, and how to test it.
How an AI coding agent like Claude Code or Codex CLI works inside: the agent loop, edit formats, repo context, sandboxing, verification, cost per task and evals.
Design a text-to-SQL agent for a large warehouse: schema retrieval, semantic layer, verified queries, clarifying questions, permissions, cost math and evals.
A system design for real-time voice agents: cascade vs speech-to-speech, an 800 ms latency budget, barge-in, telephony, cost per minute and evals.
How to design an AI agent that runs for hours or days: triggers, a persisted plan, checkpoints, idempotent tools, approval gates, budgets and evals.
How to design a computer-use or browser agent: screenshots vs accessibility tree, loop latency, sandboxing, prompt injection, cost per task and evaluation.
The write side of RAG: connectors, change detection, parsing, chunking, embedding cost math, index memory, blue/green re-indexing and monitoring.
How to ship an LLM app safely: what counts as a change, versioned prompts, eval gates with real cost math, handling noisy scores, canaries and rollback.
Five AI system design interview questions worked through: RAG ingestion, RAG query, a coding agent, a support agent and a text-to-SQL agent.