The LLM engineering roadmap for 2027
Eight phases from Python and APIs to GenAI system design, each with what to learn, what to build and how to tell you are done.
This roadmap has eight phases. Each one says what to learn, what to build, and how to tell you're done, because a list of topics alone doesn't tell you when to move on.
Each phase has four parts:
- Learn: the ideas you need.
- Build: one thing that forces you to use them.
- Done when: a test you can check yourself against.
- Time: a rough guess for someone working 8 to 10 hours a week next to a job.
The order matters. Each phase uses what the one before it built. If you skip RAG and jump to agents, you will write agents that retrieve badly and you won't know why.
Phase 1: Python and model APIs (2 to 3 weeks)#
Learn
- Python you will use every day: type hints, dataclasses,
async/await, virtual environments, reading JSON, writing small CLI tools. - How a chat API call works: messages with roles, system prompts, temperature, max tokens, streaming.
- Tokens: what they are, why they decide your cost and your context limit, and how to count them.
- Errors you will hit in real use: rate limits, timeouts, retries with backoff.
Build
A command-line tool that takes a folder of text files and writes a one-paragraph summary of each to a CSV. Add streaming output, retries and a token count per file.
Done when
You can explain why a request failed from the error alone, and your tool survives being rate-limited halfway through without losing work.
Library: Python Engineering, LLM Foundations
Phase 2: Prompting and structured output (2 weeks)#
Learn
- Writing instructions a model follows: clear task, examples, constraints, output format.
- Structured output: asking for JSON, validating it with Pydantic, and retrying when validation fails.
- Tool or function calling as a way to get structured data, not only to "do things".
- How models fail: format drift, made-up fields, ignored instructions at long context.
Build
An extractor that reads job posts and returns a validated object: title, company, location, salary range (or null), required skills. Write 30 test inputs by hand, including messy ones, and measure how many come back valid on the first try.
Done when
You have a number, such as "27 of 30 valid first time, 30 of 30 after one retry", and you can say which prompt change moved it.
Library: LLMs, Generative AI
Phase 3: Retrieval-augmented generation (3 to 4 weeks)#
Learn
- The pipeline: load, clean, chunk, embed, index, retrieve, rerank, generate with citations.
- Keyword search (BM25) and vector search, and why combining them often helps.
- Chunk size and overlap, metadata filters, and reranking.
- Retrieval metrics (recall@k, MRR) and answer metrics (faithfulness, relevance).
Build
A question-answering bot over a real document set you care about: your company's public docs, a textbook, a set of RFCs. Every answer must cite the chunk it came from. Build a test set of 40 questions with the right source passage marked.
Done when
You can show retrieval recall@5 on your test set, change one thing (chunk size, hybrid search, a reranker) and show whether the number went up or down.
If you want a starting point, my RAG cheat sheet lists the stages and a symptom-to-fix table, and the PDF question-answering bot is a small working version.
Library: RAG, Search & Relevance
Phase 4: Agents and MCP (3 to 4 weeks)#
Learn
- The agent loop: the model picks a tool, your code runs it, the result goes back, repeat until done.
- When you need an agent and when a fixed workflow is better. Most tasks need the workflow.
- State, memory and stopping conditions. Human approval before actions that cost money or send messages.
- Model Context Protocol (MCP): exposing tools and data to any compatible client through one server.
- A graph framework such as LangGraph for branching, retries and checkpoints.
Build
An agent with three to five tools over something real: your calendar, a SQLite database, a public API. Put one tool behind a human approval step. Then expose the same tools as an MCP server and call them from a different client.
Done when
You can show a trace of the agent's steps for a task, point to where it went wrong on a failed run, and fix it without rewriting the whole prompt.
Read next: MCP from zero.
Library: AI Agents, Deep Agents, Claude
Phase 5: Evals (2 to 3 weeks)#
I put evals after agents on purpose. By now you have three systems that work some of the time, and you need a way to say how often.
Learn
- Golden sets: inputs with expected outputs or expected behaviour, built by hand at first.
- Code-based checks (exact match, schema valid, right tool called) and model-graded checks (LLM as judge), and when each is trustworthy.
- Checking your judge against human labels before you rely on it.
- Running evals on every prompt or model change, the way you run unit tests.
Build
An eval harness for your Phase 4 agent: 25 cases, checks on tool choice, arguments and final answer, and a report with a pass rate per check. Make it fail a build when the pass rate drops.
Done when
You change the model or a prompt, run one command, and get a yes-or-no answer about whether things got worse.
Read next: Evaluating an AI agent in Python.
Library: Evals & LLMOps
Phase 6: Serving, inference and cost (2 to 3 weeks)#
Learn
- What happens at inference: prefill and decode, KV cache, batching, why the first token is slow.
- Running open models yourself with an inference server (vLLM, llama.cpp, Ollama) and what quantization trades away.
- Latency numbers that matter: time to first token, tokens per second, p95.
- Cost control: caching, prompt trimming, routing easy requests to a smaller model.
Build
Serve one open model locally and put a small API in front of it. Measure time to first token and throughput at 1, 4 and 16 concurrent requests. Then add a response cache and a router that sends short, simple prompts to a smaller model.
Done when
You can say what a request costs and how long it takes at p95, and show one change that cut either number.
Library: Inference Engineering, MLOps
Phase 7: Security and guardrails (1 to 2 weeks)#
Learn
- Prompt injection, direct and indirect (through retrieved documents and tool results).
- Data leakage: system prompts, other users' data, secrets in context.
- Limiting what tools can do: read-only by default, allow-lists, approval for writes.
- Input and output filters, and their limits.
Build
Attack your own Phase 3 and Phase 4 systems. Plant instructions inside a document your RAG bot retrieves. Try to make your agent call a tool it shouldn't. Write down each attack, whether it worked and what you changed.
Done when
You have a short list of attacks that used to work and now don't, and you can explain why a filter alone is not enough.
Library: AI Security, Responsible AI
Phase 8: GenAI system design (ongoing)#
Learn
- Putting the pieces together under real limits: latency budget, cost budget, data privacy, failure handling.
- Where state lives, what gets cached, what runs async, how you roll back a bad prompt.
- How to talk through a design: requirements first, then a simple version, then what breaks at scale.
Build
Write design docs for two systems you have not built, for example a support assistant for a bank or a code review bot for a team of 200. One page each: requirements, diagram, components, evals, risks. Then take one of your earlier projects and write the doc for its next version.
Done when
You can talk through a design for 30 minutes, take questions, and change your answer when a requirement changes without starting over.
Library: GenAI System Design, ML System Design
The whole plan on one page#
| Phase | Focus | Build | Time |
|---|---|---|---|
| 1 | Python and APIs | Batch summarizer CLI | 2 to 3 weeks |
| 2 | Prompting, structured output | Validated job-post extractor | 2 weeks |
| 3 | RAG | Cited Q&A bot with a test set | 3 to 4 weeks |
| 4 | Agents and MCP | Tool-using agent + MCP server | 3 to 4 weeks |
| 5 | Evals | Eval harness that fails a build | 2 to 3 weeks |
| 6 | Serving and cost | Local model with cache and router | 2 to 3 weeks |
| 7 | Security | Attack log for your own systems | 1 to 2 weeks |
| 8 | System design | Two design docs | Ongoing |
Phases 1 to 7 add up to 15 to 21 weeks, so roughly four to five months at 8 to 10 hours a week, with Phase 8 running alongside. If you already write Python at work, Phase 1 takes days, not weeks.
What I'd skip, at least at first#
- Training models from scratch. It's worth understanding how a transformer works (the "How LLMs work" video path covers it), but most LLM engineering jobs build on models, not train them.
- Fine-tuning before you've measured prompting and RAG. Fine-tune when you have evals that show the cheaper options have hit a wall.
- Learning three agent frameworks. Learn one well. The ideas carry over.
- Chasing every new model release. Your eval harness will tell you whether a new model matters for your use case.
How to show this work#
Each phase leaves you with a project and a number. Put both in the README: what it does, how you measured it, what you changed and what happened. One honest table with a before and after counts for more in an interview than a long list of tools.
If you prefer a project list to a phase plan, 12 AI engineering projects that prove you can ship covers similar ground organised by project.