GyaanStack. Blog

Articles / agentslanggraphprojects

Build a multi-agent research assistant with LangGraph that runs locally

A planner, parallel researchers, a writer and a reviewer in one LangGraph graph, running on Ollama with a local corpus and no API key.

By Raghunath · · 11 min read

This post builds a small research assistant from four kinds of agents:

  • a planner that splits a question into two or three search questions,
  • researchers that run in parallel, one per search question, over a local set of notes,
  • a writer that drafts an answer with citations,
  • a reviewer that sends the draft back to the writer if it breaks the rules, up to a fixed number of times.

Everything runs on my laptop: LangGraph for the graph, Ollama for the model, and four Markdown files as the "library". There is no web search and no API key. I picked a local corpus on purpose. When the sources are files you wrote, you can check every citation the system produces, which you can't do with live search results.

I tested this with Python 3.15, langgraph 1.2.24, langchain-ollama 1.1.0 and gemma3:12b in Ollama.

Setup#

ollama pull gemma3:12b
python -m venv .venv && source .venv/bin/activate
pip install langgraph langchain-ollama

Make a corpus/ folder with a few short Markdown files. Mine are four notes on retrieval: vector_dbs.md, bm25.md, reranking.md and costs.md, each three or four short paragraphs. For example, reranking.md:

# Reranking
A reranker is a cross-encoder that reads the query and one passage together and outputs a relevance score. It is slower than vector search, so it runs only on the top 20 to 100 candidates.
Adding a reranker after hybrid retrieval is one of the cheapest ways to improve answer quality in a RAG system.

Write your own notes on any topic you know. The code doesn't care what they say.

The search tool#

All the Python in this post goes in one file, research.py, in the same folder as corpus/.

The researchers need something to search. To keep the post about the agent graph, I used the simplest search that works: split each file into paragraphs and score each paragraph by how many query words it contains.

import json
import operator
import re
from pathlib import Path
from typing import Annotated, TypedDict

from langchain_ollama import ChatOllama
from langgraph.graph import END, START, StateGraph
from langgraph.types import Send

MODEL = "gemma3:12b"
MAX_REVISIONS = 2
llm = ChatOllama(model=MODEL, temperature=0)

DOCS = {p.name: p.read_text() for p in Path("corpus").glob("*.md")}

def search(query: str, k: int = 2) -> list[tuple[str, str]]:
    """Score each paragraph by how many query words it contains."""
    words = set(re.findall(r"[a-z0-9]+", query.lower())) - {"the", "a", "of", "and", "to", "is", "for", "what", "how"}
    hits = []
    for name, text in DOCS.items():
        for para in text.split("\n"):
            if para.startswith("#") or not para.strip():
                continue
            score = sum(w in para.lower() for w in words)
            if score:
                hits.append((score, name, para))
    hits.sort(reverse=True)
    return [(name, para) for _, name, para in hits[:k]]

Swap this for BM25 or a vector store later. Each researcher only needs a function that takes a string and returns (source, text) pairs.

The shared state#

LangGraph passes one state dictionary between nodes. The field to look at is notes:

class State(TypedDict):
    question: str
    sub_questions: list[str]
    notes: Annotated[list[dict], operator.add]   # parallel researchers append here
    draft: str
    feedback: str
    revisions: int
    approved: bool

class ResearchTask(TypedDict):
    sub_question: str

Several researchers write to notes at the same time. Without the operator.add annotation, the last researcher to finish would overwrite the others. With it, LangGraph concatenates their lists. If your parallel branches seem to lose results, check for a missing reducer first.

ResearchTask is the small private input each researcher gets. A researcher doesn't need the whole state, only its one search question.

The planner and the fan-out#

The planner asks the model for JSON and parses it:

def planner(state: State) -> dict:
    prompt = (
        "Split this question into 2 or 3 short search questions. "
        'Reply with JSON only: {"sub_questions": [...]}\n\n'
        f"Question: {state['question']}"
    )
    raw = llm.invoke(prompt).content
    subs = json.loads(re.search(r"\{.*\}", raw, re.S).group(0))["sub_questions"][:3]
    print(f"[planner] {subs}")
    return {"sub_questions": subs, "revisions": 0}

def fan_out(state: State) -> list[Send]:
    return [Send("researcher", {"sub_question": q}) for q in state["sub_questions"]]

The regex pulls out the first {...} block because models often wrap JSON in a code fence or add a sentence before it. The [:3] cap stops a chatty model from creating ten researchers.

fan_out is the parallel part. It returns a list of Send objects, one per sub-question. LangGraph runs a researcher node for each and waits for all of them before it moves on. The number of researchers is decided at run time by the planner, not fixed in the graph.

Researchers and the writer#

def researcher(task: ResearchTask) -> dict:
    hits = search(task["sub_question"])
    print(f"[researcher] {task['sub_question']!r} -> {[n for n, _ in hits]}")
    return {"notes": [{"q": task["sub_question"], "source": n, "text": t} for n, t in hits]}

def writer(state: State) -> dict:
    notes = "\n".join(f"[{n['source']}] {n['text']}" for n in state["notes"])
    prompt = (
        "Answer the question in at most 120 words using ONLY the notes. "
        "Cite sources in square brackets like [bm25.md] after each claim.\n\n"
        f"Question: {state['question']}\n\nNotes:\n{notes}"
    )
    if state.get("feedback"):
        prompt += f"\n\nA reviewer rejected your last draft. Fix this: {state['feedback']}\n\nLast draft:\n{state['draft']}"
    draft = llm.invoke(prompt).content.strip()
    print(f"[writer] draft {state['revisions'] + 1}, {len(draft.split())} words")
    return {"draft": draft, "revisions": state["revisions"] + 1}

The researcher here doesn't call the model at all. A "researcher agent" doesn't have to be an LLM call. If plain code does the job, use plain code: it's faster, it's free, and you can test it.

On a revision, the writer gets the reviewer's feedback and its own last draft. Without the last draft, it would start over and might break something that was already right.

The reviewer: code first, model second#

def reviewer(state: State) -> dict:
    # Cheap deterministic checks first; the model only judges what code can't.
    cited = set(re.findall(r"\[([\w.-]+\.md)\]", state["draft"]))
    known = {n["source"] for n in state["notes"]}
    if not cited:
        return {"approved": False, "feedback": "Add [source.md] citations after each claim."}
    if cited - known:
        return {"approved": False, "feedback": f"These sources are not in the notes: {sorted(cited - known)}."}
    if len(state["draft"].split()) > 140:
        return {"approved": False, "feedback": "Too long. Stay under 120 words."}
    verdict = llm.invoke(
        "Does the draft answer the question and stay within the notes? "
        'Reply with JSON only: {"ok": true or false, "fix": "one sentence"}\n\n'
        f"Question: {state['question']}\n\nDraft:\n{state['draft']}"
    ).content
    v = json.loads(re.search(r"\{.*\}", verdict, re.S).group(0))
    print(f"[reviewer] ok={v['ok']} fix={v.get('fix', '')!r}")
    return {"approved": bool(v["ok"]), "feedback": v.get("fix", "")}

def route_after_review(state: State) -> str:
    if state["approved"] or state["revisions"] >= MAX_REVISIONS:
        return END
    return "writer"

I run the rule checks in code before asking the model anything. Code can tell for certain whether a draft has citations, whether each cited file was actually retrieved, and whether the draft is too long. An LLM judge can get any of those wrong. The model only answers the question code can't: does the draft answer the question?

The check for invented sources is the most useful one. A writer model will sometimes cite a plausible file name that was never in its notes. Here is that check on its own, with two bad drafts:

from research import reviewer

notes = [{"q": "x", "source": "bm25.md", "text": "BM25 is strong on exact terms."}]
for draft in [
    "BM25 is good at exact terms.",                    # no citation
    "BM25 is good at exact terms [splade.md].",        # invented source
]:
    print(reviewer({"question": "When is BM25 useful?", "draft": draft, "notes": notes}))
{'approved': False, 'feedback': 'Add [source.md] citations after each claim.'}
{'approved': False, 'feedback': "These sources are not in the notes: ['splade.md']."}

Neither draft gets to the model.

route_after_review contains the loop limit. MAX_REVISIONS = 2 means the writer gets at most two attempts in total. Without a limit, a strict reviewer and a stubborn writer can loop until you run out of time or money. Once the limit is reached, the graph returns the last draft with approved=False, and the caller decides what to do with it.

Wiring the graph#

g = StateGraph(State)
g.add_node("planner", planner)
g.add_node("researcher", researcher)
g.add_node("writer", writer)
g.add_node("reviewer", reviewer)
g.add_edge(START, "planner")
g.add_conditional_edges("planner", fan_out, ["researcher"])
g.add_edge("researcher", "writer")
g.add_edge("writer", "reviewer")
g.add_conditional_edges("reviewer", route_after_review, ["writer", END])
app = g.compile()

if __name__ == "__main__":
    q = "Should a small team use pgvector or a dedicated vector database, and how can they improve retrieval quality cheaply?"
    out = app.invoke({"question": q, "notes": []})
    print("\n--- answer ---")
    print(out["draft"])
    print(f"\nrevisions={out['revisions']} approved={out['approved']}")

That gives you the whole flow: planner, fan-out, researchers, writer, reviewer, and either back to the writer or end.

A real run#

[planner] ['pgvector vs dedicated vector database for small teams: pros and cons', 'Cost-effective methods to improve vector search retrieval quality', 'pgvector limitations for small teams and scaling']
[researcher] 'pgvector vs dedicated vector database for small teams: pros and cons' -> ['vector_dbs.md', 'costs.md']
[researcher] 'pgvector limitations for small teams and scaling' -> ['vector_dbs.md', 'vector_dbs.md']
[researcher] 'Cost-effective methods to improve vector search retrieval quality' -> ['reranking.md', 'costs.md']
[writer] draft 1, 85 words
[reviewer] ok=True fix='The answer effectively addresses both parts of the question and references the provided notes.'

--- answer ---
For small teams, pgvector within PostgreSQL is often sufficient, especially if already using Postgres [vector_dbs.md]. It avoids the cost and complexity of a separate vector database [costs.md]. Dedicated vector databases like Qdrant or Milvus offer advanced features like sharding, but are likely unnecessary for smaller datasets (under a few million vectors) [vector_dbs.md].

To cheaply improve retrieval quality, implement a reranker after the initial vector search [reranking.md]. Embedding the corpus is a one-time expense, while query costs include embedding the query, vector search, and reranking [costs.md].

revisions=1 approved=True

The whole run took about 25 seconds on my machine. Three things in the output are worth looking at.

The research logs finish in a different order from the plan: the third sub-question's researcher finished before the second one. That's the parallelism working.

The planner made a weak third sub-question ("pgvector limitations…"), and its search returned the same file twice. The writer coped because the notes were short. With a large corpus, you'd remove duplicate notes before the writer sees them.

There's a real citation slip that my reviewer missed. The sentence "It avoids the cost and complexity of a separate vector database [costs.md]" is closer to vector_dbs.md, which says teams on Postgres "can avoid a new service". costs.md only says self-hosted pgvector costs the database server. The citation points to a real file, so it passes my checks, but the claim leans on a different file. Catching that needs a per-sentence check: for each sentence, test whether the cited paragraph supports it. That's the next thing I'd add to the reviewer, and the reason I'd always read a few outputs by hand before trusting the green light.

Where to take it next#

  • Better search. Replace the word-overlap scorer with BM25 or embeddings. The graph doesn't change.
  • Per-claim verification. Split the draft into sentences and ask the model, for each one, whether the cited note supports it. This catches the slip above.
  • Remove duplicate notes by (source, text) before the writer sees them.
  • An eval set. Write ten questions with known answers from your corpus, run the graph on all of them, and track how many pass and how many revisions each needs. When you change a prompt, run the set again.
  • Human review. Compile with a checkpointer and interrupt_before=["writer"] (g.compile(checkpointer=MemorySaver(), interrupt_before=["writer"])) so the run pauses after research. You read the notes, then resume the same thread to let the writer continue.

The planner, fan-out and review-loop structure is the part to keep. The pieces inside each node are small and easy to swap out.