GyaanStack. Blog

Articles / evalsllmopspythonprojects

Build an LLM evaluation dashboard in Python with Streamlit

A fixed dataset, repeatable runs stored in SQLite, prompt-version comparison and a Streamlit page that shows which cases regressed. Runs offline.

By Raghunath · · 18 min read

You change a prompt, ask the app three questions, and the answers look better. Then a user finds the question that got worse. An evaluation dashboard exists to stop that. Every change runs against the same fixed set of questions, every run is saved, and the page answers two questions: is the new version better than the old one, and which cases did it break?

This post builds one in about 200 lines of Python: a JSONL dataset, a runner that stores every run in SQLite, a comparison script that can fail a CI job, and a Streamlit page on top. The model is a deterministic stub, so everything runs offline with no API key. I show where the real model call goes.

Two earlier posts cover pieces this one leans on. Evaluating an AI agent in Python builds a golden set and checks tool calls for an agent, and explains how to grow the set from real failures. The LLM cost and latency tracker records tokens, cost and p50/p95 latency for every production call. Here the focus is narrower: comparing versions of an app over time on a dataset that does not change.

Tested with Python 3.15.2, Streamlit 1.67.0 and pandas 3.1.1 in a venv.

How the pieces fit#

dataset.jsonl ──┐
prompts.py ─────┼──> run_eval.py ──> evals.db (runs + results)
model.py ───────┘                        │
                         ┌───────────────┴──────────────┐
                         v                              v
                   compare.py A B                    app.py
                 (exit 1 on regression)      (trend, metrics, per-case diff)

The runner does not know about the dashboard, and the dashboard never calls the model. It only reads the database. That split means you can run evals from a laptop, a cron job or CI, and the page shows all of them.

python3 -m venv .venv
.venv/bin/pip install streamlit pandas

A fixed dataset with a fingerprint#

The dataset is a JSONL file in git. Each line has a question, the context the app would retrieve, and what a correct answer must contain. Two cases have no answer in the context at all. Those test whether the app admits it does not know, which is the failure that hurts most in a support bot.

{"id": "refund-window", "question": "How many days do customers have to ask for a refund?", "context": "Customers can request a refund within 30 days of purchase. Refunds go back to the original payment method.", "must_include": [["30 days", "thirty days"]], "should_refuse": false}
{"id": "refund-method", "question": "Where does refunded money go?", "context": "Customers can request a refund within 30 days of purchase. Refunds go back to the original payment method.", "must_include": [["original payment method"]], "should_refuse": false}
{"id": "plan-seats", "question": "How many seats does the Team plan include?", "context": "The Team plan includes 10 seats. Extra seats cost 400 rupees per month each.", "must_include": [["10 seats", "ten seats"]], "should_refuse": false}
{"id": "extra-seat-price", "question": "What does an extra seat cost?", "context": "The Team plan includes 10 seats. Extra seats cost 400 rupees per month each.", "must_include": [["400"], ["per month", "a month", "monthly"]], "should_refuse": false}
{"id": "sso", "question": "Which plans support single sign-on?", "context": "Single sign-on is available on the Business and Enterprise plans. It supports SAML and OIDC.", "must_include": [["Business"], ["Enterprise"]], "should_refuse": false}
{"id": "data-region", "question": "Can I keep my data in India?", "context": "All customer data is stored in the Mumbai region by default. Customers on Enterprise can pick Frankfurt instead.", "must_include": [["Mumbai"]], "should_refuse": false}
{"id": "phone-support", "question": "Is there phone support on weekends?", "context": "Support is available by email from 9am to 6pm IST, Monday to Friday.", "must_include": [], "should_refuse": true}
{"id": "student-discount", "question": "Do you offer a student discount?", "context": "The Team plan includes 10 seats. Extra seats cost 400 rupees per month each.", "must_include": [], "should_refuse": true}

must_include is a list of groups. Every group must match, and any string inside a group counts. So extra-seat-price needs "400" and also one of "per month", "a month" or "monthly". This lets you accept reasonable rewordings without accepting anything.

The runner hashes the file and stores the hash with each run. If someone edits a case, later runs get a new hash, and the dashboard refuses to compare runs across it. Without this check, a pass rate that went from 75% to 88% might mean the app improved or that someone deleted a hard case.

Eight cases is a demo. With eight, one case is 12.5 points of pass rate. Start with 30 to 50 real questions from logs and grow from there.

The model behind one function#

Everything that talks to the model goes through generate(prompt) -> str. The stub picks the context sentence that shares the most words with the question. It has two behaviours worth testing: with a weak match it either invents an answer or, if the prompt allows it, refuses. When the prompt asks for short answers it cuts the reply to seven words.

# model.py
"""The model sits behind one function: generate(prompt) -> str.

FakeModel below is a deterministic stub so the pipeline runs offline with no
API key. To use a real model, replace generate() with your provider call, e.g.
with Ollama:  ollama.chat(model=..., messages=[{"role": "user", "content": prompt}]).message.content
"""
import re
import time

STOP = {"the", "a", "an", "do", "does", "is", "are", "what", "how", "many", "which",
        "can", "i", "my", "you", "there", "on", "to", "of", "in", "where", "have", "for"}


def _words(text):
    return {w for w in re.findall(r"[a-z0-9]+", text.lower()) if w not in STOP}


class FakeModel:
    name = "fake-model-0"

    def generate(self, prompt: str) -> str:
        context = prompt.split("Context:\n")[1].split("\n\nQuestion:")[0]
        question = prompt.split("Question: ")[1]
        sentences = [s.strip() for s in context.split(".") if s.strip()]
        # pick the context sentence that shares most words with the question
        best = max(sentences, key=lambda s: len(_words(s) & _words(question)))
        overlap = len(_words(best) & _words(question))
        if overlap < 2 and "not contain the answer" in prompt:
            answer = "I don't have enough information."
        elif overlap < 2:
            answer = "Yes, that is available on all plans."   # confident and wrong
        else:
            answer = best + "."
        if "under eight words" in prompt:
            answer = " ".join(answer.split()[:7]).rstrip(".") + "."
        time.sleep(0.01 + 0.003 * len(answer.split()))  # pretend to be slow
        return answer

The comment at the top marks where a real model goes. The Ollama Python client returns the text at response.message.content. A hosted API needs a key and a different field name, and nothing else in the pipeline changes.

Prompt versions as data#

Prompts live in one dictionary, keyed by version. The runner stores the version name and a hash of the template text, so if someone edits v2 in place without renaming it, the hash changes and you can see it in the runs table.

# prompts.py
PROMPTS = {
    "v1": (
        "Answer the question using the context.\n\n"
        "Context:\n{context}\n\nQuestion: {question}"
    ),
    "v2": (
        "Answer the question using only the context. If the context does not "
        "contain the answer, reply exactly: I don't have enough information.\n\n"
        "Context:\n{context}\n\nQuestion: {question}"
    ),
    "v3": (
        "Answer the question using only the context. If the context does not "
        "contain the answer, reply exactly: I don't have enough information. "
        "Keep the answer under eight words.\n\n"
        "Context:\n{context}\n\nQuestion: {question}"
    ),
}

v2 adds a refusal rule. v3 asks for shorter answers, the kind of change someone makes to cut output tokens.

Scoring: checks first, similarity as a footnote#

A common first scorer is string similarity against a reference answer, for example difflib.SequenceMatcher. Here is why it should not decide pass or fail:

# similarity_demo.py
from difflib import SequenceMatcher

reference = "Customers can request a refund within 30 days of purchase."
answers = {
    "right, reworded": "You have a month after buying to ask for your money back.",
    "wrong number":    "Customers can request a refund within 90 days of purchase.",
}
for label, answer in answers.items():
    ratio = SequenceMatcher(None, answer.lower(), reference.lower()).ratio()
    print(f"{label:<16} {ratio:.2f}")
right, reworded  0.31
wrong number     0.98

The correct answer scores 0.31 and the wrong one scores 0.98, because one changed character is all that separates them. ratio() measures how many characters two strings share, and it has no idea which ones matter.

So a case passes on two checks. The required strings must be present, and the app must refuse exactly when it should. Similarity is still recorded, because a sudden drop across many cases is a hint that the answers changed shape, but it does not gate anything. For answers that are too open for string checks, an LLM judge is the next step; the agent eval post covers when to add one.

The runner#

# run_eval.py
import hashlib
import json
import sqlite3
import sys
import time
import uuid
from difflib import SequenceMatcher

from model import FakeModel
from prompts import PROMPTS

REFUSAL = "don't have enough information"
DB_PATH = "evals.db"


def sha(text):
    return hashlib.sha256(text.encode()).hexdigest()[:10]


def load_dataset(path="dataset.jsonl"):
    raw = open(path, encoding="utf-8").read()
    return [json.loads(line) for line in raw.splitlines() if line.strip()], sha(raw)


def score(case, answer):
    low = answer.lower()
    refused = REFUSAL in low
    refusal_ok = refused == case["should_refuse"]
    keywords_ok = all(any(alt.lower() in low for alt in group) for group in case["must_include"])
    reference = case["context"] if not case["should_refuse"] else REFUSAL
    similarity = SequenceMatcher(None, low, reference.lower()).ratio()
    return {"passed": refusal_ok and keywords_ok, "keywords_ok": keywords_ok,
            "refusal_ok": refusal_ok, "similarity": round(similarity, 3)}


def connect():
    db = sqlite3.connect(DB_PATH)
    db.executescript("""
    CREATE TABLE IF NOT EXISTS runs (run_id TEXT PRIMARY KEY, ts REAL, prompt_version TEXT,
        prompt_hash TEXT, model TEXT, dataset_hash TEXT, n INTEGER, note TEXT);
    CREATE TABLE IF NOT EXISTS results (run_id TEXT, case_id TEXT, answer TEXT, passed INTEGER,
        keywords_ok INTEGER, refusal_ok INTEGER, similarity REAL, latency_ms REAL);
    """)
    return db


def run(prompt_version, note=""):
    cases, dataset_hash = load_dataset()
    template = PROMPTS[prompt_version]
    model = FakeModel()
    run_id = uuid.uuid4().hex[:8]
    rows = []
    for case in cases:
        prompt = template.format(context=case["context"], question=case["question"])
        t0 = time.perf_counter()
        try:
            answer = model.generate(prompt)
        except Exception as exc:          # a crash is a failed case, not a dead run
            answer = f"ERROR: {exc!r}"
        latency_ms = (time.perf_counter() - t0) * 1000
        s = score(case, answer)
        rows.append((run_id, case["id"], answer, s["passed"], s["keywords_ok"],
                     s["refusal_ok"], s["similarity"], round(latency_ms, 1)))
    db = connect()
    db.execute("INSERT INTO runs VALUES (?,?,?,?,?,?,?,?)",
               (run_id, time.time(), prompt_version, sha(template), model.name,
                dataset_hash, len(cases), note))
    db.executemany("INSERT INTO results VALUES (?,?,?,?,?,?,?,?)", rows)
    db.commit()
    passed = sum(r[3] for r in rows)
    print(f"run {run_id}  prompt {prompt_version} ({sha(template)})  dataset {dataset_hash}  "
          f"passed {passed}/{len(rows)}")
    return run_id


if __name__ == "__main__":
    run(sys.argv[1], note=" ".join(sys.argv[2:]))

Each run gets a row in runs with the prompt hash, model name and dataset hash, and one row per case in results with the raw answer. Keeping the raw answer is what makes the dashboard useful later: a pass rate tells you something moved, and the answer text tells you why.

Four runs, in the order a team might make them (v2 twice, to check the run repeats):

python run_eval.py v1
python run_eval.py v2
python run_eval.py v2 rerun
python run_eval.py v3
run 13da66f7  prompt v1 (f7dbd42f97)  dataset 8e37522ed9  passed 4/8
run e16d5e00  prompt v2 (7a706d2977)  dataset 8e37522ed9  passed 6/8
run 05f6dfb2  prompt v2 (7a706d2977)  dataset 8e37522ed9  passed 6/8
run 97333b19  prompt v3 (b5efc4bbb2)  dataset 8e37522ed9  passed 4/8

The stub is deterministic, so the v2 rerun matches. A real model at non-zero temperature will not always match, and a one-case swing on a small set is noise. Run each version several times and compare the mean and the worst run, as shown in the agent eval post.

Comparing two runs, per case#

The pass rate is the wrong unit for a comparison. Two runs can have the same pass rate with different cases passing. compare.py lines up two runs case by case and labels each one as fixed, regressed or the same. Run from the command line, it exits with status 1 when anything regressed, so a CI job can block the merge.

# compare.py
import sqlite3
import sys

import pandas as pd

DB_PATH = "evals.db"


def load(db_path=DB_PATH):
    db = sqlite3.connect(db_path)
    runs = pd.read_sql("SELECT * FROM runs ORDER BY ts", db)
    results = pd.read_sql("SELECT * FROM results", db)
    return runs, results


def diff(results, base_id, cand_id):
    base = results[results.run_id == base_id].set_index("case_id")
    cand = results[results.run_id == cand_id].set_index("case_id")
    both = base[["passed", "answer"]].merge(cand[["passed", "answer"]], left_index=True,
                                            right_index=True, suffixes=("_base", "_cand"))
    both["change"] = "same"
    both.loc[(both.passed_base == 1) & (both.passed_cand == 0), "change"] = "REGRESSED"
    both.loc[(both.passed_base == 0) & (both.passed_cand == 1), "change"] = "fixed"
    return both


def gate(base_id, cand_id):
    runs, results = load()
    hashes = runs.set_index("run_id").loc[[base_id, cand_id], "dataset_hash"]
    if hashes.nunique() != 1:
        sys.exit("runs used different datasets; the comparison means nothing")
    d = diff(results, base_id, cand_id)
    print(d[d.change != "same"][["change", "answer_base", "answer_cand"]].to_string())
    regressed = (d.change == "REGRESSED").sum()
    print(f"\n{regressed} regressed, {(d.change == 'fixed').sum()} fixed")
    return 1 if regressed else 0


if __name__ == "__main__":
    sys.exit(gate(sys.argv[1], sys.argv[2]))

Comparing v2 with v3:

python compare.py e16d5e00 97333b19; echo "exit $?"
                  change                                                        answer_base                                   answer_cand
case_id
refund-window  REGRESSED         Customers can request a refund within 30 days of purchase.     Customers can request a refund within 30.
sso            REGRESSED  Single sign-on is available on the Business and Enterprise plans.  Single sign-on is available on the Business.

2 regressed, 0 fixed
exit 1

The shorter-answer prompt cut "days" from the refund answer and dropped "Enterprise" from the SSO answer. Both are real losses for a user, and the answer columns show them without anyone rerunning anything.

The dashboard#

The Streamlit page reads the database and shows three things: pass rate per run over time, headline metrics for a candidate run against a baseline, and the per-case diff with regressions sorted to the top.

# app.py
import streamlit as st

from compare import diff, load

st.set_page_config(page_title="LLM eval dashboard", layout="wide")
st.title("LLM eval dashboard")

runs, results = load()
summary = (results.groupby("run_id")
           .agg(pass_rate=("passed", "mean"),
                p50_ms=("latency_ms", "median"),
                p95_ms=("latency_ms", lambda s: s.quantile(0.95)))
           .reset_index())
runs = runs.merge(summary, on="run_id")
runs["label"] = runs.prompt_version + " · " + runs.run_id

# 1. History: one point per run, in time order
st.subheader("Pass rate over time")
st.line_chart(runs.set_index("label")["pass_rate"])

# 2. Pick two runs to compare
labels = runs.label.tolist()
c1, c2 = st.columns(2)
base_label = c1.selectbox("Baseline run", labels, index=max(len(labels) - 2, 0))
cand_label = c2.selectbox("Candidate run", labels, index=len(labels) - 1)
base = runs[runs.label == base_label].iloc[0]
cand = runs[runs.label == cand_label].iloc[0]

if base.dataset_hash != cand.dataset_hash:
    st.error("These runs used different datasets. Pick two runs on the same dataset.")
    st.stop()

m1, m2, m3 = st.columns(3)
m1.metric("Pass rate", f"{cand.pass_rate:.0%}", f"{(cand.pass_rate - base.pass_rate) * 100:+.0f} pts")
m2.metric("p50 latency", f"{cand.p50_ms:.0f} ms", f"{cand.p50_ms - base.p50_ms:+.0f} ms",
          delta_color="inverse")
m3.metric("p95 latency", f"{cand.p95_ms:.0f} ms", f"{cand.p95_ms - base.p95_ms:+.0f} ms",
          delta_color="inverse")

# 3. Per-case changes: regressions first
d = diff(results, base.run_id, cand.run_id).reset_index()
order = {"REGRESSED": 0, "fixed": 1, "same": 2}
d = d.sort_values("change", key=lambda s: s.map(order))
st.subheader(f"{(d.change == 'REGRESSED').sum()} regressed, {(d.change == 'fixed').sum()} fixed")
st.dataframe(d, hide_index=True, width="stretch")

By default the page compares the two most recent runs. st.metric draws the delta under each number, and delta_color="inverse" paints a latency increase red instead of green. Start it with:

streamlit run app.py

You can also test the page without a browser. Streamlit's AppTest runs the script headlessly and lets you read widgets and change them. This is the test I used, and its real output:

# test_app.py
from streamlit.testing.v1 import AppTest

at = AppTest.from_file("app.py").run()
assert not at.exception, at.exception
print("runs:", at.selectbox[0].options)
for m in at.metric:
    print(f"{m.label:<12} {m.value:>8}  {m.delta}")
print(at.subheader[-1].value)
print(at.dataframe[0].value[["case_id", "change", "passed_base", "passed_cand"]].to_string(index=False))

at.selectbox[0].select(at.selectbox[0].options[0]).run()   # baseline = v1
print("\nbaseline v1:", at.subheader[-1].value)
runs: ['v1 · 13da66f7', 'v2 · e16d5e00', 'v2 · 05f6dfb2', 'v3 · 97333b19']
Pass rate         50%  -25 pts
p50 latency     31 ms  -1 ms
p95 latency     34 ms  -9 ms
2 regressed, 0 fixed
         case_id    change  passed_base  passed_cand
   refund-window REGRESSED            1            0
             sso REGRESSED            1            0
   refund-method      same            0            0
      plan-seats      same            1            1
extra-seat-price      same            1            1
     data-region      same            0            0
   phone-support      same            1            1
student-discount      same            1            1

baseline v1: 2 regressed, 2 fixed

The last line is the reason to compare per case. v1 and v3 both pass 4 of 8. A chart of pass rates shows a flat line between them, but v3 fixed the two refusal cases and broke two answers that v1 got right. If you only watched the average, you would ship v3 thinking nothing changed.

What the runs show#

  • v1 to v2: +25 points. The refusal rule fixed both unanswerable questions. Before, the app told users phone support and a student discount existed.
  • v2 also over-refuses. refund-method and data-region have answers in the context, and v2 says "I don't have enough information" for both. In this stub the cause is crude word matching ("refunded" does not match "Refunds"). With a real model, watch for the same pattern after you add an "only use the context" rule: refusals on questions that should have been answered. The case-level table is where you spot it, because those two cases fail under v1 too, so neither the pass rate nor the regression count moves.
  • v2 to v3: two regressions for a small latency gain. p50 dropped 1 ms and p95 dropped 9 ms. On a real model shorter outputs save more time and tokens, and that is the trade the page lets you weigh case by case.

Things that go wrong#

The dataset drifts. Someone fixes a typo in a case, the hash changes, and every older run stops being comparable. That is the correct outcome. Rerun the current baseline on the new dataset, then compare. Do not strip the hash check to make the chart continuous.

p95 on eight cases is almost the maximum. pandas interpolates between the two slowest calls, so the number moves with one slow request. Use eval latency to catch large changes, like a prompt that doubles output length. For real latency numbers, record production traffic with the cost and latency tracker.

The baseline moves silently. If "baseline" always means "the previous run", a slow decline of one case per week never trips the gate. Pin the baseline to the version that is in production, and update it only when you deploy.

"Fixed" and "regressed" hide changed failures. A case that went from a wrong answer to a wrong refusal shows as "same" because it failed both times. Read the answer columns for failing cases now and then, not only the change labels.

The scorer becomes the bug. If must_include is too strict, good rewordings fail and people stop trusting the dashboard. When a case fails, check the answer before you change the prompt. Sometimes the right fix is another alternative in the group.

Where to take it next#

Run run_eval.py and compare.py in CI on every pull request that touches prompts, retrieval or the model name, with the production run as the baseline. Add a model choice next to prompt_version so the same page compares model swaps. Keep the database somewhere shared, so the dashboard shows everyone's runs, not one laptop's. Grow the dataset from production failures, one case per bug, before you fix the bug.

For more on evals, the GyaanStack evals and LLMOps subject and the "Evaluating LLM apps" video path linked below collect the free material in order.

Sources#