GyaanStack. Blog

Articles / agentspythonprojects

A chat web UI for your AI agent: Streamlit vs Gradio vs FastAPI

One stubbed agent, three chat UIs in Python. Streaming tokens, visible tool calls, history, a stop button, CSV upload and tests for each, all run for real.

By Raghunath · · 21 min read

An agent that only runs in a terminal has one user: the person who wrote it. Put a chat page in front of it and a teammate, a reviewer or an interviewer can try it in a browser. The page is also where most of the bugs move. Tokens that stream fine in a terminal arrive as one block in the browser, tool calls disappear, a refresh wipes the conversation, and there is no way to stop a long answer.

This post builds the same chat UI three ways in Python: Streamlit, Gradio, and a small FastAPI server with one hand-written HTML page. Each version does the same six things:

  1. Streams the answer token by token.
  2. Shows each tool call and its result while the agent works.
  3. Keeps the conversation history across turns.
  4. Has a stop button that ends the run on the server, not only on screen.
  5. Accepts a CSV upload and passes it to a tool.
  6. Has an automated test that drives the UI without a browser.

All code below ran on Python 3.15.2 with Streamlit 1.67.0, Gradio 6.32.0 (gradio_client 2.7.2) and FastAPI 0.165.0, the current PyPI releases when this was written. The outputs are copied from those runs.

The agent is a stub on purpose#

There is no API key and no model here. The agent is a small generator that yields the same events a real tool-calling loop produces: a tool call, a tool result, then answer tokens. That keeps the UI code honest. If your UI handles these four event kinds, it will handle a real model.

If you want a real local model behind it, the local RAG with Ollama post covers model choice, RAM and num_ctx. If you want the tool-calling loop itself, see LLM function calling.

# agent.py
"""A stub agent with the same event shape a real tool-calling loop produces."""
import csv
import io
import time
from dataclasses import dataclass
from typing import Iterator


@dataclass
class Event:
    kind: str          # "tool_call" | "tool_result" | "token" | "done"
    text: str = ""
    name: str = ""


def describe_csv(raw: bytes) -> str:
    rows = list(csv.reader(io.StringIO(raw.decode("utf-8"))))
    header, body = rows[0], rows[1:]
    empty = sum(1 for r in body for cell in r if cell.strip() == "")
    return f"{len(body)} rows, {len(header)} columns ({', '.join(header)}), {empty} empty cells"


def run_agent(history: list[dict], file_bytes: bytes | None = None,
              delay: float = 0.02) -> Iterator[Event]:
    """history: [{"role": "user"|"assistant", "content": str}, ...], last item is the new user turn.

    Real version: send `history` plus tool schemas to your model, execute any tool
    calls it returns, append the results, and loop until it answers in text. Swap the
    body below for that loop (Ollama, OpenAI, Anthropic, LangGraph); keep the events.
    """
    question = history[-1]["content"]
    if file_bytes is not None:
        yield Event("tool_call", name="describe_csv", text="{\"file\": \"upload\"}")
        result = describe_csv(file_bytes)
        time.sleep(delay * 5)
        yield Event("tool_result", name="describe_csv", text=result)
        answer = f"The file has {result}. Ask me about any column."
    else:
        turns = sum(1 for m in history if m["role"] == "user")
        answer = f"You said: {question}. This is turn {turns} of our chat."
    for word in answer.split(" "):
        time.sleep(delay)
        yield Event("token", text=word + " ")
    yield Event("done")

The test file is a four-line CSV with two blank cells:

city,temp_c,rain_mm
Bangalore,24,
Pune,29,3
Delhi,,0

Every UI below has to turn that event stream into something like this:

 browser                    UI layer                       agent
 -------                    --------                       -----
 "what is in it?" + csv --> append user turn to history
                            call run_agent(history, file) --> tool_call describe_csv
 [Calling describe_csv] <-- draw a step box             <--
 [Used describe_csv]    <-- fill the step box           <-- tool_result "3 rows, ..."
 "The file has 3 ..."   <-- append token to reply       <-- token, token, token ...
                            save reply in history       <-- done
 [Stop] pressed ----------> stop pulling from generator     (no more tokens made)

Version 1: Streamlit#

Streamlit reruns your whole script from the top on every interaction. That one fact explains most Streamlit chat bugs. Anything you do not keep in st.session_state is gone on the next message, so the history lives there and the script redraws it each run.

Three newer pieces of the API do most of the work here:

  • st.chat_input takes accept_file=True and file_type=["csv"], so the upload sits in the chat box instead of a separate sidebar widget. With files on, it returns an object with .text and .files instead of a plain string.
  • The same widget takes submit_mode="stop". The docs say the submit button turns into a stop button while the script runs, and clicking it stops the script, the same as the Stop control in the app's status widget.
  • st.write_stream renders a generator chunk by chunk with a cursor.

Tool steps go in st.status, an expandable box with running, complete and error states.

# streamlit_app.py
import os
import streamlit as st
from agent import run_agent

DELAY = float(os.environ.get("AGENT_DELAY", "0.03"))
st.set_page_config(page_title="Agent chat")
st.title("Agent chat")

if "messages" not in st.session_state:
    st.session_state.messages = []   # {"role", "content", "steps": [str]}


def show_steps(steps):
    if steps:
        with st.status(f"Used {len(steps) // 2} tool call(s)", state="complete"):
            for s in steps:
                st.code(s, language=None)


def stream_reply(events, reply):
    """Draw tool steps as they happen; yield only answer text to write_stream."""
    status = None
    for e in events:
        if e.kind == "tool_call":
            status = st.status(f"Calling {e.name}...", expanded=True)
            status.code(f"{e.name}({e.text})", language=None)
            reply["steps"].append(f"call {e.name}({e.text})")
        elif e.kind == "tool_result":
            status.code(e.text, language=None)
            status.update(label=f"{e.name} done", state="complete", expanded=False)
            reply["steps"].append(f"result {e.text}")
        elif e.kind == "token":
            reply["content"] += e.text
            yield e.text


# Streamlit reruns this file top to bottom on every interaction,
# so the whole conversation is redrawn from session_state each time.
for m in st.session_state.messages:
    with st.chat_message(m["role"]):
        show_steps(m.get("steps"))
        st.markdown(m["content"])

prompt = st.chat_input("Ask, or attach a CSV", accept_file=True,
                       file_type=["csv"], submit_mode="stop")
if prompt:
    text = prompt.text or "Describe this file."
    file_bytes = prompt.files[0].getvalue() if prompt.files else None
    st.session_state.messages.append({"role": "user", "content": text})
    with st.chat_message("user"):
        st.markdown(text)

    history = [{"role": m["role"], "content": m["content"]}
               for m in st.session_state.messages]
    # Save the reply slot before streaming. If the user presses stop,
    # the words that already arrived stay in history after the rerun.
    reply = {"role": "assistant", "content": "", "steps": []}
    st.session_state.messages.append(reply)

    with st.chat_message("assistant"):
        st.write_stream(stream_reply(run_agent(history, file_bytes, DELAY), reply))

Two details in there are easy to get wrong.

The reply dict goes into session_state before streaming starts, and tokens are appended to it as they arrive. The common pattern appends the finished answer after write_stream returns. With a stop button that pattern loses the partial answer, because stopping the script means the line after write_stream never runs.

The tool steps are saved with the message. Without that, the st.status box shows during the live run and vanishes on the next rerun, because the redraw loop at the top has nothing to draw it from.

If your agent object is expensive to build (a model client, a vector index), wrap its constructor in st.cache_resource. Otherwise every message rebuilds it.

Start it with streamlit run streamlit_app.py --server.port 8801 --server.headless true. The health endpoint answered ok at localhost:8801/_stcore/health.

Testing it with AppTest#

AppTest runs the script in-process, with no browser and no server. In 1.67.0 its chat input has set_value() and also upload(), which stages a file for the next submit, so the upload path is testable too:

# test_streamlit.py
import os
os.environ["AGENT_DELAY"] = "0"
from streamlit.testing.v1 import AppTest


def test_history_survives_reruns():
    at = AppTest.from_file("streamlit_app.py").run()
    at.chat_input[0].set_value("hello").run()
    at.chat_input[0].set_value("again").run()
    msgs = at.session_state.messages
    assert [m["role"] for m in msgs] == ["user", "assistant"] * 2
    assert "turn 2" in msgs[-1]["content"]
    assert len(at.chat_message) == 4
    assert not at.exception


def test_csv_upload_runs_tool_and_shows_step():
    at = AppTest.from_file("streamlit_app.py").run()
    csv = b"city,temp_c,rain_mm\nBangalore,24,\nPune,29,3\nDelhi,,0\n"
    at.chat_input[0].upload("weather.csv", csv, "text/csv").set_value("what is in it?").run()
    reply = at.session_state.messages[-1]
    assert reply["steps"][0].startswith("call describe_csv")
    assert "3 rows, 3 columns" in reply["content"]
    assert "2 empty cells" in reply["content"]
    assert at.status[0].label == "describe_csv done"
$ python -m pytest -q test_streamlit.py
..                                                                       [100%]
2 passed in 0.43s

The first test catches the most common regression: someone moves the history into a plain variable and turn two forgets turn one. AppTest does not click the stop button, so the stop path still needs a manual check in a browser.

Version 2: Gradio#

gr.ChatInterface wraps a function fn(message, history). If the function is a generator, each yield replaces the bot's reply on screen, which is how streaming works. In Gradio 6.32 the constructor has no type argument any more; history arrives as a list of role and content dicts. stop_btn defaults to True, so a stop button appears during generation without extra code.

Tool steps use gr.ChatMessage with a metadata dict. A message with a title renders as a collapsible "thought" bubble, and status set to "pending" or "done" shows a spinner or a finished state. With multimodal=True, message is a dict with text and files, and files holds paths to temp copies of the uploads.

# gradio_app.py
import os
import gradio as gr
from agent import run_agent

DELAY = float(os.environ.get("AGENT_DELAY", "0.03"))


def chat(message, history):
    # multimodal=True: message is {"text": str, "files": [path, ...]}
    # history is a list of {"role", "content", "metadata"?} dicts
    files = message.get("files") or []
    file_bytes = open(files[0], "rb").read() if files else None
    # Tool-step bubbles carry a metadata title; keep them out of the model context.
    convo = [{"role": m["role"], "content": m["content"]} for m in history
             if isinstance(m["content"], str) and not (m.get("metadata") or {}).get("title")]
    convo.append({"role": "user", "content": message["text"] or "Describe this file."})

    steps, answer = [], ""
    for e in run_agent(convo, file_bytes, DELAY):
        if e.kind == "tool_call":
            steps.append({"title": f"Calling {e.name}", "body": f"{e.name}({e.text})",
                          "status": "pending"})
        elif e.kind == "tool_result":
            steps[-1].update(title=f"Used {e.name}", status="done",
                             body=steps[-1]["body"] + f"\n-> {e.text}")
        elif e.kind == "token":
            answer += e.text
        # Yield a fresh list every time. Gradio sends diffs between yields,
        # and mutating objects you already yielded makes the client drift.
        out = [gr.ChatMessage(content=s["body"],
                              metadata={"title": s["title"], "status": s["status"]})
               for s in steps]
        if answer:
            out.append(gr.ChatMessage(content=answer))
        yield out


demo = gr.ChatInterface(
    chat,
    multimodal=True,
    textbox=gr.MultimodalTextbox(file_types=[".csv"], file_count="single"),
    title="Agent chat",
    api_name="chat",
)

if __name__ == "__main__":
    demo.launch(server_port=int(os.environ.get("PORT", "8802")))

The comment about fresh lists comes from a bug in an earlier version of this file. That version created one gr.ChatMessage per step and changed its .content and .metadata in place on later yields. On screen it looked fine. Through the Python client, the final output was a step box still marked pending followed by a reply that read The, one word out of 18. Building new message objects on each yield fixed it.

The history filter matters once you call a real model. The thought bubbles are in history on the next turn, and sending "describe_csv(...) -> 3 rows" back as an assistant message confuses most models. Keep tool results in the format your model's API expects, or leave them out.

Testing it with gradio_client#

The Python client calls the running app over its API, the same endpoint the browser uses. submit() returns a job you can iterate to see each streamed update:

# test_gradio_client.py
from gradio_client import Client, handle_file

client = Client("http://127.0.0.1:8802/", verbose=False)

reply = client.predict({"text": "hello", "files": []}, api_name="/chat")
print("text only:", reply)

job = client.submit({"text": "what is in it?", "files": [handle_file("weather.csv")]},
                    api_name="/chat")
for i, partial in enumerate(job):      # one item per yield in chat()
    if i < 2:
        print("partial", i, partial)
print("updates:", len(job.outputs()))
print("final:", job.outputs()[-1])

Output against python gradio_app.py on port 8802 (long lines trimmed):

text only: [{'content': 'You said: hello. This is turn 1 of our chat. ', 'role': 'assistant', 'metadata': {}, 'options': []}]
partial 0 [{'content': 'describe_csv({"file": "upload"})', 'role': 'assistant', 'metadata': {'title': 'Calling describe_csv', 'status': 'pending'}, 'options': []}]
partial 1 [{'content': 'describe_csv({"file": "upload"})\n-> 3 rows, 3 columns (city, temp_c, rain_mm), 2 empty cells', 'role': 'assistant', 'metadata': {'title': 'Used describe_csv', 'status': 'done'}, 'options': []}]
updates: 21
final: [{... 'title': 'Used describe_csv', 'status': 'done' ...}, {'content': 'The file has 3 rows, 3 columns (city, temp_c, rain_mm), 2 empty cells. Ask me about any column. ', 'role': 'assistant', ...}]

21 updates is one per event: the tool call, the tool result, 18 words and the final done. client.view_api() listed a single input, message, with no history parameter. Each API call starts a new conversation, which is why the text-only reply says "turn 1". So this client tests the streaming and tool display of one turn. Multi-turn behaviour needs a test that calls chat() directly with a built history list.

Version 3: FastAPI plus one HTML page#

Streamlit and Gradio give you a chat page for free and decide its shape for you. When the agent has to sit inside an existing product, or the history must live in your database, a plain API and a small page are easier to own. The server streams server-sent events with FastAPI's StreamingResponse and takes the upload as a form file (this needs python-multipart installed).

Timeouts, retries, rate limits and load tests for this layer are covered in a production LLM API with FastAPI. The version here only has what a chat page needs.

# fastapi_app.py
import asyncio
import json
import os
from fastapi import FastAPI, File, Form, Request, UploadFile
from fastapi.responses import HTMLResponse, StreamingResponse
from agent import run_agent

DELAY = float(os.environ.get("AGENT_DELAY", "0.03"))
app = FastAPI()
SESSIONS: dict[str, list[dict]] = {}   # session_id -> history. Use Redis/SQL in production.


@app.post("/chat")
async def chat(request: Request, session_id: str = Form(...), text: str = Form(""),
               file: UploadFile | None = File(None)):
    history = SESSIONS.setdefault(session_id, [])
    history.append({"role": "user", "content": text or "Describe this file."})
    file_bytes = await file.read() if file else None
    reply = {"role": "assistant", "content": ""}
    history.append(reply)

    async def sse():
        events = run_agent(history[:-1], file_bytes, DELAY)
        while True:
            # run_agent is a sync generator; step it in a thread so the event loop stays free
            e = await asyncio.to_thread(next, events, None)
            if e is None:
                return
            if await request.is_disconnected():   # browser pressed stop (fetch aborted)
                return
            if e.kind == "token":
                reply["content"] += e.text
            yield f"data: {json.dumps(e.__dict__)}\n\n"

    return StreamingResponse(sse(), media_type="text/event-stream")


@app.get("/history/{session_id}")
def get_history(session_id: str):
    return SESSIONS.get(session_id, [])


@app.get("/", response_class=HTMLResponse)
def page():
    return open(os.path.join(os.path.dirname(__file__), "chat.html")).read()

The browser cannot use EventSource here, because EventSource only sends GET requests and we need to POST a file. The page uses fetch, reads the body stream, and splits it on blank lines. The stop button calls abort() on an AbortController, which closes the connection. On the server, Starlette's request.is_disconnected() sees the closed connection and the generator returns. This is the script part of chat.html:

<script>
const sid = sessionStorage.sid ||= crypto.randomUUID();
const log = document.getElementById("log");
let ctrl = null;
const add = (cls, text) => {
  const d = document.createElement("div"); d.className = cls; d.textContent = text;
  log.append(d); return d;
};
document.getElementById("stop").onclick = () => ctrl?.abort();
document.getElementById("f").onsubmit = async (ev) => {
  ev.preventDefault();
  const body = new FormData();
  body.append("session_id", sid);
  body.append("text", t.value);
  if (file.files[0]) body.append("file", file.files[0]);
  add("user", t.value || "(file)"); t.value = ""; file.value = "";
  const out = add("assistant", "");
  ctrl = new AbortController(); stop.disabled = false; send.disabled = true;
  try {
    const res = await fetch("/chat", { method: "POST", body, signal: ctrl.signal });
    const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
    let buf = "";
    for (;;) {
      const { value, done } = await reader.read();
      if (done) break;
      buf += value;
      const parts = buf.split("\n\n"); buf = parts.pop();   // keep a partial event for later
      for (const p of parts) {
        const e = JSON.parse(p.slice(6));                    // drop "data: "
        if (e.kind === "token") out.textContent += e.text;
        if (e.kind === "tool_call") log.insertBefore(add("step", `-> ${e.name}(${e.text})`), out);
        if (e.kind === "tool_result") log.insertBefore(add("step", `<- ${e.text}`), out);
      }
    }
  } catch (err) {
    if (err.name !== "AbortError") throw err;
    out.textContent += " [stopped]";
  } finally { stop.disabled = true; send.disabled = false; }
};
</script>

Two details here prevent subtle bugs. buf = parts.pop() keeps a half-received event for the next read, because network chunks do not line up with event boundaries. textContent instead of innerHTML means a model that outputs <img onerror=...> cannot run script in your page. If you render Markdown, pass it through a sanitizer first.

Proving the stop button stops the server#

A stop button that only hides the text is common. The tokens keep generating, and a paid model keeps billing for them. To check this one, the server ran with a 0.2 s delay per token, and a client hung up partway through:

# stop_check.py
import httpx, time
with httpx.Client(base_url="http://127.0.0.1:8803") as c:
    t0 = time.time()
    with c.stream("POST", "/chat", data={"session_id": "stop", "text": "a long question here"}) as r:
        for i, line in enumerate(r.iter_lines()):
            if line.startswith("data: ") and i >= 6:
                break                      # the client hangs up, like AbortController.abort()
    time.sleep(1.0)
    print(f"hung up after {time.time() - t0 - 1.0:.1f}s")
    print("saved reply:", repr(c.get("/history/stop").json()[-1]["content"]))
$ AGENT_DELAY=0.2 uvicorn fastapi_app:app --port 8803
$ python stop_check.py
hung up after 0.8s
saved reply: 'You said: a long '

Two seconds later /history/stop still returned 'You said: a long '. The full answer is 13 words, so the server stopped after 4 and saved the partial reply, which is what the user saw. With a real model, close the SDK's stream object in that same branch so the connection to the provider closes too.

Testing it with TestClient#

# test_fastapi.py
import json
import os
os.environ["AGENT_DELAY"] = "0"
from fastapi.testclient import TestClient
from fastapi_app import app, SESSIONS

client = TestClient(app)


def events(resp):
    return [json.loads(line[6:]) for line in resp.text.splitlines() if line.startswith("data: ")]


def test_stream_and_history():
    r = client.post("/chat", data={"session_id": "s1", "text": "hi"})
    assert r.headers["content-type"].startswith("text/event-stream")
    ev = events(r)
    assert ev[-1]["kind"] == "done"
    client.post("/chat", data={"session_id": "s1", "text": "again"})
    hist = client.get("/history/s1").json()
    assert [m["role"] for m in hist] == ["user", "assistant"] * 2
    assert "turn 2" in hist[-1]["content"]


def test_upload_emits_tool_events_first():
    csv = b"city,temp_c,rain_mm\nBangalore,24,\nPune,29,3\nDelhi,,0\n"
    r = client.post("/chat", data={"session_id": "s2", "text": ""},
                    files={"file": ("weather.csv", csv, "text/csv")})
    kinds = [e["kind"] for e in events(r)]
    assert kinds[:2] == ["tool_call", "tool_result"]
    assert "2 empty cells" in SESSIONS["s2"][-1]["content"]
$ python -m pytest -q test_streamlit.py test_fastapi.py
....                                                                     [100%]
4 passed in 1.13s

Which one to pick#

Streamlit 1.67 Gradio 6.32 FastAPI + HTML
Lines for this app 61 47 45 Python + 53 HTML/JS
Streaming st.write_stream yield from fn SSE + fetch reader
Tool steps st.status box ChatMessage metadata your own markup
History st.session_state, per browser tab passed into fn each turn your store, keyed by session id
Stop submit_mode="stop" stops the script stop_btn, on by default AbortController + is_disconnected()
File upload chat_input(accept_file=True) multimodal=True UploadFile form field
Test without a browser AppTest, in-process gradio_client, over HTTP TestClient, in-process
Free API for other programs no yes, /chat yes, it is the API

Pick Streamlit when the chat sits next to other widgets, like a data preview, a chart or filters, and the audience is internal. Pick Gradio when you want a chat demo fast, want to host it on Hugging Face Spaces, or want other programs to call the same agent through the client. Pick FastAPI when the agent goes into a real product, history has to survive a server restart, or several frontends will share one backend.

None of the three keeps history after a server restart in the code above. Streamlit's session state lives in server memory per browser session, and a page refresh starts a new one. Gradio's save_history=True stores conversations in the browser's local storage. The FastAPI dict is gone when the process stops. For anything beyond a demo, write each turn to a database keyed by user and conversation. Agent memory system design covers what to store beyond raw turns.

  • Answer appears all at once. Something between the model and the browser buffers the stream. Common causes are a non-generator function in Gradio, st.markdown instead of st.write_stream, or a reverse proxy buffering the response (nginx does this by default for proxied responses; set X-Accel-Buffering: no or turn off proxy_buffering).
  • Stop hides the text but the run continues. Check the server, not the page, as in the stop check above.
  • Partial answer lost on stop. Save the reply slot before streaming, then fill it.
  • Tool steps vanish on the next message. Store them with the message and redraw them.
  • Tool bubbles sent back to the model as assistant text. Filter or convert them when you build the next prompt.
  • Upload forgotten on the next turn. In Streamlit the file only comes with the submit that carried it; store the bytes or a parsed summary in session state if later turns need it.
  • Two users sharing one history. A global list in a FastAPI or Gradio app is shared by every visitor. Key everything by session.

To check whether the agent's answers are good, not only displayed well, see evaluating an AI agent in Python. To package any of these three apps, see dockerize an AI agent.

Sources#