Articles / agentspythonprojects
Build an AI agent for exploratory data analysis that cannot make up numbers
An EDA agent for any CSV: pandas computes the stats, the LLM plans charts and explains, its code runs in a limited subprocess, and every claim is checked.
The usual "AI does your EDA" demo pastes df.describe() into a prompt, asks the model for plotting code, runs it with exec() and prints whatever the model says about the data. It looks good on a clean CSV. It also has three holes: the model can quote a correlation that does not exist, the code it writes runs with your permissions, and nobody checks the summary against the numbers.
This post builds a small EDA agent that closes those holes. The split of work is strict:
- pandas computes every number (shape, types, missing values, distributions, outliers, correlations);
- the model plans the charts, writes plotting code and explains the results;
- the plotting code runs in a separate process with a time limit, a memory limit and an import allowlist;
- every finding in the report must cite a statistic from the profile, and a checker compares the cited value with the computed one before the finding is kept;
- charts are saved as PNG files next to the report, not shown in a notebook.
Everything below ran on Python 3.12 with pandas 3.1.1 and matplotlib 3.11.2. The model is a stub with canned answers, so no API key is needed; the section near the end shows where the real call goes.
The shape of the pipeline#
data.csv
|
v
[1] profile (pandas) ---------------------------> profile.json
| |
| profile, not raw rows |
v |
[2] LLM: write chart code |
| |
v |
[3] sandbox: AST check -> subprocess |
| (timeout, CPU, RSS watch, temp dir) |
|-- error? feed it back to [2], max 3 tries |
v |
charts/*.png |
v
[4] LLM: findings as JSON, each citing a stat path --> [5] checker
|
kept findings ------+--> report.md
rejected findings -----> log
The model never sees raw rows, never computes a number that ends up in the report, and never runs code in the agent's own process. Those three rules matter more than which model you pick.
The dataset#
The run uses the Palmer penguins data: 344 penguins from three species on three islands in Antarctica, with bill length, bill depth, flipper length, body mass and sex. It comes from Gorman, Williams and Fraser (2014), is packaged by palmerpenguins under a CC0 licence, and a CSV copy lives in the seaborn-data repository. It is small, but it has missing values and a real statistical trap, which makes it a good test.
curl -sSL -o data/penguins.csv \
https://raw.githubusercontent.com/mwaskom/seaborn-data/master/penguins.csv
python3 -m venv .venv && .venv/bin/pip install pandas matplotlib
Step 1: profile the data without a model#
Sending the whole CSV to a model is slow, costs tokens, breaks on large files and invites the model to do arithmetic it is bad at. A profile of a few kilobytes carries what a chart planner needs. Compute it in plain pandas:
import pandas as pd
def profile(df: pd.DataFrame, top_k: int = 5) -> dict:
num = df.select_dtypes("number")
cat = df.select_dtypes(exclude="number")
p = {
"rows": len(df),
"columns": {c: str(t) for c, t in df.dtypes.items()},
"missing": {c: int(n) for c, n in df.isna().sum().items() if n},
"duplicate_rows": int(df.duplicated().sum()),
"numeric": {},
"categorical": {},
"correlations": [],
}
for c in num:
s = num[c].dropna()
q1, q3 = s.quantile([0.25, 0.75])
iqr = q3 - q1
outliers = int(((s < q1 - 1.5 * iqr) | (s > q3 + 1.5 * iqr)).sum())
p["numeric"][c] = {
"mean": round(float(s.mean()), 2), "std": round(float(s.std()), 2),
"min": float(s.min()), "median": float(s.median()), "max": float(s.max()),
"skew": round(float(s.skew()), 2), "iqr_outliers": outliers,
}
for c in cat:
vc = df[c].value_counts()
p["categorical"][c] = {"unique": int(df[c].nunique()),
"top": {str(k): int(v) for k, v in vc.head(top_k).items()}}
corr = num.corr()
cols = list(corr.columns)
for i, a in enumerate(cols):
for b in cols[i + 1:]:
p["correlations"].append({"a": a, "b": b, "r": round(float(corr.loc[a, b]), 3)})
p["correlations"].sort(key=lambda x: -abs(x["r"]))
return p
On the penguins file, part of the output is:
columns: species: str, island: str, bill_length_mm: float64, bill_depth_mm: float64,
flipper_length_mm: float64, body_mass_g: float64, sex: str
missing: bill_length_mm 2, bill_depth_mm 2, flipper_length_mm 2, body_mass_g 2, sex 11
body_mass_g: mean 4201.75, std 801.95, min 2700, median 4050, max 6300, skew 0.47, iqr_outliers 0
species top: Adelie 152, Gentoo 124, Chinstrap 68
correlations (by |r|):
flipper_length_mm ~ body_mass_g 0.871
bill_length_mm ~ flipper_length_mm 0.656
bill_length_mm ~ body_mass_g 0.595
bill_depth_mm ~ flipper_length_mm -0.584
bill_depth_mm ~ body_mass_g -0.472
bill_length_mm ~ bill_depth_mm -0.235
The text columns show up as str, not object. That is pandas 3.0, which made a dedicated string dtype the default. Older tutorials that test dtype == object to find categorical columns break silently on pandas 3; select_dtypes(exclude="number") works on both. The migration guide lists the other changes.
The profile JSON is 1,784 bytes. That size does not grow with row count, only with column count, so a 2 GB CSV with 40 columns still produces a prompt of a few kilobytes. For very wide tables, send the top correlations and the columns with problems rather than every pair.
A small helper resolves a dotted path into the profile. The checker in step 4 uses it:
def lookup(p: dict, path: str):
"""Resolve a dotted path like 'numeric.body_mass_g.mean' or 'corr.a|b'."""
if path.startswith("corr."):
a, b = path[5:].split("|")
for c in p["correlations"]:
if {c["a"], c["b"]} == {a, b}:
return c["r"]
raise KeyError(path)
node = p
for part in path.split("."):
node = node[part]
return node
Step 2: put the model behind one method#
The agent only ever calls complete(system, user) -> str. That keeps the pipeline testable and the provider replaceable:
from typing import Protocol
class LLM(Protocol):
def complete(self, system: str, user: str) -> str: ...
The stub used for this run plays back what a model might return. On purpose, its first chart script uses a column name that does not exist (body_mass instead of body_mass_g), and its report contains one wrong correlation and one claim with no statistic behind it. Both are common model mistakes in this task, and they give the repair loop and the checker something to catch.
The two system prompts are short and specific about the output contract:
CODE_SYSTEM = """CHART CODE. You write matplotlib code for exploratory charts.
A DataFrame named df is already loaded. Save each chart with fig.savefig("NN_name.png").
Allowed imports: pandas, numpy, matplotlib.pyplot, math. No file or network access.
Return only Python code."""
REPORT_SYSTEM = """REPORT. You explain a dataset profile to an analyst.
Return JSON: {"findings": [{"text": str, "stat": str, "value": number}]}.
Every finding must cite one stat path from the profile and the value you read there.
Paths look like numeric.<col>.<field>, missing.<col>, categorical.<col>.top.<value>, corr.<a>|<b>."""
Asking for "only Python code" is not enough on its own, because models still wrap code in Markdown fences. The agent strips them with a regex before running anything.
Step 3: run chart code in a limited subprocess#
Model-written code is untrusted input. Calling exec() on it inside the agent gives it your files, your environment variables (API keys included) and your network. The sandbox here has three layers.
First, a static check with Python's ast module rejects imports outside an allowlist, calls to names such as open, eval and getattr, and any dunder attribute access, which blocks the usual ().__class__.__base__.__subclasses__() escape:
ALLOWED_IMPORTS = {"pandas", "matplotlib", "matplotlib.pyplot", "numpy", "math"}
BANNED_NAMES = {"open", "exec", "eval", "compile", "__import__", "input",
"globals", "locals", "getattr", "setattr", "delattr", "breakpoint"}
def check_code(code: str) -> list[str]:
try:
tree = ast.parse(code)
except SyntaxError as e:
return [f"syntax error: {e}"]
problems = []
for node in ast.walk(tree):
if isinstance(node, ast.Import):
names = [a.name for a in node.names]
elif isinstance(node, ast.ImportFrom):
names = [node.module or ""]
else:
names = []
for n in names:
if n not in ALLOWED_IMPORTS:
problems.append(f"import not allowed: {n}")
if isinstance(node, ast.Name) and node.id in BANNED_NAMES:
problems.append(f"name not allowed: {node.id}")
if isinstance(node, ast.Attribute) and node.attr.startswith("__"):
problems.append(f"dunder attribute not allowed: {node.attr}")
return problems
Second, the code runs in a fresh process started with python -I (isolated mode, which ignores PYTHONPATH and the user site directory, per the command-line docs), inside a temporary directory that holds only a copy of the CSV, with an environment that contains only PATH, HOME and MPLCONFIGDIR. No API keys leak in. A prelude loads df, switches matplotlib to the non-interactive Agg backend and sets resource limits:
PRELUDE = """
import resource, sys
try: # enforced on Linux; macOS ignores RLIMIT_AS, so the parent also watches RSS
resource.setrlimit(resource.RLIMIT_AS, ({mem}, {mem}))
except (ValueError, OSError):
pass
resource.setrlimit(resource.RLIMIT_CPU, ({cpu}, {cpu}))
import matplotlib
matplotlib.use("Agg")
import pandas as pd
df = pd.read_csv("data.csv")
"""
Third, the parent process watches the child. On macOS, setrlimit(RLIMIT_AS, ...) fails with "current limit exceeds maximum limit", so an address-space cap from the resource module does nothing there. The parent polls the child's resident memory with ps every 50 ms and kills it when it crosses the cap or the wall-clock timeout:
proc = subprocess.Popen([sys.executable, "-I", "job.py"], cwd=work, env=env,
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
start, peak, reason = time.time(), 0.0, None
while proc.poll() is None:
peak = max(peak, _rss_mb(proc.pid))
if peak > mem_mb:
reason = f"memory limit: {peak:.0f} MB > {mem_mb} MB"
elif time.time() - start > timeout_s:
reason = f"timeout after {timeout_s}s"
if reason:
proc.kill()
break
time.sleep(0.05)
After the run, any .png in the temp directory is moved to the output folder and the directory is deleted. The full file is in the agent's folder; the parts above are the ones that do the work.
When the script fails, the agent sends the last 300 characters of stderr back to the model and asks for a fix, up to three attempts:
def make_charts(llm, prof, csv_path, out_dir, max_attempts=3):
user = "Profile:\n" + json.dumps(prof)[:6000]
for attempt in range(1, max_attempts + 1):
code = strip_fences(llm.complete(CODE_SYSTEM, user))
result = sandbox.run(code, csv_path, out_dir)
print(f"chart attempt {attempt}: ok={result['ok']} "
f"{result.get('error', '')[-90:]}".rstrip())
if result["ok"]:
return result
user += f"\n\nYour last code failed:\n{result['error']}\nFix it."
return result
The successful chart script took 0.67 to 0.73 seconds across three runs and peaked at 114 MB of resident memory. A 512 MB cap leaves room for real plots on mid-sized files.
What the sandbox stops, and what it does not#
A test file throws six bad scripts at it with a 5 second timeout:
shell escape ok=False stage=static import not allowed: os
read a secret ok=False stage=static name not allowed: open
dunder walk ok=False stage=static dunder attribute not allowed: __subclasses__
infinite loop ok=False stage=run timeout after 5s
memory bomb ok=False stage=run memory limit: 782 MB > 512 MB
pandas file write ok=True stage=run
The last line is the one to remember. df.to_csv('/tmp/leak.csv') passed every check and wrote a file outside the temp directory, because pandas methods can read and write paths and the AST check cannot tell a harmless method call from a harmful one. The same goes for pd.read_csv("https://..."), which opens a network connection. An allowlist of imports narrows the attack surface; it does not stop a determined or prompt-injected model.
For anything beyond your own trusted CSVs on your own laptop, run the child inside a container with no network, a read-only root filesystem and a memory cap enforced by the kernel. The post on dockerizing an AI agent covers that setup, including running tools in a separate locked-down container. Keep the AST check and the timeout anyway: they turn the common failures into fast, readable errors that the repair loop can use.
Step 4: check every claim against the profile#
The report prompt forces each finding to carry a stat path and the value the model read there. The checker resolves the path in the real profile and keeps the finding only if the numbers agree within 2 percent:
def check_findings(findings, prof, tol=0.02):
kept, rejected = [], []
for f in findings:
if not f.get("stat"):
rejected.append((f, "no stat cited"))
continue
try:
actual = lookup(prof, f["stat"])
except KeyError:
rejected.append((f, "stat path does not exist"))
continue
claimed = float(f["value"])
if abs(claimed - actual) <= tol * max(1.0, abs(actual)):
kept.append({**f, "actual": actual})
else:
rejected.append((f, f"claimed {claimed}, profile says {actual}"))
return kept, rejected
The tolerance uses max(1.0, abs(actual)) so that a correlation of 0.87 against 0.871 passes, while a sign flip on a correlation fails. The report shows the computed value next to the model's sentence, so a reader can see what each claim rests on.
This is the same idea as checking claims against retrieved sources in RAG, applied to a table of numbers. The post on detecting LLM hallucinations in Python covers the general version, including how to measure a hallucination rate over many runs.
The full run#
$ python agent.py data/penguins.csv
profiled 344 rows x 7 columns
chart attempt 1: ok=False xes/base.py", line 3823, in get_loc
raise KeyError(key) from err
KeyError: 'body_mass'
chart attempt 2: ok=True
kept 3 findings, rejected 2:
REJECTED: 'Bill depth rises with flipper length (r = 0.58).' -> claimed 0.58, profile says -0.584
REJECTED: 'Body mass is right-skewed, so use a log transform before modelling.' -> no stat cited
And out/report.md:
# EDA report
344 rows, 7 columns, 0 duplicate rows.
## Findings (verified)
- Flipper length and body mass move together strongly (r = 0.87). [corr.flipper_length_mm|body_mass_g = 0.871]
- Sex is missing for 11 penguins. [missing.sex = 11]
- Adelie is the most common species with 152 rows. [categorical.species.top.Adelie = 152]
## Charts



The rejected correlation is the dangerous kind of error. The magnitude is right and the sign is wrong, and a human skimming the report would likely accept it. The skew claim was dropped for a different reason: a body-mass skew of 0.47 is mild, and the profile alone does not justify a log transform.
What the checker cannot catch#
The checker verifies numbers. It does not verify reasoning. Two examples from this dataset:
Across all penguins, bill length and bill depth have r = -0.235. Within each species the relationship is positive: 0.391 for Adelie, 0.654 for Chinstrap and 0.643 for Gentoo. A model that writes "longer bills are shallower (r = -0.235)" passes the check, because the number is real, and the conclusion is still wrong. This is Simpson's paradox, and a flat profile cannot expose it. Adding per-group correlations for each low-cardinality categorical column to the profile gives the model, and the checker, the numbers needed to see it.
The second gap is judgement wrapped around a real number. "Body mass skew is 0.47, so drop the column" would pass, because the stat is correct. Keep recommendations in a separate section of the report, labelled as suggestions, and do not let them go through the same "verified" path as factual findings.
Failure modes#
These are the ways this design breaks:
- Hallucinated column names. The model guesses
body_massfrom a profile that saysbody_mass_g. The repair loop fixes most of these on the second try. Passing the exact column list as its own field in the prompt cuts the rate further. - Repair loops that never converge. Some errors survive three tries. Cap attempts, and fall back to a fixed set of template charts (histograms, a missing-values bar, a correlation heatmap) so the report still has charts.
- Stat paths that almost match. The model writes
corr.body_mass_g|flipper_length_mmin the reverse order.lookupcompares column pairs as sets, so order does not matter. Paths with typos are rejected as "does not exist", which is the right outcome. - Profiles that leak data. The
topvalues of a categorical column are raw data. On a customer table, that can be names or emails. Drop or hash high-cardinality text columns before the profile reaches the model. - Huge files.
pd.read_csvon a multi-gigabyte file is the bottleneck, not the model. Profile a sample, or use DuckDB or Polars for the profiling step, and record the sample size in the report. - Charts nobody looks at. Saving PNGs is cheap, so models write twenty. Ask for at most five, with one sentence each saying what the chart shows.
Swapping in a real model#
Replace the stub with a class that implements complete. With the Anthropic SDK the body is a client.messages.create(...) call that returns msg.content[0].text; with a local model through Ollama it is a single HTTP request. If you use LangChain for the local route, note that langchain_community.llms.Ollama, which many tutorials still import, is marked deprecated since langchain-community 0.3.1 in favour of OllamaLLM from the separate langchain-ollama package.
Whatever the model, validate the report JSON and retry once on a parse error before you give up.
Before you trust any model with this, run the same CSV through it twenty times and count three things: chart scripts that pass on the first try, findings rejected by the checker, and findings that passed but are wrong on reading (like the Simpson's paradox case). The first two come straight from the logs above. The third needs a person to read a sample.
If the next step is letting analysts ask questions of a warehouse rather than profile one CSV, the text-to-SQL agent design post covers schema retrieval, the semantic layer and permissions, which this pipeline skips.
Sources#
- Gorman, Williams and Fraser, Ecological sexual dimorphism and environmental variability within a community of Antarctic penguins (genus Pygoscelis), PLOS ONE, 2014.
- palmerpenguins package and its CC0 licence; CSV from mwaskom/seaborn-data.
- pandas, What's new in 3.0.0 and Migration guide for the new string data type; pandas on PyPI.
- matplotlib, Backends; matplotlib on PyPI.
- Python docs, ast, resource, subprocess and command line and environment (
-Iisolated mode). - langchain-ollama on PyPI; the deprecation notice on
langchain_community.llms.Ollamais in the langchain-community 0.4.2 source.