# MAGI benchmark run 20261015T124724Z-suite-nist_univariate-gemini-28801c

Status: `success`.

## Overview

A run of the plan `suite-nist_univariate` at commit `38e979372a0f7382f285079e4380813081bdd032` (tree clean: yes) with the agent `gemini` (`gemini-3.8-flash`):
1 task(s) of `nist_univariate` 0.1.0, 1 result line(s) (one per task, epoch and attempt). The per-suite
numbers are in `summary.json`, rendered as `summary.md`; each is defined by the code of `verify_bundle.py`,
which recomputes them from `results.jsonl`.

Anyone can run verify_bundle.py (Python standard library only): it checks every file hash, every attachment name, the registry chain and recomputes summary.json from results.jsonl. Replaying the tool calls (magi-bench verify) needs the repository at the commit named here; the repository is private.

## Data availability and provenance statements

No third-party data file is redistributed. Each dataset is fetched from its publisher at a pinned URL and
verified by the sha256 of its bytes, or regenerated from a seeded generator (`manifest.json` → `data`):

| dataset | kind | URL | sha256 of the file | licence |
|---|---|---|---|---|
| d | file | https://raw.githubusercontent.com/usnistgov/dataplot/d837c4af1282e51667ed75a02433f5e5590f1c6d/lib/data/PIDIGITS.DAT | `e9421a2f4c0a31c505099332b4d22e507e808953c3c57bddae9226f25587d7d2` | Public domain (US Government work, NIST) |

Published values quoted for verification are in `sources/`, each with its locator and an excerpt of at most
15 words; a benchmark's own gold is quoted from its pinned release with its licence.

## Computational requirements

`env/platform.json` lists the Python version, the operating system and architecture, the BLAS, `uv
--version`, the sha256 of `uv.lock` and every installed package as name==version. Tool replay
(`magi-bench verify`) needs the repository at the commit above and `uv sync --frozen --all-packages`.

## Description of the files

- `manifest.json`: what was run, by which code, agent and model, with which environment, at what cost.
- `results.jsonl`: one task result per task, epoch and attempt, sorted.
- `specs/`: each task exactly as it ran (canonical JSON; its sha256 is the result's `task_sha256`).
- `traces/`: the raw records of every call (the source of truth); `trajectories/`: the event view derived
  from them.
- `attachments/`: prompts, tool declarations, vendor contents and raw responses, each named by the sha256
  of its canonical bytes.
- `sources/`, `native/`, `tools/registry.json`, `env/platform.json`.
- `SHA256SUMS`: the sha256 of every other file; its own sha256 is the run digest.

Canonical JSON: keys sorted, no whitespace, ASCII, floats as Python's `repr`, non-finite numbers as the
strings `"NaN"`, `"Infinity"` and `"-Infinity"`.

## Instructions to replicators

1. Integrity, without the repository: `python3 verify_bundle.py` (standard library only).
2. With the repository at the commit: `uv sync --frozen --all-packages`, then
   `uv run magi-bench verify <this folder>` replays the tool calls and rescores every task.

## List of tasks

- `nist_univariate.pidigits.agent` (T2, agent, split public)