# MAGI benchmark run 20261015T171414Z-suite-official-scripted-391f79

Status: `success`.

## Overview

A run of the plan `suite-official` at commit `2925aeab226272b243436386a1c4e01d97c1377e` (tree clean: yes) with the agent `scripted`:
11 task(s) of `official` 0.1.0, 11 result line(s) (one per task, epoch and attempt). The per-suite
numbers are in `summary.json`, rendered as `summary.md`; each is defined by the code of `verify_bundle.py`,
which recomputes them from `results.jsonl`.

Anyone can run verify_bundle.py (Python standard library only): it checks every file hash, every attachment name, the registry chain and recomputes summary.json from results.jsonl. Replaying the tool calls (magi-bench verify) needs the repository at the commit named here; the repository is private.

## Data availability and provenance statements

No third-party data file is redistributed. Each dataset is fetched from its publisher at a pinned URL and
verified by the sha256 of its bytes, or regenerated from a seeded generator (`manifest.json` → `data`):

| dataset | kind | URL | sha256 of the file | licence |
|---|---|---|---|---|
| pessoas | generator | — | `—` | — |
| pessoas | generator | — | `—` | — |

Published values quoted for verification are in `sources/`, each with its locator and an excerpt of at most
15 words; a benchmark's own gold is quoted from its pinned release with its licence.

## Computational requirements

`env/platform.json` lists the Python version, the operating system and architecture, the BLAS, `uv
--version`, the sha256 of `uv.lock` and every installed package as name==version. Tool replay
(`magi-bench verify`) needs the repository at the commit above and `uv sync --frozen --all-packages`.

## Description of the files

- `manifest.json`: what was run, by which code, agent and model, with which environment, at what cost.
- `results.jsonl`: one task result per task, epoch and attempt, sorted.
- `specs/`: each task exactly as it ran (canonical JSON; its sha256 is the result's `task_sha256`).
- `traces/`: the raw records of every call (the source of truth); `trajectories/`: the event view derived
  from them.
- `attachments/`: prompts, tool declarations, vendor contents and raw responses, each named by the sha256
  of its canonical bytes.
- `sources/`, `native/`, `tools/registry.json`, `env/platform.json`.
- `SHA256SUMS`: the sha256 of every other file; its own sha256 is the run digest.

Canonical JSON: keys sorted, no whitespace, ASCII, floats as Python's `repr`, non-finite numbers as the
strings `"NaN"`, `"Infinity"` and `"-Infinity"`.

## Instructions to replicators

1. Integrity, without the repository: `python3 verify_bundle.py` (standard library only).
2. With the repository at the commit: `uv sync --frozen --all-packages`, then
   `uv run magi-bench verify <this folder>` replays the tool calls and rescores every task.

## List of tasks

- `official.ipea_td2823.burden_decile_1` (T1, tool, split public)
- `official.ipea_td2823.burden_decile_10` (T1, tool, split public)
- `official.ipea_td2823.burden_mean` (T1, tool, split public)
- `official.sis2024.extreme_poverty_2023` (T1, tool, split public)
- `official.sis2024.gini_2023` (T1, tool, split public)
- `official.sis2024.poverty_2023` (T1, tool, split public)
- `official.sis2025.extreme_poverty_2024` (T1, tool, split public)
- `official.sis2025.gini_2024` (T1, tool, split public)
- `official.sis2025.gini_without_programs_2023` (T1, tool, split public)
- `official.sis2025.gini_without_programs_2024` (T1, tool, split public)
- `official.sis2025.poverty_2024` (T1, tool, split public)