# MAGI benchmark run 20261015T171300Z-suite-closed_form-scripted-9fb914 Status: `success`. ## Overview A run of the plan `suite-closed_form` at commit `2925aeab226272b243436386a1c4e01d97c1377e` (tree clean: yes) with the agent `scripted`: 48 task(s) of `closed_form` 0.1.0, 48 result line(s) (one per task, epoch and attempt). The per-suite numbers are in `summary.json`, rendered as `summary.md`; each is defined by the code of `verify_bundle.py`, which recomputes them from `results.jsonl`. Anyone can run verify_bundle.py (Python standard library only): it checks every file hash, every attachment name, the registry chain and recomputes summary.json from results.jsonl. Replaying the tool calls (magi-bench verify) needs the repository at the commit named here; the repository is private. ## Data availability and provenance statements No third-party data file is redistributed. Each dataset is fetched from its publisher at a pinned URL and verified by the sha256 of its bytes, or regenerated from a seeded generator (`manifest.json` → `data`): | dataset | kind | URL | sha256 of the file | licence | |---|---|---|---|---| | base | generator | — | `—` | — | | deciles | generator | — | `—` | — | | grid | generator | — | `—` | — | | grid | generator | — | `—` | — | | grid | generator | — | `—` | — | | grid | generator | — | `—` | — | | grid | generator | — | `—` | — | | grid | generator | — | `—` | — | | grid | generator | — | `—` | — | | grid | generator | — | `—` | — | | grid | generator | — | `—` | — | | grid | generator | — | `—` | — | | households | inline | — | `—` | — | | households | inline | — | `—` | — | | narrow | generator | — | `—` | — | | people | inline | — | `—` | — | | pof_expenditure | inline | — | `—` | — | | pof_products | inline | — | `—` | — | | pof_units | inline | — | `—` | — | | rct | generator | — | `—` | — | | reform | generator | — | `—` | — | | reform | generator | — | `—` | — | | wide | generator | — | `—` | — | Published values quoted for verification are in `sources/`, each with its locator and an excerpt of at most 15 words; a benchmark's own gold is quoted from its pinned release with its licence. ## Computational requirements `env/platform.json` lists the Python version, the operating system and architecture, the BLAS, `uv --version`, the sha256 of `uv.lock` and every installed package as name==version. Tool replay (`magi-bench verify`) needs the repository at the commit above and `uv sync --frozen --all-packages`. ## Description of the files - `manifest.json`: what was run, by which code, agent and model, with which environment, at what cost. - `results.jsonl`: one task result per task, epoch and attempt, sorted. - `specs/`: each task exactly as it ran (canonical JSON; its sha256 is the result's `task_sha256`). - `traces/`: the raw records of every call (the source of truth); `trajectories/`: the event view derived from them. - `attachments/`: prompts, tool declarations, vendor contents and raw responses, each named by the sha256 of its canonical bytes. - `sources/`, `native/`, `tools/registry.json`, `env/platform.json`. - `SHA256SUMS`: the sha256 of every other file; its own sha256 is the run digest. Canonical JSON: keys sorted, no whitespace, ASCII, floats as Python's `repr`, non-finite numbers as the strings `"NaN"`, `"Infinity"` and `"-Infinity"`. ## Instructions to replicators 1. Integrity, without the repository: `python3 verify_bundle.py` (standard library only). 2. With the repository at the commit: `uv sync --frozen --all-packages`, then `uv run magi-bench verify ` replays the tool calls and rescores every task. ## List of tasks - `closed_form.annuity.eac_federal_rate_thirty_years` (T1, tool, split public) - `closed_form.atkinson.lognormal_e0.5` (T1, tool, split public) - `closed_form.atkinson.lognormal_e1` (T1, tool, split public) - `closed_form.atkinson.lognormal_e2` (T1, tool, split public) - `closed_form.balance_budget.transfer_paid_by_irpf` (T1, tool, split public) - `closed_form.balance_budget.upper_bound_too_low` (T1, tool, split public) - `closed_form.breakeven.declines_the_city_population_as_users` (T1, tool, split public) - `closed_form.breakeven.per_million` (T1, tool, split public) - `closed_form.breakeven.share_of_stock` (T1, tool, split public) - `closed_form.breakeven.total_year` (T1, tool, split public) - `closed_form.combine.decile_share_ratio` (T1, tool, split public) - `closed_form.combine.gini_percent_change` (T1, tool, split public) - `closed_form.combine.top_decile_share_of_gini` (T1, tool, split public) - `closed_form.describe.inline_with_missing_values` (T1, tool, split public) - `closed_form.discount.annuity_factor_ten_percent_ten_years` (T1, tool, split public) - `closed_form.discount.declines_a_horizon_of_twelve_and_a_half_years` (T1, tool, split public) - `closed_form.discount.declines_a_rate_calc_combine_made` (T1, tool, split public) - `closed_form.discount.declines_a_rate_of_850_percent` (T1, tool, split public) - `closed_form.discount.zero_rate_sums_exactly` (T1, tool, split public) - `closed_form.equivalent_variation.cobb_douglas_prices` (T1, tool, split public) - `closed_form.equivalent_variation.income_only` (T1, tool, split public) - `closed_form.fiscal_impact.with_a_schedule` (T1, tool, split public) - `closed_form.fiscal_impact.without_a_schedule` (T1, tool, split public) - `closed_form.indirect_incidence.deciles` (T1, tool, split public) - `closed_form.indirect_incidence.monetary_only` (T1, tool, split public) - `closed_form.indirect_incidence.quintiles_exclusive` (T1, tool, split public) - `closed_form.kakwani.progressive_tax` (T1, tool, split public) - `closed_form.kakwani.proportional_tax` (T1, tool, split public) - `closed_form.kakwani.regressive_tax` (T1, tool, split public) - `closed_form.mvpf.cost_not_positive` (T1, tool, split public) - `closed_form.mvpf.periods_differ` (T1, tool, split public) - `closed_form.mvpf.same_units` (T1, tool, split public) - `closed_form.mvpf.scaled_units` (T1, tool, split public) - `closed_form.pair_frames.benefit_share_lognormal` (T1, tool, split public) - `closed_form.pair_frames.different_populations` (T1, tool, split public) - `closed_form.pair_frames.transfer_by_hand` (T1, tool, split public) - `closed_form.pair_frames.welfare_of_the_paired_frame` (T1, tool, split public) - `closed_form.quantile_shares.lognormal_deciles` (T1, tool, split public) - `closed_form.quantile_shares.lognormal_quintiles` (T1, tool, split public) - `closed_form.reynolds_smolensky.residual_progression` (T1, tool, split public) - `closed_form.scale_effect.log_points_to_percent` (T1, tool, split public) - `closed_form.scale_effect.percent_of_baseline` (T1, tool, split public) - `closed_form.scale_effect.sd_units` (T1, tool, split public) - `closed_form.theil.lognormal_s0.5` (T1, tool, split public) - `closed_form.theil.lognormal_s1.0_weighted` (T1, tool, split public) - `closed_form.welfare_change.compression_e0.5` (T1, tool, split public) - `closed_form.welfare_change.compression_e2` (T1, tool, split public) - `closed_form.welfare_change.uniform_growth_e1` (T1, tool, split public)