# MAGI benchmark run 20261015T171333Z-suite-e2e_memo-scripted-8076a5

Status: `success`.

## Overview

A run of the plan `suite-e2e_memo` at commit `2925aeab226272b243436386a1c4e01d97c1377e` (tree clean: yes) with the agent `scripted` (`fake`):
7 task(s) of `e2e_memo` 0.1.0, 7 result line(s) (one per task, epoch and attempt). The per-suite
numbers are in `summary.json`, rendered as `summary.md`; each is defined by the code of `verify_bundle.py`,
which recomputes them from `results.jsonl`.

Anyone can run verify_bundle.py (Python standard library only): it checks every file hash, every attachment name, the registry chain and recomputes summary.json from results.jsonl. Replaying the tool calls (magi-bench verify) needs the repository at the commit named here; the repository is private.

## Data availability and provenance statements

No third-party data file is redistributed. Each dataset is fetched from its publisher at a pinned URL and
verified by the sha256 of its bytes, or regenerated from a seeded generator (`manifest.json` → `data`):

| dataset | kind | URL | sha256 of the file | licence |
|---|---|---|---|---|
| — | — | — | — | — |

Published values quoted for verification are in `sources/`, each with its locator and an excerpt of at most
15 words; a benchmark's own gold is quoted from its pinned release with its licence.

## Computational requirements

`env/platform.json` lists the Python version, the operating system and architecture, the BLAS, `uv
--version`, the sha256 of `uv.lock` and every installed package as name==version. Tool replay
(`magi-bench verify`) needs the repository at the commit above and `uv sync --frozen --all-packages`.

## Description of the files

- `manifest.json`: what was run, by which code, agent and model, with which environment, at what cost.
- `results.jsonl`: one task result per task, epoch and attempt, sorted.
- `specs/`: each task exactly as it ran (canonical JSON; its sha256 is the result's `task_sha256`).
- `traces/`: the raw records of every call (the source of truth); `trajectories/`: the event view derived
  from them.
- `attachments/`: prompts, tool declarations, vendor contents and raw responses, each named by the sha256
  of its canonical bytes.
- `sources/`, `native/`, `tools/registry.json`, `env/platform.json`.
- `SHA256SUMS`: the sha256 of every other file; its own sha256 is the run digest.

Canonical JSON: keys sorted, no whitespace, ASCII, floats as Python's `repr`, non-finite numbers as the
strings `"NaN"`, `"Infinity"` and `"-Infinity"`.

## Instructions to replicators

1. Integrity, without the repository: `python3 verify_bundle.py` (standard library only).
2. With the repository at the commit: `uv sync --frozen --all-packages`, then
   `uv run magi-bench verify <this folder>` replays the tool calls and rescores every task.

## List of tasks

- `e2e_memo.londrina_brc_2026` (T3, protocol, split public)
- `e2e_memo.londrina_brc_2026.planned` (T3, protocol, split public)
- `e2e_memo.londrina_creche_2027` (T3, protocol, split public)
- `e2e_memo.londrina_creche_2027.planned` (T3, protocol, split public)
- `e2e_memo.londrina_parque_2027.framed` (T3, protocol, split public)
- `e2e_memo.londrina_saneamento_2026` (T3, protocol, split public)
- `e2e_memo.londrina_saneamento_2026.planned` (T3, protocol, split public)