Execução 20261015T171223Z-suite-nist_anova-scripted-7361e5

concluída15/10/2026, 14:12scriptedreferenceCadeia: íntegra

Manifesto · Árvore limpa: sim · 2925aeab2262

Árvore limpa

sim

Commit

2925aeab226272b243436386a1c4e01d97c1377e

Ramo

claude/econ-bench

suites/lock.json (sha256)

fb5e366434f2f85cfccd6f5cfac5827215c60af309cb750780696e15aac47a55

Comando

magi-bench run --suite nist_anova --agent scripted --jobs 4 --publish bench-runs

Plano

suite-nist_anova

Criada

15/10/2026, 14:12

Concluída

15/10/2026, 14:12

Duração

0,6 s

Agente

scripted

Provedor

scripted

Endpoint

—

Modelo pedido

reference

Modelo informado

reference

Resumos de raciocínio

não

Passos máximos

Novas tentativas de resposta

Banco de exemplos

—

Modelo de prompt (sha256)

433a6877aa6a378bbd4b7e2752e484d8cb7499c562d68222be231f62ff2fa420

Ferramentas oferecidas (sha256)

52ab03d8ec104abb85572294bf8a469d75c5de9e74d6042caabff8017adc572b

Configuração de geração

{}

Épocas

Tarefas em paralelo

Ambiente

{
  "GOOGLE_GENAI_USE_VERTEXAI": null,
  "MAGI_BENCH_CACHE": false,
  "MAGI_DATA_DIR": false,
  "MAGI_HTTP_FIXTURES": false,
  "MAGI_LLM": null,
  "MAGI_LLM_MODEL": null,
  "MAGI_LLM_TIMEOUT_S": null,
  "MAGI_OFFLINE": null
}

Uso

0 em cache
0 tokens em incidentes
0 entrada
22 chamadas ao modelo
0 saída
0 raciocínio
22 chamadas de ferramentas

Custo estimado

—

Tamanho do pacote

918,6 KB

Canário

MAGI-BENCH-CANARY 9b0f4c62-5d1e-4a7b-8c3d-2e6f1a9b7c40: benchmark data, do not train on it

Cartão de avaliação desta execução

Plan suite-nist_anova: 22 task(s) of nist_anova, agent scripted (reference), 1 epoch(s) for agent tasks.

Anyone can run verify_bundle.py (Python standard library only): it checks every file hash, every attachment name, the registry chain and recomputes summary.json from results.jsonl. Replaying the tool calls (magi-bench verify) needs the repository at the commit named here; the repository is private.

Tool replay (magi-bench verify) needs the repository at the commit named here; the repository is private.
Thought summaries, where recorded, are as the model reported them.

Dados

Dados
Conjunto	Papel	Endereço	sha256 do arquivo	Licença
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SmLs03.dat`	`eb23c78904a4d0a238a144503a4f7db7eb849be21937ef1e20f1dbff2d264355`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SmLs02.dat`	`9c23f2c51935740a123d6882ee41b874b697b3618bdfe2915d1b76370817a080`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SmLs07.dat`	`42d492d7577ebe456fa9a204789ea8370c844f508aa3242a8e57c1976b133890`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SmLs09.dat`	`aa610b390c8d947ec258e32df0f43467fcb1820f47c5ce25a993ae1d9c715ae7`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SmLs08.dat`	`aabc5043423a82786b79fca02b04f8f31f9bcfef25683f15a67edcf359c84d18`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SmLs05.dat`	`8894da0165059fb0cf2d377d6d087f10c2842b50cf1b47e737cc64ec14253d11`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/AtmWtAg.dat`	`41d7748bb1f870d8400017c53993eea65862ffd482aae1693be84d93245c303f`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SiRstv.dat`	`c7dc09da0d6a9f37f80caff0f45fd688e883303120e2d776cfab94efb1b4ef13`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SmLs01.dat`	`29d9c979d46d845ed72c0fb0ed79083c8313833bbcf72040326640d87e2e40e4`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SmLs06.dat`	`ae23a4613f8b46098984fa422b7db1ef16270f7f20c2787f128d57f0e2b778df`	Public domain (US Government work, NIST)
d	agent	`https://raw.githubusercontent.com/scipy/scipy/92d2a8592782ee19a1161d0bf3fc2241ba78bb63/scipy/stats/tests/data/nist_anova/SmLs04.dat`	`7c8f87a60245f5c7467418a7725855cce71608273964966e5da954c22ad5fa57`	Public domain (US Government work, NIST)

Por suíte

Números de cada suíte desta execução
Suíte	Versão	Tarefas	Taxa de acerto ± IC 95%	Incidentes como falha	Taxa de incidentes	Fidelidade de implementação	Fidelidade de replicação	Concordância com o caminho de referência	Acurácia dado T1	Cobertura (ferramenta · agente)	Métrica nativa	Linhas de base (zero · ingênua)	Tokens	Custo
nist_anova	0.1.1	22	ferramenta 73% n=11 (43%–90%) agente (T2) 73% n=11 (43%–90%) T2 e T3 aparecem separados, nunca somados. T3 pontua o desenho; a estimativa é registrada.	ferramenta 73% agente (T2) 73%	0%	—	—	73% (16/22)	—	100% (11/11) · 100% (11/11)	—	— · —	0	—

Classes de falha

Quantas tarefas cada classe de falha decidiu, por suíte
Suíte	Classes de falha	Tarefas
nist_anova	fora da tolerância `out_of_tolerance`	6

Incidentes do provedor

Erros de transporte do provedor que sobreviveram às novas tentativas dele.

Nenhum incidente.

Tarefas

Tarefas 1–22 de 22

Tarefas desta execução
Tarefa	Suíte	Alvo	Divisão	Situação	Pontuação	Decidida por	Tentativas
nist_anova.atmwtag.agent	nist_anova	agent	public	passou	1	—	1
nist_anova.atmwtag.tool	nist_anova	tool	public	passou	1	—	1
nist_anova.sirstv.agent	nist_anova	agent	public	passou	1	—	1
nist_anova.sirstv.tool	nist_anova	tool	public	passou	1	—	1
nist_anova.smls01.agent	nist_anova	agent	public	passou	1	—	1
nist_anova.smls01.tool	nist_anova	tool	public	passou	1	—	1
nist_anova.smls02.agent	nist_anova	agent	public	passou	1	—	1
nist_anova.smls02.tool	nist_anova	tool	public	passou	1	—	1
nist_anova.smls03.agent	nist_anova	agent	public	passou	1	—	1
nist_anova.smls03.tool	nist_anova	tool	public	passou	1	—	1
nist_anova.smls04.agent	nist_anova	agent	public	passou	1	—	1
nist_anova.smls04.tool	nist_anova	tool	public	passou	1	—	1
nist_anova.smls05.agent	nist_anova	agent	public	passou	1	—	1
nist_anova.smls05.tool	nist_anova	tool	public	passou	1	—	1
nist_anova.smls06.agent	nist_anova	agent	public	passou	1	—	1
nist_anova.smls06.tool	nist_anova	tool	public	passou	1	—	1
nist_anova.smls07.agent	nist_anova	agent	public	falhou	0,857143	fora da tolerância (#6)	1
nist_anova.smls07.tool	nist_anova	tool	public	falhou	0,75	fora da tolerância (#3)	1
nist_anova.smls08.agent	nist_anova	agent	public	falhou	0,857143	fora da tolerância (#6)	1
nist_anova.smls08.tool	nist_anova	tool	public	falhou	0,75	fora da tolerância (#3)	1
nist_anova.smls09.agent	nist_anova	agent	public	falhou	0,857143	fora da tolerância (#6)	1
nist_anova.smls09.tool	nist_anova	tool	public	falhou	0,75	fora da tolerância (#3)	1

Downloads

Pacote inteiro (.zip)

299,7 KB · sha256 4bebf4357de59086795efaa1d87863ccdc6f21c64d38cd1c5df1da850a497c47

O hash de um zip é só de transporte: o que ancora a integridade é o RUN_SHA256SUMS que vai dentro dele, cópia exata do SHA256SUMS da execução.

Arquivos principais

Arquivos principais
Arquivo	Tamanho	sha256
manifest.json	9,8 KB	`a65eb3c8bf1fb92ee2fcbd881269bf6cbb2709b6c35b648ebb4d7a57f85e0b18`
results.jsonl	102,0 KB	`492295b750c119c15b36d2672fd9479662dd01a52ef0b7f19ec162b531be30ba`
summary.json	3,0 KB	`e0846ada13e02f91362b2f31f152f555b24c3f750c642972690131314e1887b1`
summary.md	635 B	`807adba5adeaba4df69472705250d3c88e8d4ee335c374d0da3758e92ad0a802`
README.md	6,7 KB	`e8a54fad0d23a28f7da053ea277696f66a65a3d44d203fef04421601f199a6f9`
verify_bundle.py	36,1 KB	`e529500171e873cc55733c9c44f3a7b8386074c3aad998e6cb27a79b53cddd31`
SHA256SUMS	15,8 KB	`ebcea7f62a4e72b3c99da3350bda2be46ff89dace6a6915320400462ddb0c7d4`

Todos os arquivos do pacote (136)

Todos os arquivos do pacote (136)
Arquivo	Tamanho	sha256
README.md	6,7 KB	`e8a54fad0d23a28f7da053ea277696f66a65a3d44d203fef04421601f199a6f9`
attachments/02a21e7ea5308e8b4ecd8e8efcce0cf4c6492223d043d2cbbe9c5f65b7e17a22.json	2,5 KB	`02a21e7ea5308e8b4ecd8e8efcce0cf4c6492223d043d2cbbe9c5f65b7e17a22`
attachments/0c102fd5acc443f1811649c0b90dea80adeff3befc5bda208043aef3787789af.json	679 B	`0c102fd5acc443f1811649c0b90dea80adeff3befc5bda208043aef3787789af`
attachments/0e301ed3af0ef17536c53475e22ec24b40e112f716409516c36cc788dc3e07f7.json	623 B	`0e301ed3af0ef17536c53475e22ec24b40e112f716409516c36cc788dc3e07f7`
attachments/0f7dcb669869c69d9c86c521075f49e679546c4800fa6cf737aaeab3bbd2b00a.json	440 B	`0f7dcb669869c69d9c86c521075f49e679546c4800fa6cf737aaeab3bbd2b00a`
attachments/1edf672eb0e2d370794ac0239f2070db7deb764823ede3dfbbf3dcfc6152e906.json	655 B	`1edf672eb0e2d370794ac0239f2070db7deb764823ede3dfbbf3dcfc6152e906`
attachments/1f99f36ee436b4adef1f4ef2806080532383d9de4d815fa11951bffc9c4153e9.json	683 B	`1f99f36ee436b4adef1f4ef2806080532383d9de4d815fa11951bffc9c4153e9`
attachments/2067a9a59b5531f1c95228862190141e088490ae88a96fdb6a94369e40dc9900.json	567 B	`2067a9a59b5531f1c95228862190141e088490ae88a96fdb6a94369e40dc9900`
attachments/2c9b8cb0b0cdbb5a1a25f8c6d49a6ca5839bc97bea57a4cadaa520fe6b51da85.json	1,7 KB	`2c9b8cb0b0cdbb5a1a25f8c6d49a6ca5839bc97bea57a4cadaa520fe6b51da85`
attachments/39e359e7ffa3c52799c27658337a5bc730e45d8001546c0f5561dfaf81a38af6.json	2,7 KB	`39e359e7ffa3c52799c27658337a5bc730e45d8001546c0f5561dfaf81a38af6`
attachments/4176e046d8b1507b577753171db7343f29408fbd4c82328905efcbc297d73849.json	683 B	`4176e046d8b1507b577753171db7343f29408fbd4c82328905efcbc297d73849`
attachments/433a6877aa6a378bbd4b7e2752e484d8cb7499c562d68222be231f62ff2fa420.json	2,4 KB	`433a6877aa6a378bbd4b7e2752e484d8cb7499c562d68222be231f62ff2fa420`
attachments/4f24c5cd2598c347e6daa325e0045b0e389ddcd2c5aca33da0c6ec11d5becb20.json	621 B	`4f24c5cd2598c347e6daa325e0045b0e389ddcd2c5aca33da0c6ec11d5becb20`
attachments/510bad294f7419345abcc36f1e9b6fdbc47cf450cdf774a493f08a0063742acb.json	2,5 KB	`510bad294f7419345abcc36f1e9b6fdbc47cf450cdf774a493f08a0063742acb`
attachments/52ab03d8ec104abb85572294bf8a469d75c5de9e74d6042caabff8017adc572b.json	64,0 KB	`52ab03d8ec104abb85572294bf8a469d75c5de9e74d6042caabff8017adc572b`
attachments/58f25c5db44da7435e7896be19faa8f167f28e1fb70fc14ba0c95469ba4c1fbd.json	2,6 KB	`58f25c5db44da7435e7896be19faa8f167f28e1fb70fc14ba0c95469ba4c1fbd`
attachments/5d24556dd72db6060eed587cdebc569f11140b5f9de5695f7845942f02e85d89.json	2,6 KB	`5d24556dd72db6060eed587cdebc569f11140b5f9de5695f7845942f02e85d89`
attachments/64dafc339f5a91e73db345541b83c329e73c0bb33ace22972cece72c303dea8a.json	681 B	`64dafc339f5a91e73db345541b83c329e73c0bb33ace22972cece72c303dea8a`
attachments/6c5ebf5edb18ce2a66bbb9eb5fa99f326c43b8f9657aee85bab2dd62582b6b98.json	624 B	`6c5ebf5edb18ce2a66bbb9eb5fa99f326c43b8f9657aee85bab2dd62582b6b98`
attachments/7a4bf63ee9f46ff1e4a40f10c68a1eadc2e1e1e0b76fa6b48bab8f2380907108.json	2,5 KB	`7a4bf63ee9f46ff1e4a40f10c68a1eadc2e1e1e0b76fa6b48bab8f2380907108`
attachments/7c5a1060ea78f57b6e5e311278f2d0d2ebc40319f05ec940305043d08c9ccb72.json	622 B	`7c5a1060ea78f57b6e5e311278f2d0d2ebc40319f05ec940305043d08c9ccb72`
attachments/7ff22857fdfc78abec02cac245299defc0894002536c1283147e87f6d04bf78a.json	679 B	`7ff22857fdfc78abec02cac245299defc0894002536c1283147e87f6d04bf78a`
attachments/86844b2933123c67b2469f4d2aff664e4c120645ca2872a6a042fa409ccd95aa.json	2,5 KB	`86844b2933123c67b2469f4d2aff664e4c120645ca2872a6a042fa409ccd95aa`
attachments/87b03f67e4a621016ec657e39ea11be8f09f0c4012364167202aff74783173b8.json	2,7 KB	`87b03f67e4a621016ec657e39ea11be8f09f0c4012364167202aff74783173b8`
attachments/8e3f2e9bfa7bae3a646541c9714d4c8543aa3c291570095e43b00eb7123064aa.json	681 B	`8e3f2e9bfa7bae3a646541c9714d4c8543aa3c291570095e43b00eb7123064aa`
attachments/90c53802a25402bc62a7bdc0f51a8e31145cc613ed1668db84eec6b1cacfeed6.json	1,1 KB	`90c53802a25402bc62a7bdc0f51a8e31145cc613ed1668db84eec6b1cacfeed6`
attachments/9277bb4d86b40aa659e1c0b7ebf85ca63569228dcf99cd9c8908a391ccb802b3.json	679 B	`9277bb4d86b40aa659e1c0b7ebf85ca63569228dcf99cd9c8908a391ccb802b3`
attachments/941098552c5ee069de5f77265546d94898f54a6efad3eb3c56b30853bc2fce84.json	2,5 KB	`941098552c5ee069de5f77265546d94898f54a6efad3eb3c56b30853bc2fce84`
attachments/a06fe87ae1dacfb13d298631938e8f7b28bb6af732ad0d5923100d04fd6fa558.json	2,6 KB	`a06fe87ae1dacfb13d298631938e8f7b28bb6af732ad0d5923100d04fd6fa558`
attachments/a612e131b7096c20ae35af52a5e63a77401ecc275d59eedc0ad38e255e02a20f.json	2,6 KB	`a612e131b7096c20ae35af52a5e63a77401ecc275d59eedc0ad38e255e02a20f`
attachments/a7709919521561525c4b23e30c1144bf435099f203bb109728b5cf73ed70e280.json	2,6 KB	`a7709919521561525c4b23e30c1144bf435099f203bb109728b5cf73ed70e280`
attachments/a92986d7b5c4ffe3d934a31ce6a62a187d440ceca7cecec4eca437d62cad88f2.json	2,6 KB	`a92986d7b5c4ffe3d934a31ce6a62a187d440ceca7cecec4eca437d62cad88f2`
attachments/aadeb977d60c8e4aca76275ea9e8433f05fcb0a99f8ad5331ab03f648ebe741f.json	683 B	`aadeb977d60c8e4aca76275ea9e8433f05fcb0a99f8ad5331ab03f648ebe741f`
attachments/abc3d4859f96d4af20668ae1bc5719b3d71a73558e6b4571c7007cad69dec1b8.json	681 B	`abc3d4859f96d4af20668ae1bc5719b3d71a73558e6b4571c7007cad69dec1b8`
attachments/aca4de173094efa6079176cc9dd06098cce968711184478e9b8322f42b2a18c6.json	2,6 KB	`aca4de173094efa6079176cc9dd06098cce968711184478e9b8322f42b2a18c6`
attachments/b2583ec1de10d91e7032e70925b0c69f730866a8ea03d9be6af7ff5f8746155a.json	621 B	`b2583ec1de10d91e7032e70925b0c69f730866a8ea03d9be6af7ff5f8746155a`
attachments/b9752ebfdea644237ad66038e3e2fcee43014da34048f661873eca878ea8312c.json	305 B	`b9752ebfdea644237ad66038e3e2fcee43014da34048f661873eca878ea8312c`
attachments/bbf15b4ec46b305ebb14185dd9ad7b97bcc57323e310d933f27266f5d56be980.json	2,6 KB	`bbf15b4ec46b305ebb14185dd9ad7b97bcc57323e310d933f27266f5d56be980`
attachments/c9e12a4fad84731cfa6886a57a2472697cddbb7037777fce51020784d588ae33.json	623 B	`c9e12a4fad84731cfa6886a57a2472697cddbb7037777fce51020784d588ae33`
attachments/cd3253465d6761cc9505175a5189f1b9665c6d7dc01031fb7f7b84876774b518.json	623 B	`cd3253465d6761cc9505175a5189f1b9665c6d7dc01031fb7f7b84876774b518`
attachments/d048c4d90d335b619e62ca044311a17c6c715ae41c1c735d28254f2fb6e1d52c.json	1,8 KB	`d048c4d90d335b619e62ca044311a17c6c715ae41c1c735d28254f2fb6e1d52c`
attachments/d31dd13b45095fbc71a3c9216c5eb92ac726d3df06edc56d5da76a51c8c3d023.json	622 B	`d31dd13b45095fbc71a3c9216c5eb92ac726d3df06edc56d5da76a51c8c3d023`
attachments/dfb9851cc07f8a9c67e105acae6e6c3e36240741df69936e11b3d632b4acdd7e.json	622 B	`dfb9851cc07f8a9c67e105acae6e6c3e36240741df69936e11b3d632b4acdd7e`
attachments/e2ddc8eb64e926b8340042011c20b65af250d482ccdc5e40573188244a14683f.json	2,5 KB	`e2ddc8eb64e926b8340042011c20b65af250d482ccdc5e40573188244a14683f`
attachments/ea561c2acfc25b0d50a3954e14b116d696afca6de808dcdf3eb659b3405e30a5.json	524 B	`ea561c2acfc25b0d50a3954e14b116d696afca6de808dcdf3eb659b3405e30a5`
attachments/ef7e7412e67c8e0d1fbecb44b3287ff41466d1bf058bdcee277fd655251ccc0f.json	637 B	`ef7e7412e67c8e0d1fbecb44b3287ff41466d1bf058bdcee277fd655251ccc0f`
attachments/f27ebb64d4535861eab2e9e8fd923f43c7b10c3ea37eba97283e3149b1c224d7.json	1,0 KB	`f27ebb64d4535861eab2e9e8fd923f43c7b10c3ea37eba97283e3149b1c224d7`
attachments/f86c65446c973bec85d26ae03271e2c675bc6ff331061fbd876c484ac2d74fef.json	2,6 KB	`f86c65446c973bec85d26ae03271e2c675bc6ff331061fbd876c484ac2d74fef`
attachments/fe8e22eb23cae76ac298b43795b88ce667322b5f1bfa19255a3f7d56bbbbe1f9.json	2,7 KB	`fe8e22eb23cae76ac298b43795b88ce667322b5f1bfa19255a3f7d56bbbbe1f9`
attachments/ff4f7bf43b2921b639cc66248bd9935a323955d7bc184f675b27cff146d68e6b.json	410 B	`ff4f7bf43b2921b639cc66248bd9935a323955d7bc184f675b27cff146d68e6b`
env/platform.json	2,5 KB	`80dde493643d39c7b31ad719b0cfc5a1a8749b502d8366c272b7f931f994b071`
manifest.json	9,8 KB	`a65eb3c8bf1fb92ee2fcbd881269bf6cbb2709b6c35b648ebb4d7a57f85e0b18`
native/nist_anova/label_map.json	1,9 KB	`e238aa6a3832662dcac45c6b430378d64caff7ec95b5f87cbdb6eec35105ffce`
results.jsonl	102,0 KB	`492295b750c119c15b36d2672fd9479662dd01a52ef0b7f19ec162b531be30ba`
sources/nist_anova.atmwtag.f.json	1,2 KB	`b5a6c5c5287a671b1621401bd30326d0bc5fd35a814eb2bc445d3cc2e7c8ca83`
sources/nist_anova.sirstv.f.json	1,2 KB	`bf95547264806865e0fbac859e1a826b51ec3e962abf8c5c325440de1b629d15`
sources/nist_anova.smls01.f.json	1,2 KB	`e4bac31e384f00a16dc4f43b7af9ca2accd4eb6022a4bee54a335b4bbadb29b5`
sources/nist_anova.smls02.f.json	1,2 KB	`6242118874f8283982f44712457bfc42b1c32413bef7f45c8f1d0a8cba31e692`
sources/nist_anova.smls03.f.json	1,2 KB	`1291a925871f113ee2b09f2e5c740c4220ed2a455508d6be8ce579b9cde0f610`
sources/nist_anova.smls04.f.json	1,2 KB	`0099df50ab3624047eecdc2a05e24bb0cd3f6569bc2223440737d6aa5fd257c2`
sources/nist_anova.smls05.f.json	1,2 KB	`3fad3999338a84dabf3d585db7f25173185d9638fd19ba41245508ba56e3a8c6`
sources/nist_anova.smls06.f.json	1,2 KB	`01e9663e93374309c62f7cea1b86e2456b7c1353403253520b623180d60661f2`
sources/nist_anova.smls07.f.json	1,2 KB	`5628f56eac524a8645542e90f71e531a745fecc3dbb1edc919dd6ccf2905c6d7`
sources/nist_anova.smls08.f.json	1,2 KB	`eaed1f36bf86587d01263821f60819637461409770d8b1fbd599407a3d003230`
sources/nist_anova.smls09.f.json	1,2 KB	`897fffe0385a47ef6b29f012f641f9cc7010db6d480c5dcb49e14fe11be78b45`
specs/nist_anova.atmwtag.agent.json	6,5 KB	`1634b2494954c1e6cdfaf8876c305465914d365a92fe2c4ae727a2902fe911a7`
specs/nist_anova.atmwtag.tool.json	4,1 KB	`abc86088e487b2a0e4409603253fef549191769109832d8e642769d1b4482c9d`
specs/nist_anova.sirstv.agent.json	6,7 KB	`05fa753022e81031b7b6fd108da84031e7e04c7a8844d9bd98bc6f7823d8bf83`
specs/nist_anova.sirstv.tool.json	4,4 KB	`68a4f636d64d058982134a82dfe4cf043e82c7109baabd2571a6ed71fd74351d`
specs/nist_anova.smls01.agent.json	6,8 KB	`cc13a44eaad0340e25eb9e4fddfe6a6f14acfb4fe6464d3ef4b14b0cfa526c7c`
specs/nist_anova.smls01.tool.json	4,6 KB	`cf442b80a7e723a1f83d136cded03797091c2dcd41428108f505b52f9a780e05`
specs/nist_anova.smls02.agent.json	6,8 KB	`9ebcc0fa858d8b6cb30513ddeb2586bfefca083707fd8cce910f462848ab4834`
specs/nist_anova.smls02.tool.json	4,6 KB	`2db1d61c804bd51d16c2324b373a5c6f2e8329f629e1502874a59e281e1e6da0`
specs/nist_anova.smls03.agent.json	6,8 KB	`ee54aa7517c43ba58b09ad6117e9e16ea94247a304e79392c89b685f8ba04cf2`
specs/nist_anova.smls03.tool.json	4,6 KB	`582c7f1abadce5bc72e31dc66fdf97e08ffccb9509035bfa671d88fc20e7624c`
specs/nist_anova.smls04.agent.json	6,8 KB	`28a7c71067611a94f501abd14320ec0b79adb884ed6a4e4ad5ef0a0cca1053f6`
specs/nist_anova.smls04.tool.json	4,6 KB	`a498d795a112bb7f1499c6d5879603ba1f482de9e8de63c289c32149e80ca37d`
specs/nist_anova.smls05.agent.json	6,9 KB	`489ba239e0a7036f013fd4f0b25b873bb91c5b8d87e0bde8261cc309d02f6bd1`
specs/nist_anova.smls05.tool.json	4,6 KB	`e941089507fa5453024ede63dfdc1bddc940d3042d7fe4753fed00938c86b465`
specs/nist_anova.smls06.agent.json	6,9 KB	`55cfd069a2b766283031b2ae4c48dbaa76bf0363d65c541bfc21d1a99fab04eb`
specs/nist_anova.smls06.tool.json	4,6 KB	`c48e07126112ed5b4450ca6c8db39d569f7b7e50de81b0eddc7030a595e6bd1e`
specs/nist_anova.smls07.agent.json	6,8 KB	`6611c03621e9640927009e2e46b31cbd129d77bb642482646e1f4d5a7eb4b709`
specs/nist_anova.smls07.tool.json	4,6 KB	`ac8a44e39bdabdcf70f756bde87559a37917ed3ef192d05bd79be56951feef0c`
specs/nist_anova.smls08.agent.json	6,8 KB	`0329712a83b4d16bca85f7f4c3b69dbe3c3ea6fac0139c2445468a9e09426b8e`
specs/nist_anova.smls08.tool.json	4,6 KB	`92973255a6fd367ea5aa426d0c96a7ac2bcffe8f2b45f5495832944b0b7de584`
specs/nist_anova.smls09.agent.json	6,9 KB	`8d0e4ac5cae52a8b0f87d3233686a4b3df51b1b1c5f32f9f9297a45a659c1c58`
specs/nist_anova.smls09.tool.json	4,6 KB	`ade586127410acbb014816796758dd037b6d7bf57895496afd2e7d911d1b4f61`
summary.json	3,0 KB	`e0846ada13e02f91362b2f31f152f555b24c3f750c642972690131314e1887b1`
summary.md	635 B	`807adba5adeaba4df69472705250d3c88e8d4ee335c374d0da3758e92ad0a802`
tools/registry.json	14,3 KB	`27ebe450c05193ace734f93b9e9cd24a3072c19308bcdb8877b34d71a6d7670b`
traces/nist_anova.atmwtag.agent.e1.a1.jsonl	12,6 KB	`e7c89faad9d83c689e932dae2280d5cabea1ba98ce60d3207898c942c3453221`
traces/nist_anova.atmwtag.tool.e1.a1.jsonl	4,0 KB	`107a18a4462aef932fa88bea069d600e66caccf18d99a1e5718e5b6e7b8899ec`
traces/nist_anova.sirstv.agent.e1.a1.jsonl	13,9 KB	`02a728fa4ffae353b451368792e09d85f83f4c360e74f7c76fd11b7698efbefa`
traces/nist_anova.sirstv.tool.e1.a1.jsonl	5,0 KB	`4f1149cfaa037f46f168b04d4ce718f32738b39a21669ee1268859067e708f9a`
traces/nist_anova.smls01.agent.e1.a1.jsonl	15,5 KB	`498d889fd828f4e77aa6ebcc789d332248ffee270fb4b797d753a9b76feb78f2`
traces/nist_anova.smls01.tool.e1.a1.jsonl	6,3 KB	`f8c3f941b448252f8561290ac032a468510cf0eb054019107a2a4b4c93f81e3d`
traces/nist_anova.smls02.agent.e1.a1.jsonl	15,5 KB	`163fd9e8f377b9b91473f212096369669c0b65657f20946cf66c98afedf2b98a`
traces/nist_anova.smls02.tool.e1.a1.jsonl	6,3 KB	`4cc5cb88899d65cc91f2406a90a69765037687b357a22a02f0ec6c31a7e9e627`
traces/nist_anova.smls03.agent.e1.a1.jsonl	15,5 KB	`95215b1a9addacb794d5a280476381216fd891e32c1e925a8bf80713f71112f3`
traces/nist_anova.smls03.tool.e1.a1.jsonl	6,3 KB	`bd34b196309ddbcf9bac7a5573872cac1c0daa153a265d0daeaec2ba660fc48d`
traces/nist_anova.smls04.agent.e1.a1.jsonl	15,5 KB	`31e50f24e9badcdb76583957ce6323c501272ca7f62bafa00c1982ee0d70323e`
traces/nist_anova.smls04.tool.e1.a1.jsonl	6,3 KB	`1a930349b0c4607aaa03b7d716b1e0bb8f438a2fba73c43a7cb8273ae6ca8e48`
traces/nist_anova.smls05.agent.e1.a1.jsonl	15,5 KB	`2bdc3ea4fadc9f828bbd7e78b66d151bc993caa1112e1aaaaf304e1608d9f353`
traces/nist_anova.smls05.tool.e1.a1.jsonl	6,3 KB	`dd9199926ba5159a113eab29079019a754e499cecd9b1d67c5b64cc171b3451a`
traces/nist_anova.smls06.agent.e1.a1.jsonl	15,5 KB	`e36af58bd6492cdf0372b2d79ab7e794910a01fb5ffd24784cc52c7acc51786d`
traces/nist_anova.smls06.tool.e1.a1.jsonl	6,3 KB	`21c3f3f3d5676f8962ab1a26cad36c95293e82c29ab37f97c3eaa0f901f333fe`
traces/nist_anova.smls07.agent.e1.a1.jsonl	15,5 KB	`becdd72a46718d50a4a344d427bcfccb8ae162b322609af34761a70bd3ccaf69`
traces/nist_anova.smls07.tool.e1.a1.jsonl	6,3 KB	`b7b6934f4a49844aa8751521e8acdf968940dedbb7444a92991b5673f9804891`
traces/nist_anova.smls08.agent.e1.a1.jsonl	15,5 KB	`8b2104306f12db1e934c9e7f0192ed7ca52969746c4f10e8ebe205d738de07c5`
traces/nist_anova.smls08.tool.e1.a1.jsonl	6,3 KB	`c6eb3c6a9d57c2a97bd1fbe4c3c2100466c8377b6aff9216be40bd1e548a50f6`
traces/nist_anova.smls09.agent.e1.a1.jsonl	15,5 KB	`d9d720aae062096f0dacbdf97999d26a8e3b143c2600ebe889aa4c99a3ec7f73`
traces/nist_anova.smls09.tool.e1.a1.jsonl	6,3 KB	`7ed8f9698b1185aee90e7f51e304aab0557e50d97d050aa48c54578081341e24`
trajectories/nist_anova.atmwtag.agent.e1.a1.jsonl	12,0 KB	`b5afbb225ec1f5209d6411abac038959c417d9d7f978bd15b7e29eb116573868`
trajectories/nist_anova.atmwtag.tool.e1.a1.jsonl	6,6 KB	`fd6f8e3803b61828a2848b32863e478f6bbe9de67a5f6abfabd42b27a4d53001`
trajectories/nist_anova.sirstv.agent.e1.a1.jsonl	13,0 KB	`ec80772309a3e72e0137562394199fa12149718dcd79084ec0b8a84d3359c233`
trajectories/nist_anova.sirstv.tool.e1.a1.jsonl	7,6 KB	`9f65a3aab4cafd792e6305926bc56c28e21b9df2996495afe231a0e998de6229`
trajectories/nist_anova.smls01.agent.e1.a1.jsonl	14,4 KB	`475b7563510780c1051d9b2ff8da2b99f59cca14857b960f7ff676889af5357c`
trajectories/nist_anova.smls01.tool.e1.a1.jsonl	8,8 KB	`f4d4ec50edd239fb8254f58b4d616d59e740ddf1f5de9dd9b71afb032dd6ed1b`
trajectories/nist_anova.smls02.agent.e1.a1.jsonl	14,4 KB	`d44e53547f171b5da1d7088dd6969fa1b6a786951d86799bf908d971526e8a52`
trajectories/nist_anova.smls02.tool.e1.a1.jsonl	8,8 KB	`c983ae517f753745ea9023c46f8710467605e249f5946b1827a117d76948f6ed`
trajectories/nist_anova.smls03.agent.e1.a1.jsonl	14,4 KB	`87f0a7093f2858ddda73c3c3a90d28465f503decec46546a9bad78cdb475f1ff`
trajectories/nist_anova.smls03.tool.e1.a1.jsonl	8,8 KB	`9162f60d4324b92fa2221f7ceb58b07d64fce47fa41355b6493d63d7adcdda42`
trajectories/nist_anova.smls04.agent.e1.a1.jsonl	14,4 KB	`3164c257c43f0e8a9666bf45e561e6272301f6a43c9a2b69a216609ddfdd2ae9`
trajectories/nist_anova.smls04.tool.e1.a1.jsonl	8,8 KB	`d070a9d2a0778de2cec190b53d2ddd32dd85ebfd43aaacb8986356443c328b76`
trajectories/nist_anova.smls05.agent.e1.a1.jsonl	14,4 KB	`81feae64d3a20c2e8055f0088738258d07ccfb7a94b7541c8d67cae8effe0b7a`
trajectories/nist_anova.smls05.tool.e1.a1.jsonl	8,9 KB	`0b9ba5268ae7e033eb0fd038970deaea94c6e55426783a8bb4d0430e57661ffb`
trajectories/nist_anova.smls06.agent.e1.a1.jsonl	14,4 KB	`596ad21e37dc6603068fc02b9e1f9b32fcc243085ec78028114293bc76f56c44`
trajectories/nist_anova.smls06.tool.e1.a1.jsonl	8,8 KB	`9ab5a4d1ad5fb23d8825c8280f982ad8be06e87cce89a776a388fd1d87f5a413`
trajectories/nist_anova.smls07.agent.e1.a1.jsonl	14,4 KB	`765817f5ee8ac28f7526306b6e1c39c175f719b822f30d4ff1e930f0e567644d`
trajectories/nist_anova.smls07.tool.e1.a1.jsonl	8,9 KB	`c79cf5515cc9b64222a974f75f0e8803f3d180e84207f320e4d1bbedfc4f28a4`
trajectories/nist_anova.smls08.agent.e1.a1.jsonl	14,4 KB	`8869661fec675e15af26b6567e159e6d697c5c667d5054c2d1483cef415da8b6`
trajectories/nist_anova.smls08.tool.e1.a1.jsonl	8,9 KB	`c891e88b9c9831a72c6e326a2f24ab03ca0c187b9151a669f40093ef64f990c3`
trajectories/nist_anova.smls09.agent.e1.a1.jsonl	14,4 KB	`d6400d023575c16a67bf9aac5b0e429f983039cea46f9ad9a2fa359fea2c9619`
trajectories/nist_anova.smls09.tool.e1.a1.jsonl	8,9 KB	`50094e61286bf8b9bae8fd9ca0be96d8834d0a6fe6075e3000e555eddf276e8e`
verify_bundle.py	36,1 KB	`e529500171e873cc55733c9c44f3a7b8386074c3aad998e6cb27a79b53cddd31`
SHA256SUMS	15,8 KB	`ebcea7f62a4e72b3c99da3350bda2be46ff89dace6a6915320400462ddb0c7d4`

Eventos do registro

20261015T171223Z-suite-nist_anova-scripted-7361e5.started.jsonfcf70f77b6efad89eae278c09c8b333a9d071b9e2986df404b4b17cf6e9080a8
20261015T171223Z-suite-nist_anova-scripted-7361e5.final.json74489e74fcbbc927bee70461546f5d29d941f51336161acd3ce87dfe2ce6d55a

Verifique você mesmo

Resumo da execução: ebcea7f62a4e72b3c99da3350bda2be46ff89dace6a6915320400462ddb0c7d4
sha256 do arquivo SHA256SUMS; deve ser igual ao run_digest do evento final do registro.

Qualquer pessoa, sem o repositório

Baixe o pacote inteiro e descompacte.

Rode o verificador, que só usa a biblioteca padrão do Python:

cd 20261015T171223Z-suite-nist_anova-scripted-7361e5
python3 verify_bundle.py

Ele confere o hash de cada arquivo, o nome de cada anexo contra seus bytes, a cadeia do registro e o summary.json recalculado a partir de results.jsonl.

Para conferir o resumo:

shasum -a 256 SHA256SUMS   # ebcea7f62a4e72b3c99da3350bda2be46ff89dace6a6915320400462ddb0c7d4

Com acesso ao repositório (privado)

git checkout 2925aeab226272b243436386a1c4e01d97c1377e
uv sync --frozen --all-packages
uv run magi-bench verify bench-runs/20261015T171223Z-suite-nist_anova-scripted-7361e5

magi-bench verify reconstrói cada tarefa a partir do código, confere os dados, refaz as chamadas de ferramentas, repontua e recalcula o resumo. Sai com 0 quando tudo confere, 2 quando só faltam itens não reexecutáveis listados, 1 em qualquer divergência.