Pular para o conteúdo
Voltar aos benchmarks

Execução 20261015T124546Z-suite-causcibench_real-gemini-c5831e

concluída15/10/2026, 09:45geminigemini-3.8-flashCadeia: íntegraseleção parcial

Execução parcial

Esta execução não selecionou todas as tarefas da suíte (um limite, um filtro de etiquetas, tarefas ou alvo, ou menos tarefas do que a suíte constrói). Ela continua no registro, mas nenhum número dela entra nos destaques.

  • causcibench_real: n = 2 de 153 · --limit 2
Manifesto · Árvore limpa: sim · 38e979372a0f
Árvore limpa
sim
Commit
38e979372a0f7382f285079e4380813081bdd032
Ramo
claude/econ-bench
suites/lock.json (sha256)
76e3f2677736062b41d01ea3d38bf1f197ad9cf758a9b99080110021c5d549b7
Comando
magi-bench run --suite causcibench_real --limit 2 --jobs 2 --agent gemini --thoughts --publish bench-runs
Plano
suite-causcibench_real
Criada
15/10/2026, 09:45
Concluída
15/10/2026, 09:47
Duração
95,8 s
Agente
gemini
Provedor
gemini
Endpoint
https://generativelanguage.googleapis.com
Modelo pedido
gemini-3.8-flash
Modelo informado
gemini-3.8-flash
Resumos de raciocínio
sim
Passos máximos
12
Novas tentativas de resposta
1
Banco de exemplos
—
Modelo de prompt (sha256)
265d63d091732725cd5b15f51e1686b68b6b276ae88794f6120dc818830a2a97
Ferramentas oferecidas (sha256)
a44ef51f03df5188f084f1e0227b68df116b942a22fd50c0a88670cf3461fedd
Configuração de geração
{
  "automatic_function_calling": {
    "disable": true,
    "maximum_remote_calls": 10
  },
  "max_output_tokens": 32000,
  "seed": null,
  "temperature": null,
  "thinking_config": {
    "include_thoughts": true
  },
  "tool_config": {
    "function_calling_config": {
      "mode": "AUTO"
    }
  },
  "top_k": null,
  "top_p": null
}
Épocas
1
Tarefas em paralelo
2
Ambiente
{
  "GOOGLE_GENAI_USE_VERTEXAI": null,
  "MAGI_BENCH_CACHE": false,
  "MAGI_DATA_DIR": false,
  "MAGI_HTTP_FIXTURES": false,
  "MAGI_LLM": null,
  "MAGI_LLM_MODEL": null,
  "MAGI_LLM_TIMEOUT_S": null,
  "MAGI_OFFLINE": null
}
Uso
  • 343.611 em cache
  • 0 tokens em incidentes
  • 436.660 entrada
  • 19 chamadas ao modelo
  • 12.452 saída
  • 10.978 raciocínio
  • 14 chamadas de ferramentas
Custo estimado
USD 0,415357estimativa pelo preço de tabela
tabela de 2026-10-15 (https://ai.google.dev/gemini-api/docs/pricing): 0,75 por milhão de tokens de entrada, 3,75 por milhão de saída (o raciocínio conta como saída) · preço válido até 2026-12-31
Tamanho do pacote
675,0 KB
Canário
MAGI-BENCH-CANARY 9b0f4c62-5d1e-4a7b-8c3d-2e6f1a9b7c40: benchmark data, do not train on it
Cartão de avaliação desta execução

Plan suite-causcibench_real: 2 task(s) of causcibench_real, agent gemini (gemini-3.8-flash), 1 epoch(s) for agent tasks.

Anyone can run verify_bundle.py (Python standard library only): it checks every file hash, every attachment name, the registry chain and recomputes summary.json from results.jsonl. Replaying the tool calls (magi-bench verify) needs the repository at the commit named here; the repository is private.

  • Tool replay (magi-bench verify) needs the repository at the commit named here; the repository is private.
  • Thought summaries, where recorded, are as the model reported them.

Dados

Dados
ConjuntoPapelEndereçosha256 do arquivoLicença
gerber_social_pressureagenthttps://huggingface.co/datasets/causal-nlp/causcibench/resolve/d8aca99042e0bf1d73120a9dc1b65d4079fa1c3a/csv_files/realdata/gerber_social_pressure.csvbac3dfd61a4d09fd0e2256db8bac088ae6054b9e05990609b86665dceec9b6e8CC0 1.0 (CauSciBench data/README.md); original source https://dataverse.yale.edu/dataset.xhtml?persistentId=doi:10.60600/YU/CGMWNW

Por suíte

Números de cada suíte desta execução
SuíteVersãoTarefasTaxa de acerto ± IC 95%Incidentes como falhaTaxa de incidentesFidelidade de implementaçãoFidelidade de replicaçãoConcordância com o caminho de referênciaAcurácia dado T1Cobertura (ferramenta · agente)Métrica nativaLinhas de base (zero · ingênua)TokensCusto
causcibench_real0.1.120% (0%–0%)
bootstrap por agrupamento
0%0%————64% (112/175) · 44% (77/175)
any_cited_would_pass 0
0% · —460.0900,415357

Classes de falha

Quantas tarefas cada classe de falha decidiu, por suíte
SuíteClasses de falhaTarefas
causcibench_realnenhuma estimativa citada no_estimate_cited2

Incidentes do provedor

Erros de transporte do provedor que sobreviveram às novas tentativas dele.

Nenhum incidente.

Tarefas

Tarefas 1–2 de 2

Tarefas desta execução
TarefaSuíteAlvoDivisãoSituaçãoPontuaçãoDecidida porTentativas
causcibench_real.q001causcibench_realagentpublicfalhou0,5nenhuma estimativa citada (#1)1
causcibench_real.q002causcibench_realagentpublicfalhou0,5nenhuma estimativa citada (#1)1

Downloads

Pacote inteiro (.zip)
329,6 KB · sha256 9985def96e26695fe3994bae9941d03860b6930b17dbf42cf965e4ca37e7b42a

O hash de um zip é só de transporte: o que ancora a integridade é o RUN_SHA256SUMS que vai dentro dele, cópia exata do SHA256SUMS da execução.

Arquivos principais

Arquivos principais
ArquivoTamanhosha256
manifest.json12,7 KB69ade86daa0a11961c6dc712909e1e00ca21e7a0f89aa681851976431495580a
results.jsonl10,4 KBf71437395f081bdcb9fd5299b75b74ce1ea3847d2393ecc4b2b23cc27481c3ee
summary.json2,2 KB1eabaea0fe646b17ab7ea3c899a14c5e60227585e87a7b4a94699f164d35c8db
summary.md587 Be35712ee21847be7e05ff5183322d310326faceeb6f3c409d21b0e5ac53cac52
README.md3,3 KB7d48aa58eb6cd9b08d872936fa63fe9675a6be22d74ed67d3fd82b0f9715dec3
verify_bundle.py34,9 KB99e037f6a241e077348d7e116b03be17e727c4b61d278ea5f39b06dd557069a3
SHA256SUMS21,7 KB999d2b83d9a490e4f3f2bafead28ab4f9a81f2dcccc8d5e249e3f17fee18ed06
Todos os arquivos do pacote (162)
Todos os arquivos do pacote (162)
ArquivoTamanhosha256
README.md3,3 KB7d48aa58eb6cd9b08d872936fa63fe9675a6be22d74ed67d3fd82b0f9715dec3
attachments/01159509c9c3995dab40da304bbac15630b3e9b63c30e0d3087ecc3b218bf494.json3,2 KB01159509c9c3995dab40da304bbac15630b3e9b63c30e0d3087ecc3b218bf494
attachments/05a293b861a0b0428684c6ab378b7efaa05ba04d5fb3802a609f2144e683ecee.json330 B05a293b861a0b0428684c6ab378b7efaa05ba04d5fb3802a609f2144e683ecee
attachments/05ef2522fa34057702b7bb014bb7910e12e070c51d035cecbded0b516201bcec.json3,7 KB05ef2522fa34057702b7bb014bb7910e12e070c51d035cecbded0b516201bcec
attachments/068c08c0af715fa0248229ffda427b2d54c5ce252cc4b377861c12f59d68bbe0.json2,3 KB068c08c0af715fa0248229ffda427b2d54c5ce252cc4b377861c12f59d68bbe0
attachments/093b19ba52a50a98294a5f3168fff1fdee016b556e49d9aa3b999a2a354ddc64.json3,1 KB093b19ba52a50a98294a5f3168fff1fdee016b556e49d9aa3b999a2a354ddc64
attachments/0b73915923b9790050c31b3286a1834428b963cfb6584b4154c766fef75637fe.json292 B0b73915923b9790050c31b3286a1834428b963cfb6584b4154c766fef75637fe
attachments/0bfac5b4bb152401945ed429fe16e10a8dde5be24a7f753d7510e3b101c2cf0f.json332 B0bfac5b4bb152401945ed429fe16e10a8dde5be24a7f753d7510e3b101c2cf0f
attachments/0d2f6bd5112bcac9925a58b26cd27fd0e387e1d84bc7e730856761d33a2bedd6.json250 B0d2f6bd5112bcac9925a58b26cd27fd0e387e1d84bc7e730856761d33a2bedd6
attachments/16dd2bf5ddc5c0207b29ab899fff2761acb0e8883bd07446c65eb04f00caaa6b.json2,0 KB16dd2bf5ddc5c0207b29ab899fff2761acb0e8883bd07446c65eb04f00caaa6b
attachments/17e8eb1eba0fac5626ce77aec5cf9ba7ddb8c54a909c329a14d7808be5b59182.json2,4 KB17e8eb1eba0fac5626ce77aec5cf9ba7ddb8c54a909c329a14d7808be5b59182
attachments/1ac08ad9e2dbfe29e57846f31523b9978c1f874bcb310ad441b50530ee8c52a6.json326 B1ac08ad9e2dbfe29e57846f31523b9978c1f874bcb310ad441b50530ee8c52a6
attachments/1d670477f44e23165691aeb0d1ec7d1f9f333cf049c7e350c89bf4cdf0706d4f.json324 B1d670477f44e23165691aeb0d1ec7d1f9f333cf049c7e350c89bf4cdf0706d4f
attachments/1d87a25ff33dabf0923b7750b6b7e8dfe81f854c914008fa852414a8efb01446.json1,6 KB1d87a25ff33dabf0923b7750b6b7e8dfe81f854c914008fa852414a8efb01446
attachments/1e0bd2f4a9348bfa5469b0def5b466f5785c3b0bc369fb1ce6909ba7e066136e.json332 B1e0bd2f4a9348bfa5469b0def5b466f5785c3b0bc369fb1ce6909ba7e066136e
attachments/1e61bf7c9a1378480509063ef21cbb64bed88e6d3f4296abd6a1b8edd44e1c82.json6,7 KB1e61bf7c9a1378480509063ef21cbb64bed88e6d3f4296abd6a1b8edd44e1c82
attachments/1eafdf3534156c2e5051ea4ffc2be9924b6a4079be2b5b15f029766a5e1375d4.json3,2 KB1eafdf3534156c2e5051ea4ffc2be9924b6a4079be2b5b15f029766a5e1375d4
attachments/203d2a2bbcc7cfeb69ca3c8c5584bce222630010a24433c2631a35397179be86.json88 B203d2a2bbcc7cfeb69ca3c8c5584bce222630010a24433c2631a35397179be86
attachments/233cbf29a9aa2a5d1a35ed4228fa6baa523aad693e6bfa42c1d58d3a006a46e0.json503 B233cbf29a9aa2a5d1a35ed4228fa6baa523aad693e6bfa42c1d58d3a006a46e0
attachments/23cea14099c13f587e351e9f45a92327661e54e221bf33c85d7fee3b94ed7841.json367 B23cea14099c13f587e351e9f45a92327661e54e221bf33c85d7fee3b94ed7841
attachments/2525106360a2db417e0cd879d8be4ba272625e4f590f62d77a617e916998e2e2.json308 B2525106360a2db417e0cd879d8be4ba272625e4f590f62d77a617e916998e2e2
attachments/264d51420cf0eece7d3c3328fe93c7e60d7dc533625e27e141f3b9c977701f5e.json2,0 KB264d51420cf0eece7d3c3328fe93c7e60d7dc533625e27e141f3b9c977701f5e
attachments/265d63d091732725cd5b15f51e1686b68b6b276ae88794f6120dc818830a2a97.json5,6 KB265d63d091732725cd5b15f51e1686b68b6b276ae88794f6120dc818830a2a97
attachments/29083e0421efed90988dc8cc4c152827aeafe18c2ef1143bf0c90e6cb011e98d.json545 B29083e0421efed90988dc8cc4c152827aeafe18c2ef1143bf0c90e6cb011e98d
attachments/295db814a5a0d102e2b408ec8d9d8bf3d0d006355dd1a2c445573f6947849163.json3,6 KB295db814a5a0d102e2b408ec8d9d8bf3d0d006355dd1a2c445573f6947849163
attachments/2c9505ac50720c48d09f654d2a5c2ad38f8747a10ba6e05a9e2a4a8f8984fb13.json5,8 KB2c9505ac50720c48d09f654d2a5c2ad38f8747a10ba6e05a9e2a4a8f8984fb13
attachments/315a5615065136c8fc67eb5d857fab9bd0ab6ba630e88fcf7e33778334e872c6.json2,3 KB315a5615065136c8fc67eb5d857fab9bd0ab6ba630e88fcf7e33778334e872c6
attachments/3597d53deb6e096156655c6df8a4893731d8f72092dab48f5b90c6c698cc0171.json2,4 KB3597d53deb6e096156655c6df8a4893731d8f72092dab48f5b90c6c698cc0171
attachments/35d81fb4c8a08e36f6fbc4bfe6dc1eb4fe5ab2fcdd7e0e3b2bdf64ac14ff472d.json2,9 KB35d81fb4c8a08e36f6fbc4bfe6dc1eb4fe5ab2fcdd7e0e3b2bdf64ac14ff472d
attachments/3690f0fd09d324f4dd8eacb103571cba44011657f627c59bbb58b516580a709d.json2,5 KB3690f0fd09d324f4dd8eacb103571cba44011657f627c59bbb58b516580a709d
attachments/3b56b9f55454a20206d924778dbee6f82aa906dee07a64f9ec9f322a80c22fad.json2,4 KB3b56b9f55454a20206d924778dbee6f82aa906dee07a64f9ec9f322a80c22fad
attachments/3d1d101c601889a044e558fc92297ce6d254655440a7ef28cbf33399970f6306.json1,7 KB3d1d101c601889a044e558fc92297ce6d254655440a7ef28cbf33399970f6306
attachments/3e34d208cd6057b5606f21399a57164b5108da58c73f079667764a817293af73.json2,0 KB3e34d208cd6057b5606f21399a57164b5108da58c73f079667764a817293af73
attachments/48f002cc77a035c14309c1dbf1f3bd66504d5f49e054e8d2b62794cb50c71087.json6,3 KB48f002cc77a035c14309c1dbf1f3bd66504d5f49e054e8d2b62794cb50c71087
attachments/4e84669250d5b4922133e860fe957baa007655440341f7e91f57d754bbc47865.json215 B4e84669250d5b4922133e860fe957baa007655440341f7e91f57d754bbc47865
attachments/5224188eefd42e1e9684c7f781c890a8b4199d60b1ce1dda882cf0464beab249.json330 B5224188eefd42e1e9684c7f781c890a8b4199d60b1ce1dda882cf0464beab249
attachments/52c68783bdb898e9562afe069b382c767f46dcb10cccbd16f15ab5f4e80503c8.json326 B52c68783bdb898e9562afe069b382c767f46dcb10cccbd16f15ab5f4e80503c8
attachments/58e62a013b21119a31c4edbc0f48e042780061db6389b853edfb0e2e8adc629f.json250 B58e62a013b21119a31c4edbc0f48e042780061db6389b853edfb0e2e8adc629f
attachments/5b16c81cae9c42f67f0cc5db8ec931cc3dbf53a1ac24e1c832b7fa433545e3ae.json292 B5b16c81cae9c42f67f0cc5db8ec931cc3dbf53a1ac24e1c832b7fa433545e3ae
attachments/5c9822a0e08e49085754b568d5cbccad23e630ff8003e0c8359b99e70b3017e9.json190 B5c9822a0e08e49085754b568d5cbccad23e630ff8003e0c8359b99e70b3017e9
attachments/5f39fc2af45abea072951b995424fd8c7e1fe14bdc7f1945b7417769d933fe43.json2,2 KB5f39fc2af45abea072951b995424fd8c7e1fe14bdc7f1945b7417769d933fe43
attachments/644df775910ebecff28780e3e068040f780d54f291af3b94144e66e0e79801d9.json1,5 KB644df775910ebecff28780e3e068040f780d54f291af3b94144e66e0e79801d9
attachments/659aca9fdd76d7ab65573ad8941f4bff50198360ffd6d5226d453729db4acf09.json215 B659aca9fdd76d7ab65573ad8941f4bff50198360ffd6d5226d453729db4acf09
attachments/65c8f50353de039a0e0af09ef2fa06fc54a019338ae487c385d0d3c9a567cfe5.json3,5 KB65c8f50353de039a0e0af09ef2fa06fc54a019338ae487c385d0d3c9a567cfe5
attachments/6d1f992fd0055c26730702a784d9f25113de2850466b0aff982d84ea120f29ec.json15,2 KB6d1f992fd0055c26730702a784d9f25113de2850466b0aff982d84ea120f29ec
attachments/6d60b5f07750bc4dc8319fe818500dc5bdf6a3efddbd57634d5610b156abd4a6.json392 B6d60b5f07750bc4dc8319fe818500dc5bdf6a3efddbd57634d5610b156abd4a6
attachments/71dec5b5f8124adc1b4ffd219bfcb051bf437426333f85411d6f1f7572f51d12.json3,7 KB71dec5b5f8124adc1b4ffd219bfcb051bf437426333f85411d6f1f7572f51d12
attachments/754944979fe30484224908a4146a251a654a2b4277fd38e6538db5038a2dbe6f.json3,6 KB754944979fe30484224908a4146a251a654a2b4277fd38e6538db5038a2dbe6f
attachments/792d4b988753670ee9cc03db4192ed518abf0c52f7623d0ea2265346bdc07367.json100 B792d4b988753670ee9cc03db4192ed518abf0c52f7623d0ea2265346bdc07367
attachments/7c0b1a6319f64d4ee478407dce92e154ae2cc7796f5e550997e2513df2f0c25c.json2,7 KB7c0b1a6319f64d4ee478407dce92e154ae2cc7796f5e550997e2513df2f0c25c
attachments/802e97350f01af7a7b701222ec8b959674a6930150cef990491d6c8078e95b2c.json2,9 KB802e97350f01af7a7b701222ec8b959674a6930150cef990491d6c8078e95b2c
attachments/81471ffb0417b8b2f0490aa5509fc51d7f0dffcf1e0eaff4258dec92eb558de5.json213 B81471ffb0417b8b2f0490aa5509fc51d7f0dffcf1e0eaff4258dec92eb558de5
attachments/82c5e6fd222efb658aea9e11b27c7e3cde1d2698d9e7d6b035c79ac975245c39.json501 B82c5e6fd222efb658aea9e11b27c7e3cde1d2698d9e7d6b035c79ac975245c39
attachments/84c256b1a1aaa561e97df14545e791c7d9dffbc0a196fd4fa9d20833ed858bee.json4,9 KB84c256b1a1aaa561e97df14545e791c7d9dffbc0a196fd4fa9d20833ed858bee
attachments/88a186a8ca8bbf838abd2f4637dce901891a2f029d213a79da4d28e227091ad4.json658 B88a186a8ca8bbf838abd2f4637dce901891a2f029d213a79da4d28e227091ad4
attachments/8bf4da78955bfad1d4e418ba22242184985348ce475699116ce22cf84d14c4ef.json231 B8bf4da78955bfad1d4e418ba22242184985348ce475699116ce22cf84d14c4ef
attachments/8c4ae5c422fc85f93c84a79a45199aa0e0253c032d3342ee70b45cbc92ae6a95.json592 B8c4ae5c422fc85f93c84a79a45199aa0e0253c032d3342ee70b45cbc92ae6a95
attachments/8ded49f8b6094f0852eb527f13c4e9b22e6e20078cc05edf432ce346d02e0650.json2,6 KB8ded49f8b6094f0852eb527f13c4e9b22e6e20078cc05edf432ce346d02e0650
attachments/8ee0d84c29b117269608a8edad648afadcb20e12f00a5d88b7d64223f4b057e2.json258 B8ee0d84c29b117269608a8edad648afadcb20e12f00a5d88b7d64223f4b057e2
attachments/8f7b4b0b2247943b8a01d67ed59fe68e4132bd8661ac91bea4d2dbda13ee00ab.json218 B8f7b4b0b2247943b8a01d67ed59fe68e4132bd8661ac91bea4d2dbda13ee00ab
attachments/93b392f90709e1439d8c1ddb8d2f645b5a13b24e4c230c4d05fc62cee176d57c.json4,5 KB93b392f90709e1439d8c1ddb8d2f645b5a13b24e4c230c4d05fc62cee176d57c
attachments/9479a60c63eddde5d07e33821debb6574a211719ecd4b768e6b7419d2dcdd3d5.json705 B9479a60c63eddde5d07e33821debb6574a211719ecd4b768e6b7419d2dcdd3d5
attachments/975a56aa97613315af5618e14f8fdfd457e5235b15deeed680e973a92cc30a3e.json324 B975a56aa97613315af5618e14f8fdfd457e5235b15deeed680e973a92cc30a3e
attachments/98141f23a166e72a434373546492c62b114885e6ec50351e191e0b83028a61b9.json2,4 KB98141f23a166e72a434373546492c62b114885e6ec50351e191e0b83028a61b9
attachments/9cd2c9d6d14441f0ffe18441498c153896f8dfbc664252158789d36d0610f186.json2,4 KB9cd2c9d6d14441f0ffe18441498c153896f8dfbc664252158789d36d0610f186
attachments/9e4b049cb99d0d44f73c1b0110c604cef2bbbcbd2011979f9d4defbcc4b0ca71.json215 B9e4b049cb99d0d44f73c1b0110c604cef2bbbcbd2011979f9d4defbcc4b0ca71
attachments/a202dc80ccb8a617115b1f37fb031a2cfac9690f010f481417b10bc848d1358a.json1,7 KBa202dc80ccb8a617115b1f37fb031a2cfac9690f010f481417b10bc848d1358a
attachments/a2e4b1b88960bcb656abaeee2363185de61b6bf9700dadf32982f5f67cb98fa6.json6,3 KBa2e4b1b88960bcb656abaeee2363185de61b6bf9700dadf32982f5f67cb98fa6
attachments/a34ea42e19034a8690b888d9343cbfbf785722ecdf002b1641c314b887822296.json592 Ba34ea42e19034a8690b888d9343cbfbf785722ecdf002b1641c314b887822296
attachments/a382f0cec7a812f186815e695971eff6f417a88887738fa709e0ded4956c09d5.json324 Ba382f0cec7a812f186815e695971eff6f417a88887738fa709e0ded4956c09d5
attachments/a44ef51f03df5188f084f1e0227b68df116b942a22fd50c0a88670cf3461fedd.json65,0 KBa44ef51f03df5188f084f1e0227b68df116b942a22fd50c0a88670cf3461fedd
attachments/aaa9e222ac9b0b21fad4e0a7923ec6b042b7bec7a18ff08b430c1cebb3065677.json116 Baaa9e222ac9b0b21fad4e0a7923ec6b042b7bec7a18ff08b430c1cebb3065677
attachments/acbca120cb7cfc116b115d352f33496d254457ff0b75daad694da60c6709a738.json705 Bacbca120cb7cfc116b115d352f33496d254457ff0b75daad694da60c6709a738
attachments/b00a914eba0b9049ce4a0dc9b129184b17a10f4c37ba23e4b7ee6c08c5cf7be7.json2,8 KBb00a914eba0b9049ce4a0dc9b129184b17a10f4c37ba23e4b7ee6c08c5cf7be7
attachments/b020c20cb40602777f1bcbbd73bb164df2e239760439762c0a37309cf93bb157.json1,4 KBb020c20cb40602777f1bcbbd73bb164df2e239760439762c0a37309cf93bb157
attachments/b08f31ae9753af9b889c211e5210da9d1d212b43c6d6c079a41d6b47fc980d06.json1,5 KBb08f31ae9753af9b889c211e5210da9d1d212b43c6d6c079a41d6b47fc980d06
attachments/b1b500746bc54e442044fd43a1f92e3d2fa450d46906d678c80c7dc2744cd88b.json324 Bb1b500746bc54e442044fd43a1f92e3d2fa450d46906d678c80c7dc2744cd88b
attachments/b1d867dd18974c1d0ebbdad5fbeaa14525cd3f2da73d4f5a7498426d6b31710b.json4,4 KBb1d867dd18974c1d0ebbdad5fbeaa14525cd3f2da73d4f5a7498426d6b31710b
attachments/b54fc060d0519983ed8a452314f9e411ed47df6d3f2863e1c70c39a8d7708da4.json326 Bb54fc060d0519983ed8a452314f9e411ed47df6d3f2863e1c70c39a8d7708da4
attachments/b99efd4bda17795ad4b0489574296f75df1c2a082193a910621df8ea07c3ef38.json658 Bb99efd4bda17795ad4b0489574296f75df1c2a082193a910621df8ea07c3ef38
attachments/bb48370615edd65d2e419432264f70cea1c32d1266458edc75d201be3befce31.json4,0 KBbb48370615edd65d2e419432264f70cea1c32d1266458edc75d201be3befce31
attachments/bbc296ca3cc3a881a50440cadf1fa013c70d8dcde6b7dd9aedc171a8c7681ff6.json326 Bbbc296ca3cc3a881a50440cadf1fa013c70d8dcde6b7dd9aedc171a8c7681ff6
attachments/c08892b07a2d4b8129e2cceffcbcda3fe62158745146e58eadc4280a52866f9a.json3,6 KBc08892b07a2d4b8129e2cceffcbcda3fe62158745146e58eadc4280a52866f9a
attachments/c11969028f956b1fbe465ef2bff11b81d38857235171024899b2566e565ba6ed.json500 Bc11969028f956b1fbe465ef2bff11b81d38857235171024899b2566e565ba6ed
attachments/c1f145de881a59c1df5968d64403303dcb9e8c53a33492b84d705d1068bf0c0e.json660 Bc1f145de881a59c1df5968d64403303dcb9e8c53a33492b84d705d1068bf0c0e
attachments/c3cbace00f23eb7b953657b4f638e04941cadb0aa30e611cb989d173a50c72d1.json264 Bc3cbace00f23eb7b953657b4f638e04941cadb0aa30e611cb989d173a50c72d1
attachments/c3fce99bad2740aa94c47df047124de044ac16769075e11f79121b3c001f476c.json285 Bc3fce99bad2740aa94c47df047124de044ac16769075e11f79121b3c001f476c
attachments/c6be1bd1076301b9e62731a7053613f18ea6777ef26208deac6be6bc4639b252.json2,4 KBc6be1bd1076301b9e62731a7053613f18ea6777ef26208deac6be6bc4639b252
attachments/c70d76661cbebe476ed93035222f392263617caba3228ec27b89f01ceb0a5a1f.json361 Bc70d76661cbebe476ed93035222f392263617caba3228ec27b89f01ceb0a5a1f
attachments/c88389b91263c3b0e9a5be418c78bcd4d246fb9ffd0650b2ec6661567d7c7c98.json1,5 KBc88389b91263c3b0e9a5be418c78bcd4d246fb9ffd0650b2ec6661567d7c7c98
attachments/d1ee3bae920651888dce45594a6af202534cf3cfcb60793beff4c7256a1cae2e.json1,9 KBd1ee3bae920651888dce45594a6af202534cf3cfcb60793beff4c7256a1cae2e
attachments/d4b73ee514009356c62bf41e4776da3ecbfad1a0b641e551135e23313fc15d0c.json2,4 KBd4b73ee514009356c62bf41e4776da3ecbfad1a0b641e551135e23313fc15d0c
attachments/d6befb569906b12b313b920edf6c4dfc6cb9b22d872fe0e1733061b80a32581f.json1,4 KBd6befb569906b12b313b920edf6c4dfc6cb9b22d872fe0e1733061b80a32581f
attachments/d9d01812838b30346d3672fd3a619295ff5dfedaad16542071873c1bc589bece.json332 Bd9d01812838b30346d3672fd3a619295ff5dfedaad16542071873c1bc589bece
attachments/da09e0d3c7b99995c2881f0f1f2a7dedeaec4db80ef67ae358fdc94bfb775f26.json104 Bda09e0d3c7b99995c2881f0f1f2a7dedeaec4db80ef67ae358fdc94bfb775f26
attachments/da13277cefc95c3b2ff841901b9c0b61e87c03868ea8bced81d52eb1ab8d9d30.json773 Bda13277cefc95c3b2ff841901b9c0b61e87c03868ea8bced81d52eb1ab8d9d30
attachments/dab696fbcd2128d938f69bdd0c4bbf765db4b662efb62d8c15ee2ba0afb0819d.json3,0 KBdab696fbcd2128d938f69bdd0c4bbf765db4b662efb62d8c15ee2ba0afb0819d
attachments/df11eb66f5d2cbd955d697cb532abc570e188ae86ce7da313da95f6439a86bf4.json2,1 KBdf11eb66f5d2cbd955d697cb532abc570e188ae86ce7da313da95f6439a86bf4
attachments/e09209beacdeae8cfdb29809dcdb4e22b85a5e59e1d87b8ee0df60914ca25451.json239 Be09209beacdeae8cfdb29809dcdb4e22b85a5e59e1d87b8ee0df60914ca25451
attachments/e28241a86349b90c8d73079ca9bc77b473917d82cc8d836e7e7773d17eabcec2.json1,5 KBe28241a86349b90c8d73079ca9bc77b473917d82cc8d836e7e7773d17eabcec2
attachments/e62a2f402eea14ac0645bfa149919f25f6c9491d0f4592724657ad5beede79e1.json707 Be62a2f402eea14ac0645bfa149919f25f6c9491d0f4592724657ad5beede79e1
attachments/e782970073bcebc925e35b4502e505ec44d39d83dcc8b82916c7c787f093c656.json212 Be782970073bcebc925e35b4502e505ec44d39d83dcc8b82916c7c787f093c656
attachments/ea40968d4354bc638bc471f64e09ec9a511a8a380afb683e8303ac72af1c5f99.json218 Bea40968d4354bc638bc471f64e09ec9a511a8a380afb683e8303ac72af1c5f99
attachments/f23aa25a251ff3e22e82c438e644e7a52ebe8f3d8a76a60a1bf3f41513f7d698.json22,0 KBf23aa25a251ff3e22e82c438e644e7a52ebe8f3d8a76a60a1bf3f41513f7d698
attachments/f376b8f7368a7c179af4fe0f65c756eb413acc42a0fc8b119a78cda1c65259cf.json2,2 KBf376b8f7368a7c179af4fe0f65c756eb413acc42a0fc8b119a78cda1c65259cf
attachments/f45c07ab8730d35a830bbbba4cce4dd69af9ff69b07d7ff9a006b2d9eed85020.json393 Bf45c07ab8730d35a830bbbba4cce4dd69af9ff69b07d7ff9a006b2d9eed85020
attachments/f4c4bd3c8cb8138ffc7c445060668433e1b882e270c630ec8340847d6621cdde.json771 Bf4c4bd3c8cb8138ffc7c445060668433e1b882e270c630ec8340847d6621cdde
attachments/f75b4d665cceb38262f8a636bbb1f52772514d9f9b0a431a8074b6eff2ed70a0.json265 Bf75b4d665cceb38262f8a636bbb1f52772514d9f9b0a431a8074b6eff2ed70a0
attachments/f92199106ccb7974d59a973104b207ca01b3f54995e29990569573a2c143773c.json502 Bf92199106ccb7974d59a973104b207ca01b3f54995e29990569573a2c143773c
attachments/fbc20c71574a74008bb9817ac8505cfc2b4ef580f7022e396b82769556385f24.json707 Bfbc20c71574a74008bb9817ac8505cfc2b4ef580f7022e396b82769556385f24
attachments/fbc4c544df788e08de2788b03fe6e44e937b9943c5a994d565043c2c7d04cb3f.json271 Bfbc4c544df788e08de2788b03fe6e44e937b9943c5a994d565043c2c7d04cb3f
attachments/feca7c003ba662c38dbddf12b4d63c5dab820e3ebb15c252c75b0ae994f64ce2.json215 Bfeca7c003ba662c38dbddf12b4d63c5dab820e3ebb15c252c75b0ae994f64ce2
env/platform.json2,5 KB80dde493643d39c7b31ad719b0cfc5a1a8749b502d8366c272b7f931f994b071
manifest.json12,7 KB69ade86daa0a11961c6dc712909e1e00ca21e7a0f89aa681851976431495580a
native/causcibench_real/causcibench_real.q001.e1.a1.json1,5 KB8bf6842acb0b5fa3760f143c073c2100fcecb1ff3b15bf4b7979c96407769056
native/causcibench_real/causcibench_real.q002.e1.a1.json1,5 KB14d6383c39cf71c700f5197574560780fb8ac31012a8d77bd290486993e92806
results.jsonl10,4 KBf71437395f081bdcb9fd5299b75b74ce1ea3847d2393ecc4b2b23cc27481c3ee
sources/causcibench_real.gold.q001.json1,2 KBa36743225dcc4d52ebf08506d9e70d53b695c269559e9d061d436cf4ac35c8ea
sources/causcibench_real.gold.q002.json1,2 KB14fdf6a33c32bc45b86ba7fc8c2aa242ff0a38b03b1e3ea6c63f00260a896e31
sources/causcibench_real.leaderboard.gemini25flashlite.mre.cot.json1,6 KB0af4992264b171a253ec5b1f019f4759f604c7997dc46a7166aae32517e60b3c
sources/causcibench_real.leaderboard.gemini25flashlite.mre.direct.json1,6 KB8f0d99f6c12056e4a021344fa0358afcde945cd52aafb02f55845cce6c37412d
sources/causcibench_real.leaderboard.gemini25flashlite.mre.pot.json1,6 KB64291472e13088ee8956950e47389038a7877e25c08f392bce137f93fe844c23
sources/causcibench_real.leaderboard.gemini25flashlite.mre.react.json1,6 KB0c443dff6dc1fb2395e640211941972026188b838f66cd81568dc0dfb7d6f4ab
sources/causcibench_real.leaderboard.gemini25flashlite.msa.cot.json1,6 KBd5748c5c921795e220f141c704da29f1ffa6bd1f2bb15fc5317cf4e181070010
sources/causcibench_real.leaderboard.gemini25flashlite.msa.direct.json1,6 KB48124da5c811ecc1eaf7d5c7721cdadc9482f1e46471a697a8873351fc7ed2ba
sources/causcibench_real.leaderboard.gemini25flashlite.msa.pot.json1,6 KB9bf161a8adb8965014b96a8ad262fa5d3040e8f75888fcabb17b84ff89660b3e
sources/causcibench_real.leaderboard.gemini25flashlite.msa.react.json1,6 KBcf409ba4b60386d42870a3c72d91b61e74dee0d77a52e303a9ee1c52fe6f9a6d
sources/causcibench_real.leaderboard.gpt4o.mre.cot.json1,5 KBc0a65816ba60c292c25566ce4d0f27a75fae14c9caa14ec1ccba1a287c63e7ec
sources/causcibench_real.leaderboard.gpt4o.mre.direct.json1,5 KB4fa4a65d1e4f86f005b6cccb9117eb57450cddee0bfd63787d7acf43e0ec7ee6
sources/causcibench_real.leaderboard.gpt4o.mre.pot.json1,5 KB1d441b6a22b2319ced8d5cc1563e4a543dec69d1d783c50e97718aa59ab4d167
sources/causcibench_real.leaderboard.gpt4o.mre.react.json1,5 KB39c20d439e14b01ae267d51e41e596a03750a2fd7944d2d15c03258f8769dc8a
sources/causcibench_real.leaderboard.gpt4o.msa.cot.json1,5 KBcb004835af947074419bc56d1472f162f3e951da83d7083ae89a998d23d41711
sources/causcibench_real.leaderboard.gpt4o.msa.direct.json1,6 KB1d263ded6f92b86e7a82592c1e53ee5b830231746891eb0cc5369897d587c30d
sources/causcibench_real.leaderboard.gpt4o.msa.pot.json1,5 KB1522366e04ac7e4c6df3881d9de51f5263cfc35b3a74407c095268c25ab03377
sources/causcibench_real.leaderboard.gpt4o.msa.react.json1,6 KB7fd1a17bfe5fff02c4e324ad94316d6a083df9180f5fd3428b3dc93b7192d8ef
sources/causcibench_real.leaderboard.gpt5mini.mre.cot.json1,5 KBf4f267ec1df17e9daca623947d5f17dfaac0d9a87557f370832eaf4beabe0db4
sources/causcibench_real.leaderboard.gpt5mini.mre.direct.json1,5 KB610ca727b5bb0adb21c156ac1d1dbad5c19fbf0e312609a9a25ade925217f056
sources/causcibench_real.leaderboard.gpt5mini.mre.pot.json1,5 KB2cd434e9dc52c799be4f11362fcd14ee19e2f53f87889e088407abd689c9a460
sources/causcibench_real.leaderboard.gpt5mini.mre.react.json1,5 KBad783d4a63e3a6889b0f962e4b645e9622c9da157a5cd59a9cd4691a137af063
sources/causcibench_real.leaderboard.gpt5mini.msa.cot.json1,6 KBc6682a36b053c24ab0c00cc249e89e62698ceb3ad922a97f1d3c21c9c1ecaf27
sources/causcibench_real.leaderboard.gpt5mini.msa.direct.json1,6 KBf99042cdb96914eacca0cf830763b8ceb0189ee94882a09f4b30445901ba83ed
sources/causcibench_real.leaderboard.gpt5mini.msa.pot.json1,6 KB730385e2c2ba909941bfce933c951d05ea26c20bbc95a626f78f29480b445d3d
sources/causcibench_real.leaderboard.gpt5mini.msa.react.json1,6 KBdf663ba62a8e078e83bf169674c05f7d0816026b93b06435de5b8238b44509b8
sources/causcibench_real.leaderboard.o3.mre.cot.json1,5 KBed990e207f14e3a9628d640f166e783da9a30bdd12c6b27a4d1ac1e32001ac2a
sources/causcibench_real.leaderboard.o3.mre.direct.json1,5 KB0d38401c809cdd6c7e89f65ac6d009be05382916d6503d029a850174b98af4c1
sources/causcibench_real.leaderboard.o3.mre.pot.json1,5 KBb091aa957db22645885019edb7198a1ad4ad37e0b8f851cec5000c239b3523c7
sources/causcibench_real.leaderboard.o3.mre.react.json1,5 KBc831ec226976aff9d9a004f78fe673286c3bce316eb4f773f1969652e463cb20
sources/causcibench_real.leaderboard.o3.msa.cot.json1,5 KB975f5b5c3784e59f5acba7a0604cd67056b3d44c8730a25ba54b94ef6c8dcaea
sources/causcibench_real.leaderboard.o3.msa.direct.json1,6 KB6fc283734f726b9b7ae849eaa19c5d6182b8a8d7b37d3aebd417bce862bdca4a
sources/causcibench_real.leaderboard.o3.msa.pot.json1,5 KB352238e8c2f18d2ba753bb5c633ede12cd0db589c536c99276a124df2bc1fc7e
sources/causcibench_real.leaderboard.o3.msa.react.json1,6 KB4c8bc674705073895ec1f3b08c06633b4f1a407c6eeedb71d94242cc7acb34b6
specs/causcibench_real.q001.json5,0 KBae1d1786fda5ea67f0ec777a2330bfa27a96efb55b38f73f46b03758d66887d1
specs/causcibench_real.q002.json5,0 KB05c8f73d3b63b31887317785abaef4f63550bcd5d2a69073321a6f59f1c27fe9
summary.json2,2 KB1eabaea0fe646b17ab7ea3c899a14c5e60227585e87a7b4a94699f164d35c8db
summary.md587 Be35712ee21847be7e05ff5183322d310326faceeb6f3c409d21b0e5ac53cac52
tools/registry.json14,3 KB644bdc32c1ef8e99eea9ba91f16d8c477a4c57d26b17990e22e346b33aa067fe
traces/causcibench_real.q001.e1.a1.jsonl75,8 KBe470d13fe6e38af6cdd243836e0accdaa1840cb9f786ec00f86afa21a8c3449b
traces/causcibench_real.q002.e1.a1.jsonl70,3 KBd72c7733af72b6499b2da35515ca928b5eff069cd5621460e1fb8a8d0d6b23f4
trajectories/causcibench_real.q001.e1.a1.jsonl64,3 KBad26622bb758728a644f3ac075cad27aa5938909d41c6ca81acc7679bcd7d1ff
trajectories/causcibench_real.q002.e1.a1.jsonl59,7 KB3208b2d660248dbd206c85f76b002bde8499e0ebe90fe7490031b81979a284a5
verify_bundle.py34,9 KB99e037f6a241e077348d7e116b03be17e727c4b61d278ea5f39b06dd557069a3
SHA256SUMS21,7 KB999d2b83d9a490e4f3f2bafead28ab4f9a81f2dcccc8d5e249e3f17fee18ed06

Eventos do registro

Verifique você mesmo

Resumo da execução
999d2b83d9a490e4f3f2bafead28ab4f9a81f2dcccc8d5e249e3f17fee18ed06
sha256 do arquivo SHA256SUMS; deve ser igual ao run_digest do evento final do registro.

Qualquer pessoa, sem o repositório

  1. Baixe o pacote inteiro e descompacte.
  2. Rode o verificador, que só usa a biblioteca padrão do Python:
    cd 20261015T124546Z-suite-causcibench_real-gemini-c5831e
    python3 verify_bundle.py

Ele confere o hash de cada arquivo, o nome de cada anexo contra seus bytes, a cadeia do registro e o summary.json recalculado a partir de results.jsonl.

Para conferir o resumo:

shasum -a 256 SHA256SUMS   # 999d2b83d9a490e4f3f2bafead28ab4f9a81f2dcccc8d5e249e3f17fee18ed06

Com acesso ao repositório (privado)

git checkout 38e979372a0f7382f285079e4380813081bdd032
uv sync --frozen --all-packages
uv run magi-bench verify bench-runs/20261015T124546Z-suite-causcibench_real-gemini-c5831e

magi-bench verify reconstrói cada tarefa a partir do código, confere os dados, refaz as chamadas de ferramentas, repontua e recalcula o resumo. Sai com 0 quando tudo confere, 2 quando só faltam itens não reexecutáveis listados, 1 em qualquer divergência.