Pular para o conteúdo

Tarefa causcibench_real.q084

causcibench_realagentT2publicépoca 1, tentativa 1Execução: 20261015T171059Z-suite-causcibench_real-scripted-ded01d

Veredito

passouPassou em todas as verificações obrigatórias.

Família do método (das chamadas de ferramentas): ols (regression)

Pergunta feita ao analista

Does labeling COVID-19 as the 'Chinese Virus' causally affect the level of blame the public assigns to the federal government?

Resposta

Estimativa (infer.ols): coef.ch = 0,9869.

Valores esperados e de onde vêm

causcibench_real.gold.q084

Fonte
CauSciBench real split, data/metadata_json/real_input.json at d9bb43a, id 84
Tipo
benchmark_gold
Localização
real_input.json, record id 84: fields effect and std_error
Trecho
id 84 effect 0.987 std_error 1.808
Valor como publicado
0.987
Erro-padrão como publicado
1.808
Valor
0,9870.987 (se 1,8081.808)
Conferido por
arquivo do benchmark · Written by scripts/freeze_causcibench_real.py from the pinned file; equal to the `answer` and `std_error` of the same id in data/metadata_csv/real_input.csv at the pin.
sha256 da cópia lida
c1f4c8930675afff63d816d8ed066941ae92697e7bfc31e9d38519398dd80438 (identifica a cópia lida, não um arquivo canônico)
Endereço
https://raw.githubusercontent.com/causalNLP/CauSciBench/d9bb43a2d90f261978cc5ac0b628c0484f6e8956/data/metadata_json/real_input.json
Licença
CauSciBench repository (MIT for code; metadata as published in the repository)
  • #1 figura da resposta próxima: 0,9870.987 · relativa 0.05

Verificações

Cada verificação, o que esperava, o que observou e a chamada que leu
#VerificaçãoObrigatóriaResultadoEsperadoObservadoTolerânciaFonteChamada
0resposta com lastro
every number backed
obrigatóriapassou—[]———
1figura da resposta próxima
0.9868697099 vs 0.987
obrigatóriapassou0,9870.9870,986870.9868697099037835relativa 0.05causcibench_real.gold.q084call_68e1c275

Caminho de referência e caminho do agente

A chamada de referência ao lado da chamada que deu a resposta, argumento por argumento.

caminho de referência do item · metadata_rule

Referência: infer.olsChamada do agente: infer.ols (call_68e1c275)igual
Argumentos da referência e do agente
ArgumentoReferênciaAgente
cluster——não fixado pela referência
controls[][]igual
dataset"xu_does_mislabelling""xu_does_mislabelling"igual
fe—[]não fixado pela referência
intercept—truenão fixado pela referência
level—0,950.95não fixado pela referência
se—"HC1"não fixado pela referência
subset——não fixado pela referência
weights——não fixado pela referência
x["ch"]["ch"]igual
y"blame""blame"igual

Trajetória

Cada evento na ordem em que aconteceu: turnos do modelo, chamadas de ferramentas, guardas, transformações e pontuação.

  1. 0Início

    Does labeling COVID-19 as the 'Chinese Virus' causally affect the level of blame the public assigns to the federal government?

    xu_does_mislabelling (agent)

    This dataset comes from an online survey experiment conducted in the United States on Amazon Mechanical Turk (MTurk) in March 2020, during the early stages of the COVID-19 pandemic. A total of 1,200 adult participants were included after excluding incomplete responses and failed manipulation checks. Respondents were randomly assigned to one of two groups: a control group where the virus was labeled COVID-19 and a treatment group where it was labeled the ""Chinese Virus"". The goal of the experiment was to test whether these labels influenced attitudes toward Chinese immigrants and shifted blame toward or away from the federal government. To capture sensitive views, the design combined a list experiment with direct survey questions. Dataset Variables: duration: the time subjects spent in survey cblame_1: blame in COVID19 group tblame_1: blame in ""Chinese Virus"" group mc: answer of the manipulation check question lc_c: list item count in 3-item COVID19 group direct_c: answer of the direct question in the 3-item COVID19 group lt_c: list item count in 4-item COVID19 group lc_t: list item count in 3-item ""Chinese Virus"" group direct_t: answer of the direct question in the 3-item ""Chinese Virus"" group lt_t: list item count in 4-item ""Chinese Virus"" group sex: 1=male; 0=female age: subjects' age race: subjects' race income: subjects' income worry: how worried subjects are about COVID-19 exposure twitter: frequency of using Twitter/Facebook party: party affiliation lean: party lean ideology: political ideology state: subjects' location state ch: 1=""Chinese virus"" group, 0=COVID19 group covid: item number in COVID19 group (1=4-item; 0=3-item) virus: item number in ""Chinese Virus"" group (1=4-item; 0=3-item) list: item subgroup (1=4-item group; 0=3-item group) blame: blame direct: answer of direct question ict.c: item count in COVID19 group ict.t: item count in ""Chinese Virus"" group ict: item count Y.c: social desirability measure in COVID19 group Y.t: social desirability measure in ""Chinese Virus"" group ido: numeric ideology measure ido1: simplified measure of ideology inc: numeric income measure wor: numeric measure of worry social: numeric measure of social media use ID: subject ID item: 3-item or 4-item ch1: a=COVID19; b=""Chinese Virus"" male: 1=male; 0=female female: 1=female; 0=male young: 1=age<=29 midage: 1=29<age<=49 old: 1=age>49 white: 1=white; 0=not white blk: 1=black; 0=not black his: 1=hispanic; 0=not hispanic asian: 1=asian; 0=not asian other: 1=other; 0=not other hwor: 1=worried or very worried; 0=not too worried or not worried at all lwor: 0=worried or very worried; 1=not too worried or not worried at all hsocial: 1=""social"">=2; 0=""social""<2 lsocial: 1=""social""<2; 0=""social"">=2 Dem: 1=Democrat; 0=other Rep: 1=Republican; 0=other Ind: 1=Independent; 0=other lib: 1=Liberal; 0=other con: 1=Conservative; 0=other mod: 1=Moderate; 0=other low: 1=Less than $25,000; 0=other med: 1=""inc"">=2 and <5; 0=other high: 1=""inc"">=5; 0=other mcr: 1=answer correct in manipulation check question; 0=answer wrong group: covid3=COVID19 3-item group; covid4=COVID19 4-item group; virus3=""Chinese Virus"" 3-item group; virus4=""Chinese Virus"" 4-item group

    file 7432cf2fb45b00f547e433f243fdbd8ecadf01b8141adbba4130824522108855 · frame —
  2. 1Turno do modelo
    call_8b9b88eereferenceFim: tool_use0 ms0 de entrada · 0 de saída · 0 de raciocínio · 0 em cache
    Texto visível

    causcibench_metadata_rule/1: ols

    Chamadas pedidas
    • infer.ols
      {
        "controls": [],
        "dataset": "xu_does_mislabelling",
        "x": [
          "ch"
        ],
        "y": "blame"
      }
  3. 2Ferramenta
    infer.olsv0.1.0call_68e1c275decidido pelo modeloreexecutável

    Motivo: causcibench_metadata_rule/1: ols

    Argumentos
    {
      "cluster": null,
      "controls": [],
      "dataset": "xu_does_mislabelling",
      "fe": [],
      "intercept": true,
      "level": 0.95,
      "se": "HC1",
      "subset": null,
      "weights": null,
      "x": [
        "ch"
      ],
      "y": "blame"
    }
    Figuras call_68e1c275
    ChaveValorValor brutoErro-padrãoIC
    coef.ch
    Coefficient on ch in the regression of blame
    0,98687 0.98686970990378351,808621.8086183986590711-2,56155 – 4,53528
    Diagnósticos
    • n_obs = 1.1991199
    • k = 22
    • r2 = 0,0002487430.0002487427166812939
    • df_resid = 1.1971197
    • se_type = "HC1"
    • n_clusters = —
    Tabelas (1)
    {
      "coefficients": [
        {
          "ci_high": 51.81427253544662,
          "ci_low": 46.95111847952842,
          "coef": 49.38269550748752,
          "p": 1.0735948976427613e-221,
          "se": 1.2393688679866675,
          "t": 39.845034664868436,
          "term": "Intercept"
        },
        {
          "ci_high": 4.535284604064961,
          "ci_low": -2.561545184257393,
          "coef": 0.9868697099037835,
          "p": 0.58540932290685,
          "se": 1.8086183986590711,
          "t": 0.5456483858814326,
          "term": "ch"
        }
      ]
    }
    • 1 rows were dropped because one of 'blame', 'ch' was missing
    Digests dos dados
    xu_does_mislabelling 4ed38f68ca3d0323406ceb338cfe173ce776e8166aa30250c169c61662b6c20f
    sha256 do resultado
    7d9f1dc8179a977671118d0f54d218be8a50de89e8873e67342908be88456daf
  4. 3Turno do modelo
    call_ad8a8e4areferenceFim: tool_use0 ms0 de entrada · 0 de saída · 0 de raciocínio · 0 em cache
    Chamadas pedidas
    • submit_answer (meta-ferramenta)
      {
        "answer": "Estimativa (infer.ols): coef.ch = 0,9869.",
        "caveats": [],
        "figure_refs": [
          "call_68e1c275:coef.ch"
        ]
      }
  5. 4Guarda
    resposta aceita
    Números encontrados
    0,9869
    Sem lastro
    —
    Não citados
    —
    1 nova(s) tentativa(s) restante(s)
  6. 5Resposta

    Estimativa (infer.ols): coef.ch = 0,9869.

    Figuras citadas: call_68e1c275:coef.ch · Parada: answered
    Figura da resposta: call_68e1c275:coef.ch = 0,986870.9868697099037835 (se 1,808621.8086183986590711)
  7. 6Pontuação
    #0resposta com lastropassouobrigatóriaevery number backed
  8. 7Pontuação
    #1figura da resposta próximapassouobrigatória0.9868697099 vs 0.987

Épocas × verificações

Resultado de cada verificação em cada época pontuada
#01
Época 1passoupassou

Tentativas

Cada tentativa é seu próprio arquivo e é publicada; só a última de cada época é pontuada.

Downloads desta tarefa

Pacote da tarefa (.zip)
100,3 KB · sha256 dd6636278438253841925b1bae21e494a42d805013bec1a80e9edd27a7002d00

O hash de um zip é só de transporte: o que ancora a integridade é o RUN_SHA256SUMS que vai dentro dele, cópia exata do SHA256SUMS da execução.

Downloads desta tarefa
ArquivoTamanhosha256
attachments/3fd37b7dca6f37f4b42a3a8a4170db47d464da2845d476f0b3be2606a52303d0.json151 B3fd37b7dca6f37f4b42a3a8a4170db47d464da2845d476f0b3be2606a52303d0
attachments/433a6877aa6a378bbd4b7e2752e484d8cb7499c562d68222be231f62ff2fa420.json2,4 KB433a6877aa6a378bbd4b7e2752e484d8cb7499c562d68222be231f62ff2fa420
attachments/52ab03d8ec104abb85572294bf8a469d75c5de9e74d6042caabff8017adc572b.json64,0 KB52ab03d8ec104abb85572294bf8a469d75c5de9e74d6042caabff8017adc572b
attachments/5601e98d9382cd409b12d9c742b8d7ef544c869534b0bd9e74e3b35a90d5e78f.json1,0 KB5601e98d9382cd409b12d9c742b8d7ef544c869534b0bd9e74e3b35a90d5e78f
attachments/c1f32b32f40f6b3f93158632162f9284b63c06da6fd563e1bfbd1c0ad260a18e.json5,2 KBc1f32b32f40f6b3f93158632162f9284b63c06da6fd563e1bfbd1c0ad260a18e
attachments/df7f8a08493adc10cca9aa3513c98146ddab2e6f88be1c7a4ff8c2fc09829210.json194 Bdf7f8a08493adc10cca9aa3513c98146ddab2e6f88be1c7a4ff8c2fc09829210
attachments/dfbde396a71243351236f2d53820dafe90c965895cb6be321491ef9abca8ed4a.json923 Bdfbde396a71243351236f2d53820dafe90c965895cb6be321491ef9abca8ed4a
native/causcibench_real/causcibench_real.q084.e1.a1.json1,7 KB43bc08937116c841a9ef2c59e4de45a1dd844f7531e38cef5626d4ab480108e2
sources/causcibench_real.gold.q084.json1,2 KB57ba0457d19e24785a170a87cc411e56f2282c83214685310e29f0f9a06fa29a
specs/causcibench_real.q084.json7,0 KBf35ebe268993cca25df31c83adcaf6e736230c847fa443fa917f9dfc7614aef2
traces/causcibench_real.q084.e1.a1.jsonl15,9 KB444ee7d9d71bc07df38f4f07084137dbfa01fde6a3cfa5c2ec57e0aaa0a79089
trajectories/causcibench_real.q084.e1.a1.jsonl11,6 KB6d3560d7dbca60480342f946b752e67162499c35bf9cde7f5b52c3e86165ff54