Pular para o conteúdo

Tarefa causcibench_real.q084

causcibench_realagentT2publicépoca 1, tentativa 1Execução: 20261015T171142Z-suite-causcibench_real-naive-d6494d

Veredito

passouPassou em todas as verificações obrigatórias.

Família do método (das chamadas de ferramentas): ols (regression)

Pergunta feita ao analista

Does labeling COVID-19 as the 'Chinese Virus' causally affect the level of blame the public assigns to the federal government?

Resposta

Estimativa ingênua (infer.ols): coef.ch = 0,9869.

Valores esperados e de onde vêm

causcibench_real.gold.q084

Fonte
CauSciBench real split, data/metadata_json/real_input.json at d9bb43a, id 84
Tipo
benchmark_gold
Localização
real_input.json, record id 84: fields effect and std_error
Trecho
id 84 effect 0.987 std_error 1.808
Valor como publicado
0.987
Erro-padrão como publicado
1.808
Valor
0,9870.987 (se 1,8081.808)
Conferido por
arquivo do benchmark · Written by scripts/freeze_causcibench_real.py from the pinned file; equal to the `answer` and `std_error` of the same id in data/metadata_csv/real_input.csv at the pin.
sha256 da cópia lida
c1f4c8930675afff63d816d8ed066941ae92697e7bfc31e9d38519398dd80438 (identifica a cópia lida, não um arquivo canônico)
Endereço
https://raw.githubusercontent.com/causalNLP/CauSciBench/d9bb43a2d90f261978cc5ac0b628c0484f6e8956/data/metadata_json/real_input.json
Licença
CauSciBench repository (MIT for code; metadata as published in the repository)
  • #1 figura da resposta próxima: 0,9870.987 · relativa 0.05

Verificações

Cada verificação, o que esperava, o que observou e a chamada que leu
#VerificaçãoObrigatóriaResultadoEsperadoObservadoTolerânciaFonteChamada
0resposta com lastro
every number backed
obrigatóriapassou—[]———
1figura da resposta próxima
0.9868697099 vs 0.987
obrigatóriapassou0,9870.9870,986870.9868697099037835relativa 0.05causcibench_real.gold.q084call_de5c098a

Caminho de referência e caminho do agente

A chamada de referência ao lado da chamada que deu a resposta, argumento por argumento.

caminho de referência do item · metadata_rule

Referência: infer.olsChamada do agente: infer.ols (call_de5c098a)igual
Argumentos da referência e do agente
ArgumentoReferênciaAgente
cluster——não fixado pela referência
controls[][]igual
dataset"xu_does_mislabelling""xu_does_mislabelling"igual
fe—[]não fixado pela referência
intercept—truenão fixado pela referência
level—0,950.95não fixado pela referência
se—"HC1"não fixado pela referência
subset——não fixado pela referência
weights——não fixado pela referência
x["ch"]["ch"]igual
y"blame""blame"igual

Trajetória

Cada evento na ordem em que aconteceu: turnos do modelo, chamadas de ferramentas, guardas, transformações e pontuação.

  1. 0Início

    Does labeling COVID-19 as the 'Chinese Virus' causally affect the level of blame the public assigns to the federal government?

    xu_does_mislabelling (agent)

    This dataset comes from an online survey experiment conducted in the United States on Amazon Mechanical Turk (MTurk) in March 2020, during the early stages of the COVID-19 pandemic. A total of 1,200 adult participants were included after excluding incomplete responses and failed manipulation checks. Respondents were randomly assigned to one of two groups: a control group where the virus was labeled COVID-19 and a treatment group where it was labeled the ""Chinese Virus"". The goal of the experiment was to test whether these labels influenced attitudes toward Chinese immigrants and shifted blame toward or away from the federal government. To capture sensitive views, the design combined a list experiment with direct survey questions. Dataset Variables: duration: the time subjects spent in survey cblame_1: blame in COVID19 group tblame_1: blame in ""Chinese Virus"" group mc: answer of the manipulation check question lc_c: list item count in 3-item COVID19 group direct_c: answer of the direct question in the 3-item COVID19 group lt_c: list item count in 4-item COVID19 group lc_t: list item count in 3-item ""Chinese Virus"" group direct_t: answer of the direct question in the 3-item ""Chinese Virus"" group lt_t: list item count in 4-item ""Chinese Virus"" group sex: 1=male; 0=female age: subjects' age race: subjects' race income: subjects' income worry: how worried subjects are about COVID-19 exposure twitter: frequency of using Twitter/Facebook party: party affiliation lean: party lean ideology: political ideology state: subjects' location state ch: 1=""Chinese virus"" group, 0=COVID19 group covid: item number in COVID19 group (1=4-item; 0=3-item) virus: item number in ""Chinese Virus"" group (1=4-item; 0=3-item) list: item subgroup (1=4-item group; 0=3-item group) blame: blame direct: answer of direct question ict.c: item count in COVID19 group ict.t: item count in ""Chinese Virus"" group ict: item count Y.c: social desirability measure in COVID19 group Y.t: social desirability measure in ""Chinese Virus"" group ido: numeric ideology measure ido1: simplified measure of ideology inc: numeric income measure wor: numeric measure of worry social: numeric measure of social media use ID: subject ID item: 3-item or 4-item ch1: a=COVID19; b=""Chinese Virus"" male: 1=male; 0=female female: 1=female; 0=male young: 1=age<=29 midage: 1=29<age<=49 old: 1=age>49 white: 1=white; 0=not white blk: 1=black; 0=not black his: 1=hispanic; 0=not hispanic asian: 1=asian; 0=not asian other: 1=other; 0=not other hwor: 1=worried or very worried; 0=not too worried or not worried at all lwor: 0=worried or very worried; 1=not too worried or not worried at all hsocial: 1=""social"">=2; 0=""social""<2 lsocial: 1=""social""<2; 0=""social"">=2 Dem: 1=Democrat; 0=other Rep: 1=Republican; 0=other Ind: 1=Independent; 0=other lib: 1=Liberal; 0=other con: 1=Conservative; 0=other mod: 1=Moderate; 0=other low: 1=Less than $25,000; 0=other med: 1=""inc"">=2 and <5; 0=other high: 1=""inc"">=5; 0=other mcr: 1=answer correct in manipulation check question; 0=answer wrong group: covid3=COVID19 3-item group; covid4=COVID19 4-item group; virus3=""Chinese Virus"" 3-item group; virus4=""Chinese Virus"" 4-item group

    file 7432cf2fb45b00f547e433f243fdbd8ecadf01b8141adbba4130824522108855 · frame —
  2. 1Turno do modelo
    call_e3d20ee9naive-olsFim: tool_use0 ms0 de entrada · 0 de saída · 0 de raciocínio · 0 em cache
    Texto visível

    Linha de base ingênua: regressão do resultado no tratamento, sem controles.

    Chamadas pedidas
    • infer.ols
      {
        "dataset": "xu_does_mislabelling",
        "x": [
          "ch"
        ],
        "y": "blame"
      }
  3. 2Ferramenta
    infer.olsv0.1.0call_de5c098adecidido pelo modeloreexecutável

    Motivo: Linha de base ingênua: regressão do resultado no tratamento, sem controles.

    Argumentos
    {
      "cluster": null,
      "controls": [],
      "dataset": "xu_does_mislabelling",
      "fe": [],
      "intercept": true,
      "level": 0.95,
      "se": "HC1",
      "subset": null,
      "weights": null,
      "x": [
        "ch"
      ],
      "y": "blame"
    }
    Figuras call_de5c098a
    ChaveValorValor brutoErro-padrãoIC
    coef.ch
    Coefficient on ch in the regression of blame
    0,98687 0.98686970990378351,808621.8086183986590711-2,56155 – 4,53528
    Diagnósticos
    • n_obs = 1.1991199
    • k = 22
    • r2 = 0,0002487430.0002487427166812939
    • df_resid = 1.1971197
    • se_type = "HC1"
    • n_clusters = —
    Tabelas (1)
    {
      "coefficients": [
        {
          "ci_high": 51.81427253544662,
          "ci_low": 46.95111847952842,
          "coef": 49.38269550748752,
          "p": 1.0735948976427613e-221,
          "se": 1.2393688679866675,
          "t": 39.845034664868436,
          "term": "Intercept"
        },
        {
          "ci_high": 4.535284604064961,
          "ci_low": -2.561545184257393,
          "coef": 0.9868697099037835,
          "p": 0.58540932290685,
          "se": 1.8086183986590711,
          "t": 0.5456483858814326,
          "term": "ch"
        }
      ]
    }
    • 1 rows were dropped because one of 'blame', 'ch' was missing
    Digests dos dados
    xu_does_mislabelling 4ed38f68ca3d0323406ceb338cfe173ce776e8166aa30250c169c61662b6c20f
    sha256 do resultado
    27e9dcfd6ef2cf0f22c13c7344fa033fbf49ce9e3466ba86473bb2f2d27667f7
  4. 3Turno do modelo
    call_9b27bf6dnaive-olsFim: tool_use0 ms0 de entrada · 0 de saída · 0 de raciocínio · 0 em cache
    Chamadas pedidas
    • submit_answer (meta-ferramenta)
      {
        "answer": "Estimativa ingênua (infer.ols): coef.ch = 0,9869.",
        "caveats": [],
        "figure_refs": [
          "call_de5c098a:coef.ch"
        ]
      }
  5. 4Guarda
    resposta aceita
    Números encontrados
    0,9869
    Sem lastro
    —
    Não citados
    —
    1 nova(s) tentativa(s) restante(s)
  6. 5Resposta

    Estimativa ingênua (infer.ols): coef.ch = 0,9869.

    Figuras citadas: call_de5c098a:coef.ch · Parada: answered
    Figura da resposta: call_de5c098a:coef.ch = 0,986870.9868697099037835 (se 1,808621.8086183986590711)
  7. 6Pontuação
    #0resposta com lastropassouobrigatóriaevery number backed
  8. 7Pontuação
    #1figura da resposta próximapassouobrigatória0.9868697099 vs 0.987

Épocas × verificações

Resultado de cada verificação em cada época pontuada
#01
Época 1passoupassou

Tentativas

Cada tentativa é seu próprio arquivo e é publicada; só a última de cada época é pontuada.

Downloads desta tarefa

Pacote da tarefa (.zip)
107,9 KB · sha256 34a8128d4754702df4fb1d8d4d2854732b4049165c09abdf4982b4647560d381

O hash de um zip é só de transporte: o que ancora a integridade é o RUN_SHA256SUMS que vai dentro dele, cópia exata do SHA256SUMS da execução.

Downloads desta tarefa
ArquivoTamanhosha256
attachments/3fd37b7dca6f37f4b42a3a8a4170db47d464da2845d476f0b3be2606a52303d0.json151 B3fd37b7dca6f37f4b42a3a8a4170db47d464da2845d476f0b3be2606a52303d0
attachments/433a6877aa6a378bbd4b7e2752e484d8cb7499c562d68222be231f62ff2fa420.json2,4 KB433a6877aa6a378bbd4b7e2752e484d8cb7499c562d68222be231f62ff2fa420
attachments/52ab03d8ec104abb85572294bf8a469d75c5de9e74d6042caabff8017adc572b.json64,0 KB52ab03d8ec104abb85572294bf8a469d75c5de9e74d6042caabff8017adc572b
attachments/532916d9dbf463bff57e4ce7dc7483a21658bbbcfc35850072a24b2077c0a1e8.json923 B532916d9dbf463bff57e4ce7dc7483a21658bbbcfc35850072a24b2077c0a1e8
attachments/72c9ceba28e619e4925d75836ec7fd19681f143df3f9178f6bf9d6aecc4dd612.json233 B72c9ceba28e619e4925d75836ec7fd19681f143df3f9178f6bf9d6aecc4dd612
attachments/c1f32b32f40f6b3f93158632162f9284b63c06da6fd563e1bfbd1c0ad260a18e.json5,2 KBc1f32b32f40f6b3f93158632162f9284b63c06da6fd563e1bfbd1c0ad260a18e
attachments/c4729c4d537b4d9f4962efc450ca74037e7f184225710e489e0236867c446397.json1,0 KBc4729c4d537b4d9f4962efc450ca74037e7f184225710e489e0236867c446397
native/causcibench_real/causcibench_real.q084.e1.a1.json1,7 KB9afdf6c4e362e367e7a357f7a4063cb05c7db2df3bbfcebf835bd57c311122bc
sources/causcibench_real.gold.q084.json1,2 KB57ba0457d19e24785a170a87cc411e56f2282c83214685310e29f0f9a06fa29a
specs/causcibench_real.q084.json7,0 KBf35ebe268993cca25df31c83adcaf6e736230c847fa443fa917f9dfc7614aef2
traces/causcibench_real.q084.e1.a1.jsonl16,0 KB60c0fc358f7ec9245e41798ed9384fb7b32c4a9efd8f00fb79b6ca37ea3a5d9d
trajectories/causcibench_real.q084.e1.a1.jsonl11,7 KB40165cda75d01b026a0eb9315a4676bb7ab184cfeeccb66e505cac506ded1774