Tarefa causcibench_real.q118
causcibench_realagentT2publicépoca 1, tentativa 1Execução: 20261015T171059Z-suite-causcibench_real-scripted-ded01dVeredito
Família do método (das chamadas de ferramentas): did (OLS on the interaction of a group indicator and a period indicator (2x2 difference in differences))
Pergunta feita ao analista
Do people consider that the skin color of hispanic people is darker when the person presented is a woman?
Estimativa (infer.ols): coef.femaleimage_x_hispanicname = 0,1441.
Valores esperados e de onde vêm
causcibench_real.gold.q118
- Fonte
- CauSciBench real split, data/metadata_json/real_input.json at d9bb43a, id 118
- Tipo
- benchmark_gold
- Localização
- real_input.json, record id 118: fields effect and std_error
- Trecho
id 118 effect 0.148 std_error 0.072
- Valor como publicado
0.148- Erro-padrão como publicado
0.072- Valor
- 0,148
0.148(se 0,0720.072) - Conferido por
- arquivo do benchmark · Written by scripts/freeze_causcibench_real.py from the pinned file; equal to the `answer` and `std_error` of the same id in data/metadata_csv/real_input.csv at the pin.
- sha256 da cópia lida
c1f4c8930675afff63d816d8ed066941ae92697e7bfc31e9d38519398dd80438(identifica a cópia lida, não um arquivo canônico)- Endereço
https://raw.githubusercontent.com/causalNLP/CauSciBench/d9bb43a2d90f261978cc5ac0b628c0484f6e8956/data/metadata_json/real_input.json- Licença
- CauSciBench repository (MIT for code; metadata as published in the repository)
- #1 figura da resposta próxima: 0,148
0.148· relativa 0.05
Verificações
| # | Verificação | Obrigatória | Resultado | Esperado | Observado | Tolerância | Fonte | Chamada |
|---|---|---|---|---|---|---|---|---|
| 0 | resposta com lastro every number backed | obrigatória | passou | — | [] | — | — | — |
| 1 | figura da resposta próxima 0.1441008369 vs 0.148 | obrigatória | passou | 0,1480.148 | 0,1441010.14410083690549091 | relativa 0.05 | causcibench_real.gold.q118 | call_c088d088 |
Caminho de referência e caminho do agente
A chamada de referência ao lado da chamada que deu a resposta, argumento por argumento.
caminho de referência do item · metadata_rule
| Argumento | Referência | Agente | |
|---|---|---|---|
| cluster | — | — | não fixado pela referência |
| controls | ["femaleimage","hispanicname","rs_color","age","gender","race","hhinc","educ","region","hispanic","born"] | ["femaleimage","hispanicname","rs_color","age","gender","race","hhinc","educ","region","hispanic","born"] | igual |
| dataset | "garcia_colored_perceptions_ref" | "garcia_colored_perceptions_ref" | igual |
| fe | — | [] | não fixado pela referência |
| intercept | — | true | não fixado pela referência |
| level | — | 0,950.95 | não fixado pela referência |
| se | — | "HC1" | não fixado pela referência |
| subset | — | — | não fixado pela referência |
| weights | — | — | não fixado pela referência |
| x | ["femaleimage_x_hispanicname"] | ["femaleimage_x_hispanicname"] | igual |
| y | "color" | "color" | igual |
Trajetória
Cada evento na ordem em que aconteceu: turnos do modelo, chamadas de ferramentas, guardas, transformações e pontuação.
- 0Início
Do people consider that the skin color of hispanic people is darker when the person presented is a woman?
garcia_colored_perceptions(agent)The present study is based on an original survey experiment that tests whether assessments of others’ skin color are affected by a subtle racial cue, a name. Random assignment achieved a fairly uniform distribution of subjects across conditions, with 2,783 unique ratings of faces assigned Hispanic names and 2,817 unique ratings of faces assigned non-Hispanic names. Because each subject rated pictures with both Hispanic and non-Hispanic names, subject characteristics were necessarily balanced across experimental conditions. Variables: - id: Respondent’s unique ID number - imageno: Unique image number, from the FEI face database - hispanicimage: Was the image presented with a distinctively Hispanic name or a distinctively non-Hispanic name? (1 = Distinctively Hispanic name, NOT INCLUDED IN DATASET) - femaleimage: Does the person in the image present as male or female? (1 = Female, 0 = Male) - hispanicimagexfemaleimage: Interaction between hispanicimage and femaleimage - namepair: To which pair of names was the image attached? (""a"" = Catalina/Catherine, ""b"" = Juana/Joann, ""c"" = Margarita/Margaret. ""d"" = Carlota/Charlotte, ""e"" = Daniela/Daniella, ""f"" = Juan/John, ""g"" = Pedro/Peter, ""h"" = Ricardo/Richard, ""i"" = Pablo/Paul, ""j"" = Jose/Joe) - color: Skin color rating of image (1 = Lightest; 11 = Darkest) - gender: Respondent's gender (1 = Male; 2 = Female) - age: Respondent's age - hispanic: Respondent's identification as Hispanic/Latino(a) (1 = Yes; 2 = No) - race: Respondent's racial identification (1 = White, 2 = Black/African-American; 3 = American Indian or Alaska Native; 4 = Asian or Pacific Islander; 5 = Some other race) - educ: Respondent's educational attainment (1 = Less than high school; 2 = High school/GED; 3 = Some college; 4 = 2-year college degree (Associate’s); 5 = 4-year college degree (BA, BS); 6 = Graduate or professional degree) - hhinc: Respondent's household inome from all sources (1 = Less than $10,000; 2 = $10,000- $19,999; 3 = $20,000-$29,999; 4 = $30,000-$39,999; 5 = $40,000-$49,999; 6 = $50,000-$59,999; 7 = $60,000- $69,999; 8 = $70,000-$79,999; 9 = $80,000-$89,999; 10 = $90,000-$99,999; 11 = $100,000-$149,999; 12 = More than $150,000; 13 = Rather not say) - born: Respondent's country of birth - region: Respondent's US Census region of residence (1 = Northeast; 2 = Midwest; 3 = South; 4 = West; 5 = Rather not say) - rs_color: Respondent's self-rated skin color (1 = Lightest; 11 = Darkest)
filec9d403a8c02120d4b354d4b751a93273ecdfa2b55563238a0fa253796951e0e5· frame —garcia_colored_perceptions_ref(reference)filec9d403a8c02120d4b354d4b751a93273ecdfa2b55563238a0fa253796951e0e5· frame —derive{"args":{"columns":["femaleimage","hispanicname"]},"name":"femaleimage_x_hispanicname","op":"interaction"}
- 1Transformação
garcia_colored_perceptions_ref·derive{"args":{"columns":["femaleimage","hispanicname"]},"name":"femaleimage_x_hispanicname","op":"interaction"}5600 → 5600 linhasFrame antes → depois:7e9b99044805→bfede1383591 - 2Turno do modelo
call_76bd3861referenceFim:tool_use0 ms0 de entrada · 0 de saída · 0 de raciocínio · 0 em cacheTexto visívelcauscibench_metadata_rule/1: ols
Chamadas pedidasinfer.ols{ "controls": [ "femaleimage", "hispanicname", "rs_color", "age", "gender", "race", "hhinc", "educ", "region", "hispanic", "born" ], "dataset": "garcia_colored_perceptions_ref", "x": [ "femaleimage_x_hispanicname" ], "y": "color" }
- 3Ferramenta
infer.olsv0.1.0call_c088d088decidido pelo modeloreexecutávelMotivo: causcibench_metadata_rule/1: ols
Argumentos
{ "cluster": null, "controls": [ "femaleimage", "hispanicname", "rs_color", "age", "gender", "race", "hhinc", "educ", "region", "hispanic", "born" ], "dataset": "garcia_colored_perceptions_ref", "fe": [], "intercept": true, "level": 0.95, "se": "HC1", "subset": null, "weights": null, "x": [ "femaleimage_x_hispanicname" ], "y": "color" }Figuras call_c088d088 Chave Valor Valor bruto Erro-padrão IC coef.femaleimage_x_hispanicname Coefficient on femaleimage_x_hispanicname in the regression of color0,144101 0.144100836905490910,0773563 0.07735629741777642-0,00754757 – 0,295749 Diagnósticosn_obs= 5.6005600k= 1313r2= 0,09645180.09645179541549964df_resid= 5.5875587se_type="HC1"n_clusters= —
Tabelas (1)
{ "coefficients": [ { "ci_high": 3.893335914630548, "ci_low": 3.079573341289158, "coef": 3.486454627959853, "p": 8.121749696210072e-62, "se": 0.2075513344427406, "t": 16.798035229793708, "term": "Intercept" }, { "ci_high": 0.2957492467311051, "ci_low": -0.007547572920123258, "coef": 0.14410083690549091, "p": 0.06254007584372823, "se": 0.07735629741777642, "t": 1.862819727878763, "term": "femaleimage_x_hispanicname" }, { "ci_high": -0.783797733598111, "ci_low": -0.9973616866917441, "coef": -0.8905797101449275, "p": 9.879749821023862e-59, "se": 0.05446979856968054, "t": -16.349972526622302, "term": "femaleimage" }, { "ci_high": 0.1063900155462285, "ci_low": -0.12433651175972198, "coef": -0.00897324810674674, "p": 0.8788107006824506, "se": 0.05884713822246482, "t": -0.15248401838717135, "term": "hispanicname" }, { "ci_high": 0.08869124178567286, "ci_low": 0.03888933840139164, "coef": 0.06379029009353225, "p": 5.271188519987459e-7, "se": 0.01270204829248107, "t": 5.022047517430136, "term": "rs_color" }, { "ci_high": -0.0418339356860952, "ci_low": -0.08181245719954992, "coef": -0.06182319644282256, "p": 1.4224221348636577e-9, "se": 0.010196580379820847, "t": -6.06313039665449, "term": "age" }, { "ci_high": 0.009624683045120153, "ci_low": -0.14548425560922193, "coef": -0.06792978628205089, "p": 0.08601631211136644, "se": 0.03956076164761172, "t": -1.717100062105397, "term": "gender" }, { "ci_high": -0.09173187426981519, "ci_low": -0.16470772498963474, "coef": -0.12821979962972496, "p": 6.242500362000085e-12, "se": 0.018612597451859735, "t": -6.888871903094508, "term": "race" }, { "ci_high": 0.048125613155083735, "ci_low": 0.024920539574846926, "coef": 0.03652307636496533, "p": 7.262452690487826e-10, "se": 0.005918487953610543, "t": 6.171014734039395, "term": "hhinc" }, { "ci_high": 0.006865986706554625, "ci_low": -0.05850540626183788, "coef": -0.025819709777641628, "p": 0.12153750940257342, "se": 0.016673069380985936, "t": -1.5485876767889284, "term": "educ" }, { "ci_high": 0.07848847444735611, "ci_low": 0.006904606150421114, "coef": 0.04269654029888861, "p": 0.019392965228341354, "se": 0.01825757029915567, "t": 2.338566391874341, "term": "region" }, { "ci_high": 0.2894260742144333, "ci_low": -0.02207784068441182, "coef": 0.13367411676501073, "p": 0.09252710997993319, "se": 0.07944952906339908, "t": 1.6825035760544478, "term": "hispanic" }, { "ci_high": 0.5465991635965926, "ci_low": 0.18093311294535713, "coef": 0.36376613827097487, "p": 0.00009715859873659921, "se": 0.09326366099812188, "t": 3.900405949947647, "term": "born" } ] }- Digests dos dados
garcia_colored_perceptions_refbfede1383591da85f0213d702c5c1eaecbd34ca11112017984fbb698e29efd18- sha256 do resultado
75c9436b4b1df454855f6cd294f37b4ebc81af438b42b01e44a5ab3a29b85b76
- 4Turno do modelo
call_e04ca68freferenceFim:tool_use0 ms0 de entrada · 0 de saída · 0 de raciocínio · 0 em cacheChamadas pedidassubmit_answer(meta-ferramenta){ "answer": "Estimativa (infer.ols): coef.femaleimage_x_hispanicname = 0,1441.", "caveats": [], "figure_refs": [ "call_c088d088:coef.femaleimage_x_hispanicname" ] }
- 5Guardaresposta aceita
- Números encontrados
- 0,1441
- Sem lastro
- —
- Não citados
- —
1 nova(s) tentativa(s) restante(s) - 6Resposta
Estimativa (infer.ols): coef.femaleimage_x_hispanicname = 0,1441.
Figuras citadas:call_c088d088:coef.femaleimage_x_hispanicname· Parada:answeredFigura da resposta:call_c088d088:coef.femaleimage_x_hispanicname= 0,1441010.14410083690549091(se 0,07735630.07735629741777642) - 7Pontuação#0resposta com lastropassouobrigatória
every number backed - 8Pontuação#1figura da resposta próximapassouobrigatória
0.1441008369 vs 0.148
Épocas × verificações
| # | 0 | 1 |
|---|---|---|
| Época 1 | passou | passou |
Tentativas
Cada tentativa é seu próprio arquivo e é publicada; só a última de cada época é pontuada.
- época 1, tentativa 1passoupontuada(exibindo)
Downloads desta tarefa
fa783abb6f249a9f45e7071b1f799f4db869ae6f5798254ab4128a20eacf5b6dO hash de um zip é só de transporte: o que ancora a integridade é o RUN_SHA256SUMS que vai dentro dele, cópia exata do SHA256SUMS da execução.
| Arquivo | Tamanho | sha256 |
|---|---|---|
| attachments/05df3e7f144874d1bc5a680562fb56fda7544aa28930b3225b5ecf693dad081c.json | 2,2 KB | 05df3e7f144874d1bc5a680562fb56fda7544aa28930b3225b5ecf693dad081c |
| attachments/2c325885ce870097010a4e5f05fecdda9663378d45e96e2dc730b980f5ce4cfd.json | 331 B | 2c325885ce870097010a4e5f05fecdda9663378d45e96e2dc730b980f5ce4cfd |
| attachments/433a6877aa6a378bbd4b7e2752e484d8cb7499c562d68222be231f62ff2fa420.json | 2,4 KB | 433a6877aa6a378bbd4b7e2752e484d8cb7499c562d68222be231f62ff2fa420 |
| attachments/52ab03d8ec104abb85572294bf8a469d75c5de9e74d6042caabff8017adc572b.json | 64,0 KB | 52ab03d8ec104abb85572294bf8a469d75c5de9e74d6042caabff8017adc572b |
| attachments/976cf7ce924c6cfa53bc90a855b31a88efb680dc18590fc45366637f9d499b9e.json | 3,9 KB | 976cf7ce924c6cfa53bc90a855b31a88efb680dc18590fc45366637f9d499b9e |
| attachments/a04138733cba3f8e75f0568834122e31c3727d451d2d8d61e42613f0c015bcf5.json | 130 B | a04138733cba3f8e75f0568834122e31c3727d451d2d8d61e42613f0c015bcf5 |
| attachments/be2daecaa38c177342bb7343a1dac7c5f4101d6fe381d99c26c0da05f1351c79.json | 2,1 KB | be2daecaa38c177342bb7343a1dac7c5f4101d6fe381d99c26c0da05f1351c79 |
| native/causcibench_real/causcibench_real.q118.e1.a1.json | 2,1 KB | 119e06476aea2cced0f0dc2ee8dd5dbbcd16c67a4be7bfb208906ceb053e6c49 |
| sources/causcibench_real.gold.q118.json | 1,2 KB | 77aa710b043bf852b05127786cef16fc5b57888fd4cded0e5318f188957fb2d4 |
| specs/causcibench_real.q118.json | 7,8 KB | e534c01cdcea2b895cd350dfe51fec87caa9c5648e3b4bf78afd02240bb038b4 |
| traces/causcibench_real.q118.e1.a1.jsonl | 18,0 KB | 699ee96bbdde6d43311d0b40c6a0038129268b3adf4ba61f6d5a36d8d5dde4e4 |
| trajectories/causcibench_real.q118.e1.a1.jsonl | 14,2 KB | b1b63ece0790f7d74243fa9f8f77af746593c4bfc94d772a6ceb9ae931277d4f |