{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number in the answer was stated by a tool call","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"CauSciBench effect accuracy: |answer figure \u2212 gold| \u2264 5% of |gold|","open_interval":false,"rel_tol":0.05,"required":true,"source_id":"causcibench_real.gold.q118","table":null,"tool":null,"value":0.148,"weight":1.0,"where":null}],"datasets":{"garcia_colored_perceptions":{"columns":null,"description":"The present study is based on an original survey experiment that tests whether assessments of others\u2019 skin color are affected by a subtle racial cue, a name. Random assignment achieved a fairly uniform distribution of subjects across conditions, with 2,783 unique ratings of faces assigned Hispanic names and 2,817 unique ratings of faces assigned non-Hispanic names. Because each subject rated pictures with both Hispanic and non-Hispanic names, subject characteristics were necessarily balanced across experimental conditions.\n\nVariables: \n- id: Respondent\u2019s unique ID number\n- imageno: Unique image number, from the FEI face database\n- hispanicimage: Was the image presented with a distinctively Hispanic name or a distinctively non-Hispanic name? (1 = Distinctively Hispanic name, NOT INCLUDED IN DATASET)\n- femaleimage: Does the person in the image present as male or female? (1 = Female, 0 = Male)\n- hispanicimagexfemaleimage: Interaction between hispanicimage and femaleimage\n- namepair: To which pair of names was the image attached? (\"\"a\"\" = Catalina/Catherine, \"\"b\"\" = Juana/Joann, \"\"c\"\" = Margarita/Margaret. \"\"d\"\" = Carlota/Charlotte, \"\"e\"\" = Daniela/Daniella, \"\"f\"\" = Juan/John, \"\"g\"\" = Pedro/Peter, \"\"h\"\" = Ricardo/Richard, \"\"i\"\" = Pablo/Paul, \"\"j\"\" = Jose/Joe)\n- color: Skin color rating of image (1 = Lightest; 11 = Darkest)\n- gender: Respondent's gender (1 = Male; 2 = Female)\n- age: Respondent's age\n- hispanic: Respondent's identification as Hispanic/Latino(a) (1 = Yes; 2 = No)\n- race: Respondent's racial identification (1 = White, 2 = Black/African-American; 3 = American Indian or Alaska Native; 4 = Asian or Pacific Islander; 5 = Some other race)\n- educ: Respondent's educational attainment (1 = Less than high school; 2 = High school/GED; 3 = Some college; 4 = 2-year college degree (Associate\u2019s); 5 = 4-year college degree (BA, BS); 6 = Graduate or professional degree)\n- hhinc: Respondent's household inome from all sources (1 = Less than $10,000; 2 = $10,000- $19,999; 3 = $20,000-$29,999; 4 = $30,000-$39,999; 5 = $40,000-$49,999; 6 = $50,000-$59,999; 7 = $60,000- $69,999; 8 = $70,000-$79,999; 9 = $80,000-$89,999; 10 = $90,000-$99,999; 11 = $100,000-$149,999; 12 = More than $150,000; 13 = Rather not say)\n- born: Respondent's country of birth\n- region: Respondent's US Census region of residence (1 = Northeast; 2 = Midwest; 3 = South; 4 = West; 5 = Rather not say)\n- rs_color: Respondent's self-rated skin color (1 = Lightest; 11 = Darkest)","file_sha256":"c9d403a8c02120d4b354d4b751a93273ecdfa2b55563238a0fa253796951e0e5","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"CC0 1.0 (CauSciBench data/README.md); original source https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/QTTA73","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"c9d403a8c02120d4b354d4b751a93273ecdfa2b55563238a0fa253796951e0e5","transforms":[],"url":"https://huggingface.co/datasets/causal-nlp/causcibench/resolve/d8aca99042e0bf1d73120a9dc1b65d4079fa1c3a/csv_files/realdata/garcia_colored_perceptions.csv"},"garcia_colored_perceptions_ref":{"columns":null,"description":"Reference frame of causcibench_metadata_rule/1: the CSV with the columns the metadata names derived.","file_sha256":"c9d403a8c02120d4b354d4b751a93273ecdfa2b55563238a0fa253796951e0e5","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"CC0 1.0 (CauSciBench data/README.md); original source https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/QTTA73","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"c9d403a8c02120d4b354d4b751a93273ecdfa2b55563238a0fa253796951e0e5","transforms":[{"args":{"args":{"columns":["femaleimage","hispanicname"]},"name":"femaleimage_x_hispanicname","op":"interaction"},"op":"derive"}],"url":"https://huggingface.co/datasets/causal-nlp/causcibench/resolve/d8aca99042e0bf1d73120a9dc1b65d4079fa1c3a/csv_files/realdata/garcia_colored_perceptions.csv"}},"difficulty":"M","id":"causcibench_real.q118","level":"T2","meta":{"agent_reach":{"blocked_by":["reference_frame:garcia_colored_perceptions_ref"],"dtypes":{"garcia_colored_perceptions":{"color":"int64","femaleimage":"int64","hispanicname":"int64"}},"named":{"outcome":["color"],"treatment":["femaleimage","hispanicname"]},"reachable":false,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":null,"unit_scale":1.0},"cluster":"paper:Colored perceptions: Racially distinctive names and assessments of skin color","gold":{"effect":0.148,"effect_as_stored":"0.148","is_significant":1,"source_id":"causcibench_real.gold.q118","std_error":0.072,"std_error_as_stored":"0.072"},"headline_keys":{"infer.ols":["coef.femaleimage_x_hispanicname"]},"metadata":{"control_variables":"hispanicname,femaleimage,rs_color,age,gender,race,hhinc,educ,region,hispanic,born","dataset_path":"data/csv_files/realdata/garcia_colored_perceptions.csv","effect":0.148,"id":118,"instrument_var":null,"interaction_var":null,"is_rct":true,"is_significant":1,"method":"ols","name":"Colored perceptions: Racially distinctive names and assessments of skin color","outcome_var":"color","running_var":null,"state_var":null,"std_error":0.072,"temporal_var":null,"treatment_var":"femaleimage:hispanicname"},"naive":null,"name":"Colored perceptions: Racially distinctive names and assessments of skin color","native_id":118,"outcome":"color","pin":{"data_repo":"causal-nlp/causcibench","data_revision":"d8aca99042e0bf1d73120a9dc1b65d4079fa1c3a","repo":"causalNLP/CauSciBench","revision":"d9bb43a2d90f261978cc5ac0b628c0484f6e8956"},"precision_limited":false,"reference":{"cite":1,"steps":[{"args":{"controls":["femaleimage","hispanicname","rs_color","age","gender","race","hhinc","educ","region","hispanic","born"],"dataset":"garcia_colored_perceptions_ref","x":["femaleimage_x_hispanicname"],"y":"color"},"tool":"infer.ols","why":"causcibench_metadata_rule/1: ols"}]},"reference_notes":["treatment femaleimage:hispanicname derived as femaleimage_x_hispanicname = femaleimage\u00b7hispanicname, with femaleimage and hispanicname entered as controls"],"reference_path":{"calls":[{"args":{"controls":["femaleimage","hispanicname","rs_color","age","gender","race","hhinc","educ","region","hispanic","born"],"dataset":"garcia_colored_perceptions_ref","x":["femaleimage_x_hispanicname"],"y":"color"},"tool":"infer.ols"}],"needs":[],"provenance":"metadata_rule","quotes":{},"reference_datasets":["garcia_colored_perceptions_ref"],"source_id":null},"reference_rule":"causcibench_metadata_rule/1","source":"causcibench","treatment":"femaleimage:hispanicname"},"question":"Do people consider that the skin color of hispanic people is darker when the person presented is a woman?","requires":[],"runner":null,"sources":["causcibench_real.gold.q118"],"split":"public","suite":"causcibench_real","tags":["econ-bench","external","causcibench","method:ols"],"target":"agent","tool":null,"truth":"CauSciBench real_input.json @ d9bb43a, id 118: effect 0.148"}