{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number of the answer is stated by a tool call","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":true,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"QRData eval.py: strictly inside gold \u00b1 3% (a '%' gold divided by 100), on the answer figure after meta.answer_format","open_interval":true,"rel_tol":null,"required":true,"source_id":"qrdata.gold.367","table":null,"tool":null,"value":[0.3783,0.4017],"weight":1.0,"where":null}],"datasets":{"learning_mindset":{"columns":null,"description":"The National Study of Learning Mindsets is a randomised study conducted in U.S. public high schools which aims at finding the impact of a growth mindset. The way it works is that students receive from the school a seminar to instill in them a growth mindset. Then, they follow up the students in their college years to measure how well they\u2019ve performed academically. This measurement was compiled into an achievement score and standardized. The CSV file learning_mindset.csv contains simulated data of this research.\nVariable Description\nintervention: the intervention of the growth mindset;\nachievement_score: the standardized academic achievement score;\nschoolid: identifier of the student\u2019s school;\nsuccess_expect: self-reported expectations for success in the future, a proxy for prior achievement, measured prior to random assignment;\nethnicity: categorical variable for student race/ethnicity;\ngender: categorical variable for student identified gender;\nfrst_in_family: categorical variable for student first-generation status, i.e. first in family to go to college;\nschool_urbanicity: school-level categorical variable for urbanicity of the school, i.e. rural, suburban, etc;\nschool_mindset: school-level mean of students\u2019 fixed mindsets, reported prior to random assignment, standardized;\nschool_achievement: school achievement level, as measured by test scores and college preparation for the previous 4 cohorts of students, standardized;\nschool_ethnic_minority: school racial/ethnic minority composition, i.e., percentage of student body that is Black, Latino, or Native American, standardized;\nschool_poverty: school poverty concentration, i.e., percentage of students who are from families whose incomes fall below the federal poverty line, standardized;\nschool_size: total number of students in all four grade levels in the school, standardized.","file_sha256":"3c3a97cca6fdd96f6856f7856b5f8bc786a31d1a9be78c9f610ef3070783e970","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"CC BY-NC 4.0","member":"data/learning_mindset.csv","object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"3c3a97cca6fdd96f6856f7856b5f8bc786a31d1a9be78c9f610ef3070783e970","transforms":[],"url":"https://raw.githubusercontent.com/xxxiaol/QRData/de450af45ff7101b328bb064c6b475f73414a7ed/benchmark/data.zip"}},"difficulty":"M","id":"qrdata_socsci.367","level":"T2","meta":{"agent_reach":{"blocked_by":["no_reference_path","no_tool:doubly_robust"],"dtypes":{"learning_mindset":{"achievement_score":"float64","intervention":"int64"}},"named":{"outcome":["achievement_score"],"treatment":["intervention"]},"reachable":false,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":2,"unit_scale":1.0},"answer_format_from":{"figure_transform":null,"round_to":"nearest hundredth","unit_scale":"1.0: an estimator's figure is on the gold's own scale"},"cluster":"learning_mindset.csv","family":"numerical","headline_keys":{},"naive":{"dataset":"learning_mindset","treatment":"intervention","y":"achievement_score"},"native":{"answer":"0.39","keywords":["Causality","Observational data","Matching","Propensity score","Average treatment effect"],"multiple_choices":null,"question_type":"numerical","reference":"Causal Inference for the Brave and True 12"},"native_id":367,"reference":{"cite":1,"note":"Sem caminho de refer\u00eancia com as ferramentas registradas: no_tool:doubly_robust. A resposta n\u00e3o cita estimativa de efeito.","steps":[{"args":{"dataset":"learning_mindset"},"tool":"data.describe","why":"Sem caminho de refer\u00eancia com as ferramentas registradas: no_tool:doubly_robust."}]},"reference_path":{"calls":[],"needs":["no_tool:doubly_robust"],"provenance":"none","quotes":{},"reference_datasets":[],"source_id":null},"reference_why":null},"question":"What is the average treatment effect of the growth mindset on the achievement score? Please conduct doubly robust estimation, use logistic regression to estimate the propensity score, and round the final answer to the nearest hundredth.","requires":[],"runner":null,"sources":["qrdata.gold.367","qrdata.eval_py"],"split":"public","suite":"qrdata_socsci","tags":["econ-bench","external","qrdata","brave_and_true","numerical","no_reference_path","naive"],"target":"agent","tool":null,"truth":"QRData gold: benchmark/QRData.json item 367 at de450af45ff7101b328bb064c6b475f73414a7ed"}