{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number traced to a tool call or the question","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":true,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"CRB estimate_within_solution_ci: the answer figure inside the gold 95% interval effect \u00b1 1.96\u00b7|SE| of source crb_papers.q008.gold (closed, evaluate_result.py; no --auto-rescale)","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":[-0.9180917125688675,1.4646413269321341],"weight":1.0,"where":null},{"abs_tol":6.078400610971943e-07,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold effect to rel 1e-6 (abs floor 1e-6\u00b7SE); never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q008.gold","table":null,"tool":null,"value":0.27327480718163333,"weight":1.0,"where":null},{"abs_tol":6.078400610971943e-07,"bind":"answer_call","column":null,"failure_class":null,"field":"se","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold SE to rel 1e-6; never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q008.gold","table":null,"tool":null,"value":0.6078400610971942,"weight":1.0,"where":null}],"datasets":{"ds_0005":{"columns":null,"description":"| Variable Name | Description |\n| --- | --- |\n| year | Calendar year of observation (1995\u20132012). Each row is a country\u2013year. |\n| percenthealth | Share of total general government expenditure allocated to public health (%). Drawn from the World Bank\u2019s World Development Indicators (sectoral spending as a fraction of total government spending). |\n| country | Country name corresponding to the national polity whose lower or single parliamentary chamber is referenced for legislative variables. |\n| ever_high_adopt_treat | Indicator equal to 1 if the country ever experienced a large first-year jump in women\u2019s parliamentary seat share associated with the first election using a quota (above the sample mean \u201cshock\u201d) during 1995\u20132012; 0 otherwise. Time-invariant within a country. |\n| time_trend | Numeric time index equal to the calendar year, included to capture linear secular change. |\n| time_trend_2 | Square of the numeric time index (year^2), allowing flexible nonlinear trends over time. |\n| demlag | Democracy indicator lagged one year: 1 if the prior year\u2019s Polity IV combined score was \u2265 6; 0 otherwise. |\n| dem_cumlag | Cumulative years of democracy since 1945 up through the previous year (count based on Polity IV threshold \u2265 6). |\n| autoclag | Autocracy indicator lagged one year: 1 if the prior year\u2019s Polity IV combined score was \u2264 \u22126; 0 otherwise. |\n| leftlag | Indicator lagged one year for whether the governing/prime minister\u2019s party was coded as left-of-center in the Database of Political Institutions. |\n| left_cumlag | Cumulative years of left-leaning executive rule since 1945 up through the previous year (based on DPI coding). |\n| lnODAlag | Natural log of official development assistance received as a percentage of GDP, lagged one year (World Bank). |\n| lnOillag | Natural log of oil rents as a percentage of GDP, lagged one year (World Bank). Oil rents reflect the value of crude oil production minus costs as a share of the economy. |\n| lnTradelag | Natural log of total trade as a percentage of GDP, lagged one year (World Bank). Total trade is exports plus imports of goods and services. |\n| lnFDIlag | Natural log of foreign direct investment, net inflows, as a percentage of GDP, lagged one year (World Bank). |\n| increaselag | Percentage-point increase in women\u2019s share of seats in the lower/single parliamentary chamber at the first election conducted under a gender quota (\u201cquota shock\u201d), lagged one year. Calculated as the difference between the last pre-quota election and the first quota-based election; coded 0 where no implementation occurred in the period. |\n| adopt_quotalag | Indicator for the presence of a parliamentary gender quota from the year of adoption onward, lagged one year. Adoption covers reserved-seat provisions, candidate-list gender requirements, and voluntary party quotas that collectively cover at least ~30% of lower-house seats; compiled from the International IDEA Quota Project and harmonized scholarly sources. |\n\nDataset Context:\nThis dataset is a country\u2013year panel spanning 1995\u20132012, a period marked by widespread introduction and use of electoral gender quota policies across regions. Fiscal variables come from the World Bank\u2019s World Development Indicators, which report sectoral expenditures as shares of total general government spending. The health-spending series used here captures the proportion of national budgets allocated to public health, including both recurrent and capital outlays, enabling consistent comparison across countries and years.\n\nInformation on quota policies\u2014adoption dates, first election under a quota, and the magnitude of changes in women\u2019s legislative representation\u2014was compiled from public administrative and research sources, notably the International IDEA Quota Project, and cross-checked with scholarly datasets and case documentation. Adoption covers three institutional designs: reserved seats for women, legal candidate-list requirements, and voluntary party quotas that meet a threshold of parliamentary seat coverage. In many systems, there is a lag between legal adoption and first use in an election; some adopters did not functionally implement quotas within the 1995\u20132012 window.\n\nLegislative composition is measured for the lower or single parliamentary chamber. The dataset includes a derived \u201cquota shock\u201d measure that records the first percentage-point jump in women\u2019s seat share at the initial quota-based election; this value reflects the immediate numerical change produced by the reform in each country. To align political and economic covariates with subsequent budgeting, several variables are lagged one year. Regime characteristics (democracy/autocracy status and cumulative democratic experience) are drawn from Polity IV. Governing party ideology is coded using the Database of Political Institutions. Macroeconomic indicators\u2014official development assistance, oil rents, trade openness, and foreign direct investment\u2014come from the World Bank and are expressed as percentages of GDP with natural-log transformations. The panel also includes linear and quadratic time indices to summarize long-run temporal patterns.\n\nCoverage emphasizes countries with available fiscal and political data in the post-1995 period. Some early quota adopters prior to 1995 are outside scope to maintain consistent timing and measurement. As with most cross-national administrative series, occasional missingness may reflect gaps in national reporting or differences in when reforms occurred relative to fiscal years. Overall, the file provides a harmonized basis for examining how legislative reforms and macro-political contexts coincide with evolving public budget compositions worldwide.","file_sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","transforms":[],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0005/data/data.csv"},"ref_a_pre":{"columns":null,"description":"year == 1997 and adopt_quotalag == 0","file_sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","transforms":[{"args":{"columns":["percenthealth","year","country"]},"op":"dropna"},{"args":{"query":"year == 1997 and adopt_quotalag == 0"},"op":"filter"},{"args":{"aggs":[{"column":"adopt_quotalag","fn":"count","name":"a_pre"}],"by":["country"]},"op":"collapse"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0005/data/data.csv"},"ref_c_post":{"columns":null,"description":"year == 1998 and adopt_quotalag == 0","file_sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","transforms":[{"args":{"columns":["percenthealth","year","country"]},"op":"dropna"},{"args":{"query":"year == 1998 and adopt_quotalag == 0"},"op":"filter"},{"args":{"aggs":[{"column":"adopt_quotalag","fn":"count","name":"c_post"}],"by":["country"]},"op":"collapse"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0005/data/data.csv"},"ref_never":{"columns":null,"description":"Each country's max adopt_quotalag; never = 1 when it is 0 or missing","file_sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","transforms":[{"args":{"columns":["percenthealth","year","country"]},"op":"dropna"},{"args":{"aggs":[{"column":"adopt_quotalag","fn":"max","name":"amax"}],"by":["country"]},"op":"collapse"},{"args":{"args":{"column":"amax","default":1,"keep_unmapped":false,"mapping":[{"source":1,"target":0}]},"name":"never","op":"map"},"op":"derive"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0005/data/data.csv"},"ref_panel":{"columns":null,"description":"Rows of 1997 and 1998: countries adopting the quota (1997 at 0, 1998 at 1) and never-adopters at 0 in both years","file_sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","transforms":[{"args":{"columns":["percenthealth","year","country"]},"op":"dropna"},{"args":{"query":"year in [1997, 1998]"},"op":"filter"},{"args":{"how":"left","on":["country"],"with":"ref_a_pre"},"op":"merge"},{"args":{"how":"left","on":["country"],"with":"ref_t_post"},"op":"merge"},{"args":{"how":"left","on":["country"],"with":"ref_c_post"},"op":"merge"},{"args":{"how":"left","on":["country"],"with":"ref_never"},"op":"merge"},{"args":{"query":"a_pre == a_pre and (t_post == t_post or (c_post == c_post and never == 1))"},"op":"filter"},{"args":{"args":{"cmp":"ge","column":"t_post","value":1},"name":"treat","op":"indicator"},"op":"derive"},{"args":{"args":{"column":"treat","default":0,"keep_unmapped":false,"mapping":[{"source":1,"target":1}]},"name":"treat","op":"map"},"op":"derive"},{"args":{"args":{"cmp":"eq","column":"year","value":1998},"name":"post","op":"indicator"},"op":"derive"},{"args":{"args":{"columns":["treat","post"]},"name":"treat_post","op":"interaction"},"op":"derive"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0005/data/data.csv"},"ref_t_post":{"columns":null,"description":"year == 1998 and adopt_quotalag == 1","file_sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"1cbd3916e41bbdaefe88cecac281fdfc042a1d9424b7a7c5db2963e596c199ee","transforms":[{"args":{"columns":["percenthealth","year","country"]},"op":"dropna"},{"args":{"query":"year == 1998 and adopt_quotalag == 1"},"op":"filter"},{"args":{"aggs":[{"column":"adopt_quotalag","fn":"count","name":"t_post"}],"by":["country"]},"op":"collapse"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0005/data/data.csv"}},"difficulty":"M","id":"crb_papers.q008","level":"T2","meta":{"agent_reach":{"blocked_by":["reference_frame:ref_a_pre","reference_frame:ref_c_post","reference_frame:ref_never","reference_frame:ref_panel","reference_frame:ref_t_post"],"dtypes":{"ds_0005":{"adopt_quotalag":"float64","percenthealth":"float64"}},"named":{"outcome":["percenthealth"],"treatment":["adopt_quotalag"]},"reachable":false,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":null,"unit_scale":1.0},"cluster":"ds_0005","crb":{"data_path":"data/ds_0005/data/data.csv","dataset_id":"ds_0005","gold":{"ci95":[-0.9180917125688675,1.4646413269321341],"ci_rule":"effect \u00b1 1.96\u00b7|standard_error| (evaluate_result.ci_bounds)","effect":0.27327480718163333,"standard_error":0.6078400610971942},"identification":{"bad_controls":["increaselag","ever_high_adopt_treat"],"causal_quantity":"ATT","controls":["demlag","dem_cumlag","autoclag","leftlag","left_cumlag","lnODAlag","lnOillag","lnTradelag"],"group_variable":"country","minimal_controlling_set":["leftlag","demlag","lnODAlag","lnOillag","lnTradelag"],"outcomes":["percenthealth"],"reason_for_bad_controls":"Both variables are post-treatment/mechanically treatment-determined. increaselag is the size of the initial quota-induced jump in women\u2019s representation (a mediator on the pathway from quota adoption to spending). ever_high_adopt_treat classifies countries based on whether they ever experience a high quota shock, which is defined using post-adoption outcomes. Adjusting for either would induce over-control or collider bias. Guidance: \u201cThe general message\u2026 is that explicitly adjusting for post-treatment variables is almost always a bad idea.\u201d and \u201cover-control bias, by controlling for a mediator.\u201d (Applied Causal Inference Powered by ML and AI, Sec. 11.3, \u2018Post-Treatment Variables\u2019, and Fig. 11.9a).","reason_for_minimal_controlling_set":"The paper motivates these as time-varying confounders affecting both quota adoption and spending. On political ideology: \u201cThis is a particularly relevant potential confounder, as leftist rule may be a crucial determinant of both gender quota adoption (particularly for voluntary party quotas) and patterns of social spending.\u201d (Control variables, p. 923). On regime characteristics: \u201cwe include\u2026 democracy\u2026 and autocracy\u2026 [and] an index of cumulative democracy\u2026 variables\u2026 that may affect both government spending choices and gender quota adoption.\u201d (Control variables, p. 922\u2013923). On global integration and resources: \u201cWe also include\u2026 official development assistance (ODA), oil rents, total trade\u2026 and foreign direct investment (FDI)\u2026 These variables also indicate the extent to which states are embedded within global networks, likely making them more receptive to\u2026 adopt gender quotas\u2026 [and] we consider them important potential confounders of our relationship of interest.\u201d (Control variables, p. 923). Given DiD with country and year effects, a minimal sufficient set among the provided controls keeps leftlag (ideology), demlag (regime), and key macro channels (lnODAlag, lnOillag, lnTradelag) to block the main backdoor paths the authors highlight, while avoiding redundant cumulative counts.","strategy":"Difference-in-Differences","time_variable":"year/time_trend/time_trend_2","treatments":["adopt_quotalag"]},"identification_strategy":"Difference-in-Differences"},"headline_keys":{"infer.ols":["coef.treat_post"]},"naive":{"dataset":"ds_0005","treatment":"adopt_quotalag","y":"percenthealth"},"native_id":"8","reference":{"cite":1,"note":"Reference path transcribed from the item's own estimation script.","steps":[{"args":{"cluster":"country","controls":["treat","post","demlag","dem_cumlag","autoclag","leftlag","left_cumlag","lnODAlag","lnOillag","lnTradelag"],"dataset":"ref_panel","x":["treat_post"],"y":"percenthealth"},"tool":"infer.ols","why":"OLS of patsy 'percenthealth ~ treat + post + treat:post + controls', cluster by country"}]},"reference_path":{"calls":[{"args":{"cluster":"country","controls":["treat","post","demlag","dem_cumlag","autoclag","leftlag","left_cumlag","lnODAlag","lnOillag","lnTradelag"],"dataset":"ref_panel","x":["treat_post"],"y":"percenthealth"},"tool":"infer.ols"}],"needs":[],"provenance":"benchmark_script","quotes":{},"reference_datasets":["ref_a_pre","ref_c_post","ref_never","ref_panel","ref_t_post"],"source_id":"crb_papers.q008.script"}},"question":"Among countries that adopted a parliamentary gender quota in 1997, how did their public-health spending share change from 1997 to 1998 relative to countries that never adopted a quota?","requires":[],"runner":null,"sources":["crb_papers.q008.gold","crb_papers.q008.script"],"split":"public","suite":"crb_papers","tags":["econ-bench","external","crb","did","ref_benchmark_script"],"target":"agent","tool":null,"truth":"CausalReasoningBenchmark gold at the pinned revision: causal_queries.json effect and standard_error (the value the item's estimation.py prints), scored as evaluate_result.py scores estimate_within_solution_ci"}