{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number traced to a tool call or the question","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":true,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"CRB estimate_within_solution_ci: the answer figure inside the gold 95% interval effect \u00b1 1.96\u00b7|SE| of source crb_papers.q052.gold (closed, evaluate_result.py; no --auto-rescale)","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":[-0.35914507340621293,-0.13213109517524546],"weight":1.0,"where":null},{"abs_tol":5.791172914055293e-08,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold effect to rel 1e-6 (abs floor 1e-6\u00b7SE); never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q052.gold","table":null,"tool":null,"value":-0.2456380842907292,"weight":1.0,"where":null},{"abs_tol":5.791172914055293e-08,"bind":"answer_call","column":null,"failure_class":null,"field":"se","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold SE to rel 1e-6; never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q052.gold","table":null,"tool":null,"value":0.05791172914055293,"weight":1.0,"where":null}],"datasets":{"ds_0027":{"columns":null,"description":"| Variable Name | Description |\n| --- | --- |\n| year | Calendar year of observation for the district-level fiscal and political records (2001\u20132012). |\n| kode_neil | Harmonized district identifier that tracks administrative units across boundary changes and splits to maintain longitudinal comparability. |\n| personnel_sh | Share of total realized district expenditures allocated to personnel (wages, salaries, allowances); expressed as a fraction between 0 and 1. |\n| capital_sh | Share of total realized district expenditures allocated to capital investment (e.g., infrastructure, equipment); expressed as a fraction between 0 and 1. |\n| ltotalexp_all_pc | Natural log of total realized district government expenditures per capita (rupiah per resident). |\n| lcapital_exp_pc | Natural log of realized district capital expenditures per capita (rupiah per resident). |\n| incumbency | Indicator for whether the sitting district head is a candidate in that year (1=yes, 0=no); relevant in years when district head elections are held. |\n| services_provision | Standardized index summarizing local service delivery (sanitation access, clean water access, school enrollment, skilled birth attendance, and road quality); higher values indicate better provision. |\n| lgdppc_l | Natural log of district GDP per capita from official statistics, lagged one year. |\n| lpop_l | Natural log of the district\u2019s population, lagged one year. |\n| rev_total_pc2_l | Total district revenue per capita from all sources (including general transfers and revenue sharing), lagged one year; measured in rupiah per resident. |\n| rev_natural_pc_l | Shared natural resource revenues (intergovernmental revenue sharing tied to extractive resources) per capita, lagged one year; measured in rupiah per resident. |\n| poverty_pc_l | Fraction of the district population below the official poverty line, lagged one year (0\u20131 scale). |\n| gini_l | Gini index of consumption inequality from household survey data, lagged one year (0\u2013100 scale; higher = more unequal). |\n| elected_leader_l | Indicator that the district head position is filled by direct popular election (1) versus selection by the local legislature (0), lagged one year; direct elections were rolled out beginning in 2005. |\n| golkar_share_all | Vote share of the Golkar party in the district\u2019s legislative elections (compiled from 1999 and 2004 results); expressed as a fraction between 0 and 1. |\n| pdip_share_all | Vote share of the Indonesian Democratic Party of Struggle (PDI\u2013P) in the district\u2019s legislative elections (compiled from 1999 and 2004 results); expressed as a fraction between 0 and 1. |\n| enp_all | Effective number of parties in the district legislature (Laakso\u2013Taagepera index); higher values indicate more fragmented party systems. |\n| election_year | Indicator that a district head election is held in the current calendar year (1=yes, 0=no); reflects the local executive election calendar. |\n| election_year_l | Indicator for the year immediately preceding a district head election (lead-up year; 1=yes, 0=no). |\n| election_year_lead1 | Indicator for the year immediately following a district head election (post-election year; 1=yes, 0=no). |\n\nDataset Context:\nThis panel assembles district-by-year information for Indonesia\u2019s post-decentralization era, covering 2001\u20132012. Fiscal figures are realized (executed) budgets from the Ministry of Finance and use the national economic classification that separates outlays into capital investment, goods and services, and personnel. Revenue items distinguish general-purpose transfers and shared natural resource revenues alongside locally generated revenue. Where noted, variables are expressed per capita using contemporaneous district population counts, and several key covariates are provided in natural logs to reduce skew.\n\nPolitical characteristics capture the local partisan environment and institutional arrangements. District-level party vote shares are compiled from the 1999 and 2004 legislative elections to describe the baseline partisan landscape. The effective number of parties quantifies legislative fragmentation. Indonesia shifted from indirect selection of district heads by local legislatures (1999\u20132004) to direct popular elections beginning in 2005. The initial schedule of local executive contests was phased according to pre-existing appointment terms, producing staggered election years across districts. Indicators mark the election year itself as well as the years immediately before and after.\n\nSocioeconomic and service delivery measures come from official statistics agencies and large-scale household surveys (including Susenas). The services index combines access and quality proxies\u2014sanitation, clean water, school enrollment, skilled birth attendance, and road conditions\u2014standardized so that higher values indicate stronger provision. Inequality (Gini) and poverty rates are derived from household consumption data. Variables with the \u201c_l\u201d suffix are lagged one calendar year to reflect prior conditions relative to the current budget year.\n\nThe administrative map of Indonesia evolved substantially during this period due to district proliferation. The dataset harmonizes identifiers to follow units through splits, yielding an unbalanced panel by design. Users should note that national statistical agencies implemented notable survey and weighting changes around 2011; while the panel extends through 2012, comparability for certain survey-based indicators across this redesign may warrant caution. District governments largely finance spending through intergovernmental transfers and shared revenues, as local borrowing was highly restricted during these years. Overall, the dataset provides a consolidated view of local public finance, political structure, and socioeconomic context during Indonesia\u2019s early decentralization and direct local election era.","file_sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","transforms":[],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0027/data/data.csv"},"ref_e04":{"columns":null,"description":"Each district's election_year in 2004, as e04","file_sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","transforms":[{"args":{"columns":["kode_neil","year","election_year","lcapital_exp_pc"]},"op":"select"},{"args":{"query":"year == 2004"},"op":"filter"},{"args":{"map":{"election_year":"e04"}},"op":"rename"},{"args":{"columns":["kode_neil","e04"]},"op":"select"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0027/data/data.csv"},"ref_e05":{"columns":null,"description":"Each district's election_year in 2005, as e05","file_sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","transforms":[{"args":{"columns":["kode_neil","year","election_year","lcapital_exp_pc"]},"op":"select"},{"args":{"query":"year == 2005"},"op":"filter"},{"args":{"map":{"election_year":"e05"}},"op":"rename"},{"args":{"columns":["kode_neil","e05"]},"op":"select"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0027/data/data.csv"},"ref_panel":{"columns":null,"description":"District rows of 2004 and 2005: units not electing in 2004, treated when electing in 2005, observed with the outcome in both years","file_sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","transforms":[{"args":{"columns":["kode_neil","year","election_year","lcapital_exp_pc"]},"op":"select"},{"args":{"query":"year in [2004, 2005]"},"op":"filter"},{"args":{"how":"inner","on":["kode_neil"],"with":"ref_e04"},"op":"merge"},{"args":{"how":"inner","on":["kode_neil"],"with":"ref_e05"},"op":"merge"},{"args":{"query":"e04 == 0 and (e05 == 1 or e05 == 0)"},"op":"filter"},{"args":{"columns":["lcapital_exp_pc"]},"op":"dropna"},{"args":{"how":"inner","on":["kode_neil"],"with":"ref_years"},"op":"merge"},{"args":{"query":"nyears == 2"},"op":"filter"},{"args":{"args":{"cmp":"eq","column":"e05","value":1},"name":"TreatGroup","op":"indicator"},"op":"derive"},{"args":{"args":{"cmp":"eq","column":"year","value":2005},"name":"Post","op":"indicator"},"op":"derive"},{"args":{"args":{"columns":["TreatGroup","Post"]},"name":"DiD","op":"interaction"},"op":"derive"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0027/data/data.csv"},"ref_years":{"columns":null,"description":"How many of 2004 and 2005 each district has a non-missing lcapital_exp_pc in","file_sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"e02ae6b363d47cf0f76a5fc37d88e1f8ffc08c6b9f51eee68d0b5d6b12c2d727","transforms":[{"args":{"columns":["kode_neil","year","election_year","lcapital_exp_pc"]},"op":"select"},{"args":{"query":"year in [2004, 2005]"},"op":"filter"},{"args":{"columns":["lcapital_exp_pc"]},"op":"dropna"},{"args":{"aggs":[{"column":"lcapital_exp_pc","fn":"count","name":"n"}],"by":["kode_neil","year"]},"op":"collapse"},{"args":{"aggs":[{"column":"n","fn":"count","name":"nyears"}],"by":["kode_neil"]},"op":"collapse"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0027/data/data.csv"}},"difficulty":"M","id":"crb_papers.q052","level":"T2","meta":{"agent_reach":{"blocked_by":["reference_frame:ref_e04","reference_frame:ref_e05","reference_frame:ref_panel","reference_frame:ref_years"],"dtypes":{"ds_0027":{"election_year":"float64","lcapital_exp_pc":"float64"}},"named":{"outcome":["lcapital_exp_pc"],"treatment":["election_year"]},"reachable":false,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":null,"unit_scale":1.0},"cluster":"ds_0027","crb":{"data_path":"data/ds_0027/data/data.csv","dataset_id":"ds_0027","gold":{"ci95":[-0.35914507340621293,-0.13213109517524546],"ci_rule":"effect \u00b1 1.96\u00b7|standard_error| (evaluate_result.ci_bounds)","effect":-0.2456380842907292,"standard_error":0.05791172914055293},"identification":{"bad_controls":["incumbency","ltotalexp_all_pc","capital_sh","personnel_sh"],"causal_quantity":"ATT","controls":["elected_leader_l","enp_all","golkar_share_all","pdip_share_all","services_provision","rev_total_pc2_l","rev_natural_pc_l","gini_l","poverty_pc_l","lgdppc_l","lpop_l"],"group_variable":"kode_neil","minimal_controlling_set":["elected_leader_l"],"outcomes":["lcapital_exp_pc"],"reason_for_bad_controls":"Incumbency is part of the mechanism by which election years affect capital spending, so adjusting for it would block part of the treatment effect (a post-treatment mediator). The paper states: \u201cThis effect is mediated by the status of incumbents.\u201d and shows the effect is \u201cdriven by election years in which incumbents are not seeking reelection\u201d (Abstract; pp. 517\u2013518, Table 3 discussion). Total spending and expenditure shares are mechanically affected by elections in the same year; controlling for them would induce over-control. The authors find \u201ca statistically significant drop in overall expenditures in election years. This drop is driven by a reduction in capital expenditures\u201d (pp. 517\u2013518, Table 1). They further note the personnel share rises mechanically: \u201cthe increase in the personnel expenditure share largely reflects a mechanical increase, due to the drop in the overall budget and capital expenditures\u201d (p. 518, Table 1 discussion). Following good/bad controls guidance, mediators and post-treatment outcomes (or their functions) are bad controls.","reason_for_minimal_controlling_set":"Identification primarily relies on the exogenous phasing of election timing plus district and year fixed effects: \u201cCausal identification of the effect is plausible for three reasons. First, the timing of direct elections followed a predetermined, exogenous schedule and is a plausible natural experiment. Second, the inclusion of district and year fixed effects controls for any unobserved time-invariant confounders\u2026 Third, the inclusion of time-varying covariates accounts for any remaining observable confounders.\u201d (Model specification, p. 516\u2013517). Among observables, the authors motivate one key pre-treatment control to guard against persistent differences in accountability regimes: \u201cWe account for any lasting effects of democratic accountability by including a dummy variable for the presence of a directly elected district head.\u201d (p. 516\u2013517). All controls are lagged: \u201cAll control variables were lagged by one year to avoid direct simultaneity.\u201d (p. 517).","strategy":"Difference-in-Differences","time_variable":"year","treatments":["election_year"]},"identification_strategy":"Difference-in-Differences"},"headline_keys":{"infer.ols":["coef.DiD"]},"naive":{"dataset":"ds_0027","treatment":"election_year","y":"lcapital_exp_pc"},"native_id":"52","reference":{"cite":1,"note":"Reference path transcribed from the item's own estimation script.","steps":[{"args":{"cluster":"kode_neil","controls":["TreatGroup","Post"],"dataset":"ref_panel","x":["DiD"],"y":"lcapital_exp_pc"},"tool":"infer.ols","why":"sm.OLS(y, const + TreatGroup + Post + DiD).fit(cov_type='cluster', groups=kode_neil)"}]},"reference_path":{"calls":[{"args":{"cluster":"kode_neil","controls":["TreatGroup","Post"],"dataset":"ref_panel","x":["DiD"],"y":"lcapital_exp_pc"},"tool":"infer.ols"}],"needs":[],"provenance":"benchmark_script","quotes":{},"reference_datasets":["ref_e04","ref_e05","ref_panel","ref_years"],"source_id":"crb_papers.q052.script"}},"question":"How did logged district capital expenditures per capita change in 2005 in districts holding a district head election that year versus districts that did not hold an election that year (restricting to districts without an election the year prior)?","requires":[],"runner":null,"sources":["crb_papers.q052.gold","crb_papers.q052.script"],"split":"public","suite":"crb_papers","tags":["econ-bench","external","crb","did","ref_benchmark_script"],"target":"agent","tool":null,"truth":"CausalReasoningBenchmark gold at the pinned revision: causal_queries.json effect and standard_error (the value the item's estimation.py prints), scored as evaluate_result.py scores estimate_within_solution_ci"}