{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number traced to a tool call or the question","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":true,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"CRB estimate_within_solution_ci: the answer figure inside the gold 95% interval effect \u00b1 1.96\u00b7|SE| of source crb_papers.q028.gold (closed, evaluate_result.py; no --auto-rescale)","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":[0.023441412809946716,2.449828160814865],"weight":1.0,"where":null},{"abs_tol":6.189762112257445e-07,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold effect to rel 1e-6 (abs floor 1e-6\u00b7SE); never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q028.gold","table":null,"tool":null,"value":1.236634786812406,"weight":1.0,"where":null},{"abs_tol":6.189762112257445e-07,"bind":"answer_call","column":null,"failure_class":null,"field":"se","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold SE to rel 1e-6; never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q028.gold","table":null,"tool":null,"value":0.6189762112257445,"weight":1.0,"where":null}],"datasets":{"ds_0015":{"columns":null,"description":"| Variable Name | Description |\n| --- | --- |\n| bfs | Swiss Federal Statistical Office (BFS) municipality code: a stable numeric identifier for each municipality in the panel. |\n| year | Calendar year of the observation (1991\u20132009). Institutional status is referenced as of January 1 of the given year, and annual rates are computed for the same year. |\n| nat_rate_ord | Ordinary naturalization rate (percent). Computed as the number of ordinary naturalizations recorded during the year divided by the stock of foreign residents who met statutory eligibility at the start of the year in the same municipality, multiplied by 100. Ordinary naturalizations refer to the standard pathway (as opposed to federally handled \u201cfacilitated\u201d cases). |\n| indirect | Local decision procedure in place on January 1: 1 if applications were decided by the elected municipal council (representative/indirect democracy); 0 if applications were decided by citizens in a local vote (direct democracy). |\n\nDataset Context:\nThis dataset is an annual municipality\u2013year panel for Switzerland that spans a period of institutional change in how local authorities decide on citizenship applications. Switzerland\u2019s \u201ctriple citizenship\u201d system assigns a central role to municipalities in evaluating and deciding ordinary naturalization requests. During the 1990s and 2000s, municipalities used different decision procedures: in some, citizens voted on applications (often at a citizens\u2019 assembly and, historically in a few places, by ballot); in others, elected municipal councils rendered the decisions. A series of Swiss Federal Court rulings in 2003\u20132005 curtailed secret-ballot referendums for naturalizations and required municipalities to provide formal justifications for rejections. Several cantons subsequently updated their laws, and many municipalities revised or formalized their procedures, frequently shifting decision authority to municipal councils. The indicator in this file captures, for each municipality and year, which procedure was in force as of January 1.\n\nThe panel combines two types of information. First, administrative registers from the Swiss Federal Statistical Office (PETRA: Foreign Population Structure and Migration Statistics) are used to derive annual ordinary naturalization rates. These rates are constructed as ordinary naturalizations completed within a year divided by the number of foreign residents who met the legal eligibility criteria at the start of that year, providing a comparable percentage measure across municipalities and over time. Second, the coding of local decision procedures relies on a nationwide survey of municipal head secretaries and systematic compilation of cantonal regulations and court decisions. The measurement convention aligns the institutional status to January 1 to enhance comparability across municipalities, recognizing that formal switches may occur during a calendar year.\n\nOrdinary naturalizations represent the standard route to citizenship and, in the study period, typically occurred after lengthy processing that could extend several years from initial application to final local vote. Facilitated naturalizations\u2014handled centrally under federal law (for example, for certain spouses of Swiss citizens)\u2014are not included in the numerator of the ordinary naturalization rate. The resulting dataset provides a consistent view of local institutions and annual naturalization activity across more than a decade and a half, covering the pre- and post-reform environment in which many municipalities updated their decision-making arrangements.","file_sha256":"e9ce75e30cb08926da17f27cb91ecb80f964f077f9a592e052d672b61239e8f9","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"e9ce75e30cb08926da17f27cb91ecb80f964f077f9a592e052d672b61239e8f9","transforms":[],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0015/data/data.csv"},"ref_first":{"columns":null,"description":"Each municipality's first year with indirect == 1 (sw)","file_sha256":"e9ce75e30cb08926da17f27cb91ecb80f964f077f9a592e052d672b61239e8f9","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"e9ce75e30cb08926da17f27cb91ecb80f964f077f9a592e052d672b61239e8f9","transforms":[{"args":{"columns":["bfs","year","nat_rate_ord","indirect"]},"op":"select"},{"args":{"columns":["bfs","year","nat_rate_ord"]},"op":"dropna"},{"args":{"query":"indirect == 1"},"op":"filter"},{"args":{"aggs":[{"column":"year","fn":"min","name":"sw"}],"by":["bfs"]},"op":"collapse"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0015/data/data.csv"},"ref_panel":{"columns":null,"description":"Municipality rows of 2005 and 2007: first treated in 2006, or not treated by 2007 (incl. never)","file_sha256":"e9ce75e30cb08926da17f27cb91ecb80f964f077f9a592e052d672b61239e8f9","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"e9ce75e30cb08926da17f27cb91ecb80f964f077f9a592e052d672b61239e8f9","transforms":[{"args":{"columns":["bfs","year","nat_rate_ord","indirect"]},"op":"select"},{"args":{"columns":["bfs","year","nat_rate_ord"]},"op":"dropna"},{"args":{"query":"year in [2005, 2007]"},"op":"filter"},{"args":{"how":"left","on":["bfs"],"with":"ref_first"},"op":"merge"},{"args":{"how":"left","on":["bfs"],"with":"ref_years"},"op":"merge"},{"args":{"query":"nyears == 2 and (sw == 2006 or sw > 2007 or sw != sw)"},"op":"filter"},{"args":{"args":{"cmp":"eq","column":"sw","value":2006},"name":"group_treated","op":"indicator"},"op":"derive"},{"args":{"args":{"column":"group_treated","default":0,"keep_unmapped":false,"mapping":[{"source":1,"target":1}]},"name":"group_treated","op":"map"},"op":"derive"},{"args":{"args":{"cmp":"eq","column":"year","value":2007},"name":"post","op":"indicator"},"op":"derive"},{"args":{"args":{"columns":["group_treated","post"]},"name":"g_post","op":"interaction"},"op":"derive"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0015/data/data.csv"},"ref_years":{"columns":null,"description":"How many of 2005 and 2007 each municipality is observed in","file_sha256":"e9ce75e30cb08926da17f27cb91ecb80f964f077f9a592e052d672b61239e8f9","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"e9ce75e30cb08926da17f27cb91ecb80f964f077f9a592e052d672b61239e8f9","transforms":[{"args":{"columns":["bfs","year","nat_rate_ord","indirect"]},"op":"select"},{"args":{"columns":["bfs","year","nat_rate_ord"]},"op":"dropna"},{"args":{"query":"year in [2005, 2007]"},"op":"filter"},{"args":{"aggs":[{"column":"nat_rate_ord","fn":"count","name":"n"}],"by":["bfs","year"]},"op":"collapse"},{"args":{"aggs":[{"column":"n","fn":"count","name":"nyears"}],"by":["bfs"]},"op":"collapse"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0015/data/data.csv"}},"difficulty":"M","id":"crb_papers.q028","level":"T2","meta":{"agent_reach":{"blocked_by":["reference_frame:ref_first","reference_frame:ref_panel","reference_frame:ref_years"],"dtypes":{"ds_0015":{"indirect":"float64","nat_rate_ord":"float64"}},"named":{"outcome":["nat_rate_ord"],"treatment":["indirect"]},"reachable":false,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":null,"unit_scale":1.0},"cluster":"ds_0015","crb":{"data_path":"data/ds_0015/data/data.csv","dataset_id":"ds_0015","gold":{"ci95":[0.023441412809946716,2.449828160814865],"ci_rule":"effect \u00b1 1.96\u00b7|standard_error| (evaluate_result.ci_bounds)","effect":1.236634786812406,"standard_error":0.6189762112257445},"identification":{"causal_quantity":"ATT","group_variable":"bfs","minimal_controlling_set":[],"outcomes":["nat_rate_ord"],"strategy":"Difference-in-Differences","time_variable":"year","treatments":["indirect"]},"identification_strategy":"Difference-in-Differences"},"headline_keys":{"infer.ols":["coef.g_post"]},"naive":{"dataset":"ds_0015","treatment":"indirect","y":"nat_rate_ord"},"native_id":"28","reference":{"cite":1,"note":"Reference path transcribed from the item's own estimation script.","steps":[{"args":{"cluster":"bfs","controls":["group_treated","post"],"dataset":"ref_panel","x":["g_post"],"y":"nat_rate_ord"},"tool":"infer.ols","why":"sm.OLS(y, add_constant([group_treated, post, g_post])).fit(cov_type='cluster', groups=bfs) (CR1 errors with statsmodels' default G/(G-1)\u00b7(N-1)/(N-K) correction)"}]},"reference_path":{"calls":[{"args":{"cluster":"bfs","controls":["group_treated","post"],"dataset":"ref_panel","x":["g_post"],"y":"nat_rate_ord"},"tool":"infer.ols"}],"needs":[],"provenance":"benchmark_script","quotes":{},"reference_datasets":["ref_first","ref_panel","ref_years"],"source_id":"crb_papers.q028.script"}},"question":"Among municipalities that switched to council decision-making in 2006, how large was the change in the ordinary naturalization rate from 2005 to 2007 compared with municipalities still using citizen referendums in 2007?","requires":[],"runner":null,"sources":["crb_papers.q028.gold","crb_papers.q028.script"],"split":"public","suite":"crb_papers","tags":["econ-bench","external","crb","did","ref_benchmark_script"],"target":"agent","tool":null,"truth":"CausalReasoningBenchmark gold at the pinned revision: causal_queries.json effect and standard_error (the value the item's estimation.py prints), scored as evaluate_result.py scores estimate_within_solution_ci"}