{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number traced to a tool call or the question","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":true,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"CRB estimate_within_solution_ci: the answer figure inside the gold 95% interval effect \u00b1 1.96\u00b7|SE| of source crb_papers.q074.gold (closed, evaluate_result.py; no --auto-rescale)","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":[-0.36172254721849234,0.02393113006444131],"weight":1.0,"where":null},{"abs_tol":9.838104012319735e-08,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold effect to rel 1e-6 (abs floor 1e-6\u00b7SE); never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q074.gold","table":null,"tool":null,"value":-0.16889570857702552,"weight":1.0,"where":null},{"abs_tol":9.838104012319735e-08,"bind":"answer_call","column":null,"failure_class":null,"field":"se","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold SE to rel 1e-6; never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q074.gold","table":null,"tool":null,"value":0.09838104012319736,"weight":1.0,"where":null}],"datasets":{"ds_0044":{"columns":null,"description":"| Variable Name  | Description                                                                                 |\n|----------------|---------------------------------------------------------------------------------------------|\n| Unnamed/Index  | Row identifier / index column carried over from the source CSV (sequence number).          |\n| city           | Municipality name and state (lowercase string, e.g., \"akron, oh\").                         |\n| YEAR           | Census year (decennial year) for the observation (numeric).                                |\n| occ            | Occupation class for municipal employment (e.g., \"blue_collar\", \"white_collar\", \"police\").  |\n| name           | Demographic group label combining race and nativity (e.g., \"white_x_native_born\").         |\n| govt           | Proportion (share) of municipal government jobs in this city\u2013year\u2013occupation held by group. |\n| pop            | Proportion (share) of the municipality's total population comprised by the named group.     |\n| treat_assign   | Binary indicator whether a statewide legislative mandate applied to the municipality that decade (0/1). |\n| treat_actual   | Binary indicator whether the municipality had the municipal civil service commission in effect that decade (0/1). |\n| bandwidth      | Numeric distance in residents between the municipality's population and the state cutoff used in assignment rules. |\n\nData Description: This dataset is a municipality-by-decade panel that links full-count U.S. decennial census records (1850\u20131940) on municipal employment and population composition with administrative indicators derived from state statutes and municipal surveys. Each observation identifies a city and census year, an occupation class, and a demographic group, and reports group shares in the municipal population and in the local government workforce; the file also includes binary flags recording whether a state-level mandate applied and whether a municipal civil service commission was in place, plus a numeric distance-from-cutoff measure used for threshold-based comparisons. The underlying sources include full-count census enumerations (industry/occupation and birthplace fields), historical surveys of municipal civil service adoption, and compiled state session-law information assembled to create a harmonized longitudinal dataset on municipal staff composition.","file_sha256":"d5035ee26421d944dd396fa22c1f412e72a32f59388ef130ad486a3c851a813b","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"d5035ee26421d944dd396fa22c1f412e72a32f59388ef130ad486a3c851a813b","transforms":[],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0044/data/data.csv"},"ref_iv":{"columns":null,"description":"Blue-collar white native-born rows, complete, with year and city dummies","file_sha256":"d5035ee26421d944dd396fa22c1f412e72a32f59388ef130ad486a3c851a813b","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"d5035ee26421d944dd396fa22c1f412e72a32f59388ef130ad486a3c851a813b","transforms":[{"args":{"query":"occ == 'blue_collar' and name == 'white_x_native_born'"},"op":"filter"},{"args":{"columns":["govt","treat_actual","treat_assign","pop","YEAR","city"]},"op":"select"},{"args":{"columns":["govt","treat_actual","treat_assign","pop","YEAR","city"]},"op":"dropna"},{"args":{"column":"YEAR","drop_first":true,"prefix":"YEAR"},"op":"dummies"},{"args":{"column":"city","drop_first":true,"prefix":"city"},"op":"dummies"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0044/data/data.csv"}},"difficulty":"M","id":"crb_papers.q074","level":"T2","meta":{"agent_reach":{"blocked_by":["reference_frame:ref_iv","needs_op:iv_se_hc0"],"dtypes":{"ds_0044":{"govt":"float64","treat_actual":"float64"}},"named":{"outcome":["govt"],"treatment":["treat_actual"]},"reachable":false,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":null,"unit_scale":1.0},"cluster":"ds_0044","crb":{"data_path":"data/ds_0044/data/data.csv","dataset_id":"ds_0044","gold":{"ci95":[-0.36172254721849234,0.02393113006444131],"ci_rule":"effect \u00b1 1.96\u00b7|standard_error| (evaluate_result.ci_bounds)","effect":-0.16889570857702552,"standard_error":0.09838104012319736},"identification":{"causal_quantity":"LATE","controls":["pop","YEAR","city"],"instrument":["treat_assign"],"is_encouragement_design":false,"minimal_controlling_set":["pop","YEAR","city"],"outcomes":["govt"],"reason_for_minimal_controlling_set":"\u201cWe also include city fixed effects (\u03b4) and decade fixed effects (\u03b6), which control for unobserved municipality and decade\u2011specific factors.\u201d And because the estimand is representational over/under\u2011representation, \u201cwe include X_mt, which is a vector of values capturing a given group\u2019s share of the total population in decade t in municipality m.\u201d (Research Design \u2014 Empirical Strategy 1, p. 207). The IV specification mirrors this setup, adding the same controls in both stages (Empirical Strategy 2, pp. 208\u2013209).","strategy":"Instrumental Variable","treatments":["treat_actual"],"variant":"fuzzy; staggered adoption instrumented by state mandates"},"identification_strategy":"Instrumental Variable"},"headline_keys":{"infer.iv_2sls":["coef.treat_actual"]},"naive":{"dataset":"ds_0044","treatment":"treat_actual","y":"govt"},"native_id":"74","reference":{"cite":1,"note":"Reference path transcribed from the item's own estimation script.","steps":[{"args":{"dataset":"ref_iv","endog":["treat_actual"],"exog":["pop","YEAR_1860.0","YEAR_1870.0","YEAR_1880.0","YEAR_1900.0","YEAR_1910.0","YEAR_1920.0","YEAR_1930.0","YEAR_1940.0","city_albany, ny","city_alexandria, la","city_allentown, pa","city_alliance, oh","city_altoona, pa","city_ames, ia","city_amsterdam, ny","city_anniston, al","city_appleton, wi","city_arnold, pa","city_ashland, oh","city_ashland, wi","city_ashtabula, oh","city_atlantic city, nj","city_attleboro, ma","city_auburn, ny","city_augusta, ga","city_barberton, oh","city_batavia, ny","city_baton rouge, la","city_beacon, ny","city_beaver falls, pa","city_bellaire, oh","city_beloit, wi","city_benton harbor, mi","city_bethlehem, pa","city_beverly, ma","city_binghamton, ny","city_boise, id","city_boone, ia","city_boston, ma","city_bradford, pa","city_brainerd, mn","city_brockton, ma","city_bucyrus, oh","city_buffalo, ny","city_burlington, ia","city_butler, pa","city_cambridge, ma","city_cambridge, oh","city_camden, nj","city_campbell, oh","city_canton, oh","city_carbondale, pa","city_casper, wy","city_cedar rapids, ia","city_chelsea, ma","city_chester, pa","city_cheyenne, wy","city_chicopee, ma","city_chillicothe, oh","city_cincinnati, oh","city_cleveland heights, oh","city_cleveland, oh","city_clifton, nj","city_clinton, ia","city_coatesville, pa","city_coffeyville, ks","city_cohoes, ny","city_columbia, sc","city_columbus, oh","city_connellsville, pa","city_corning, ny","city_corpus christi, tx","city_cortland, ny","city_coshocton, oh","city_council bluffs, ia","city_cudahy, wi","city_cuyahoga falls, oh","city_dallas, tx","city_davenport, ia","city_dayton, oh","city_dearborn, mi","city_denver, co","city_des moines, ia","city_dover, nj","city_du bois, pa","city_dubuque, ia","city_dunkirk, ny","city_duquesne, pa","city_east cleveland, oh","city_east liverpool, oh","city_east orange, nj","city_easton, pa","city_eau claire, wi","city_el paso, tx","city_elizabeth, nj","city_elmira, ny","city_elyria, oh","city_endicott, ny","city_erie, pa","city_euclid, oh","city_eugene, or","city_everett, ma","city_fall river, ma","city_farrell, pa","city_findlay, oh","city_fitchburg, ma","city_floral park, ny","city_fond du lac, wi","city_fort dodge, ia","city_fort madison, ia","city_fort smith, ar","city_fostoria, oh","city_franklin, pa","city_freeport, ny","city_fremont, oh","city_fulton, ny","city_gadsden, al","city_gardner, ma","city_garfield heights, oh","city_gary, in","city_geneva, ny","city_glen cove, ny","city_glens falls, ny","city_gloucester, ma","city_gloversville, ny","city_greeley, co","city_green bay, wi","city_greensburg, pa","city_greenville, sc","city_hamilton, oh","city_harrisburg, pa","city_hastings, ne","city_haverhill, ma","city_hazleton, pa","city_hempstead, ny","city_hibbing, mn","city_holyoke, ma","city_hornell, ny","city_houston, tx","city_hudson, ny","city_hutchinson, ks","city_ilion, ny","city_indianapolis, in","city_iowa city, ia","city_ironton, oh","city_irvington, nj","city_ithaca, ny","city_jackson, mi","city_janesville, wi","city_jeannette, pa","city_jersey city, nj","city_johnson city, ny","city_johnstown, ny","city_johnstown, pa","city_kansas city, ks","city_kearny, nj","city_kenmore, ny","city_kenosha, wi","city_keokuk, ia","city_kingston, ny","city_klamath falls, or","city_la crosse, wi","city_lackawanna, ny","city_lafayette, la","city_lake charles, la","city_lakewood, oh","city_lancaster, oh","city_lancaster, pa","city_lawrence, ma","city_lebanon, pa","city_leominster, ma","city_lima, oh","city_little falls, ny","city_little rock, ar","city_lockport, ny","city_lorain, oh","city_lowell, ma","city_lynbrook, ny","city_lynn, ma","city_madison, wi","city_malden, ma","city_mamaroneck, ny","city_manitowoc, wi","city_mankato, mn","city_mansfield, oh","city_marietta, oh","city_marinette, wi","city_marion, oh","city_marlborough, ma","city_marshalltown, ia","city_martins ferry, oh","city_mason city, ia","city_massillon, oh","city_mckeesport, pa","city_meadville, pa","city_medford, ma","city_melrose, ma","city_middletown, ny","city_middletown, oh","city_milwaukee, wi","city_monroe, la","city_morristown, nj","city_muscatine, ia","city_nanticoke, pa","city_new bedford, ma","city_new castle, pa","city_new philadelphia, oh","city_new rochelle, ny","city_new york, ny","city_newark, nj","city_newark, oh","city_newburgh, ny","city_newburyport, ma","city_newton, ia","city_newton, ma","city_niagara falls, ny","city_niles, oh","city_north adams, ma","city_north little rock, ar","city_north tonawanda, ny","city_northampton, ma","city_norwood, oh","city_nutley, nj","city_ogdensburg, ny","city_oil city, pa","city_olean, ny","city_omaha, ne","city_oneida, ny","city_oneonta, ny","city_orange, nj","city_orlando, fl","city_oshkosh, wi","city_oskaloosa, ia","city_ossining, ny","city_oswego, ny","city_ottumwa, ia","city_painesville, oh","city_parma, oh","city_parsons, ks","city_paterson, nj","city_peabody, ma","city_peekskill, ny","city_philadelphia, pa","city_pine bluff, ar","city_piqua, oh","city_pittsburgh, pa","city_pittsfield, ma","city_pittston, pa","city_plattsburg, ny","city_pocatello, id","city_port chester, ny","city_port huron, mi","city_port jervis, ny","city_portland, or","city_portsmouth, oh","city_pottsville, pa","city_poughkeepsie, ny","city_quincy, ma","city_racine, wi","city_rahway, nj","city_reading, pa","city_revere, ma","city_rochester, mn","city_rochester, ny","city_rome, ny","city_rutherford, nj","city_salem, ma","city_salem, oh","city_salem, or","city_salt lake city, ut","city_san antonio, tx","city_sandusky, oh","city_saratoga springs, ny","city_scranton, pa","city_shaker heights, oh","city_sharon, pa","city_sheboygan, wi","city_shorewood, wi","city_shreveport, la","city_sioux city, ia","city_somerville, ma","city_south milwaukee, wi","city_south orange, nj","city_spartanburg, sc","city_springfield, ma","city_springfield, oh","city_st. augustine, fl","city_st. cloud, mn","city_steubenville, oh","city_stevens point, wi","city_struthers, oh","city_sunbury, pa","city_superior, wi","city_syracuse, ny","city_taunton, ma","city_teaneck, nj","city_tiffin, oh","city_toledo, oh","city_tonawanda, ny","city_topeka, ks","city_trenton, nj","city_troy, ny","city_tuscaloosa, al","city_two rivers, wi","city_union, nj","city_uniontown, pa","city_utica, ny","city_waco, tx","city_waltham, ma","city_warren, oh","city_washington, pa","city_waterloo, ia","city_watertown, ny","city_watertown, wi","city_watervliet, ny","city_waukesha, wi","city_wausau, wi","city_wauwatosa, wi","city_west allis, wi","city_west orange, nj","city_westfield, ma","city_white plains, ny","city_williamsport, pa","city_woburn, ma","city_wooster, oh","city_worcester, ma","city_xenia, oh","city_yonkers, ny","city_york, pa","city_youngstown, oh","city_zanesville, oh"],"instruments":["treat_assign"],"se":"HC1","y":"govt"},"tool":"infer.iv_2sls","why":"IV2SLS.from_formula('govt ~ 1 + pop + C(YEAR) + C(city) + [treat_actual ~ treat_assign]').fit(cov_type='robust')"}]},"reference_path":{"calls":[{"args":{"dataset":"ref_iv","endog":["treat_actual"],"exog":["pop","YEAR_1860.0","YEAR_1870.0","YEAR_1880.0","YEAR_1900.0","YEAR_1910.0","YEAR_1920.0","YEAR_1930.0","YEAR_1940.0","city_albany, ny","city_alexandria, la","city_allentown, pa","city_alliance, oh","city_altoona, pa","city_ames, ia","city_amsterdam, ny","city_anniston, al","city_appleton, wi","city_arnold, pa","city_ashland, oh","city_ashland, wi","city_ashtabula, oh","city_atlantic city, nj","city_attleboro, ma","city_auburn, ny","city_augusta, ga","city_barberton, oh","city_batavia, ny","city_baton rouge, la","city_beacon, ny","city_beaver falls, pa","city_bellaire, oh","city_beloit, wi","city_benton harbor, mi","city_bethlehem, pa","city_beverly, ma","city_binghamton, ny","city_boise, id","city_boone, ia","city_boston, ma","city_bradford, pa","city_brainerd, mn","city_brockton, ma","city_bucyrus, oh","city_buffalo, ny","city_burlington, ia","city_butler, pa","city_cambridge, ma","city_cambridge, oh","city_camden, nj","city_campbell, oh","city_canton, oh","city_carbondale, pa","city_casper, wy","city_cedar rapids, ia","city_chelsea, ma","city_chester, pa","city_cheyenne, wy","city_chicopee, ma","city_chillicothe, oh","city_cincinnati, oh","city_cleveland heights, oh","city_cleveland, oh","city_clifton, nj","city_clinton, ia","city_coatesville, pa","city_coffeyville, ks","city_cohoes, ny","city_columbia, sc","city_columbus, oh","city_connellsville, pa","city_corning, ny","city_corpus christi, tx","city_cortland, ny","city_coshocton, oh","city_council bluffs, ia","city_cudahy, wi","city_cuyahoga falls, oh","city_dallas, tx","city_davenport, ia","city_dayton, oh","city_dearborn, mi","city_denver, co","city_des moines, ia","city_dover, nj","city_du bois, pa","city_dubuque, ia","city_dunkirk, ny","city_duquesne, pa","city_east cleveland, oh","city_east liverpool, oh","city_east orange, nj","city_easton, pa","city_eau claire, wi","city_el paso, tx","city_elizabeth, nj","city_elmira, ny","city_elyria, oh","city_endicott, ny","city_erie, pa","city_euclid, oh","city_eugene, or","city_everett, ma","city_fall river, ma","city_farrell, pa","city_findlay, oh","city_fitchburg, ma","city_floral park, ny","city_fond du lac, wi","city_fort dodge, ia","city_fort madison, ia","city_fort smith, ar","city_fostoria, oh","city_franklin, pa","city_freeport, ny","city_fremont, oh","city_fulton, ny","city_gadsden, al","city_gardner, ma","city_garfield heights, oh","city_gary, in","city_geneva, ny","city_glen cove, ny","city_glens falls, ny","city_gloucester, ma","city_gloversville, ny","city_greeley, co","city_green bay, wi","city_greensburg, pa","city_greenville, sc","city_hamilton, oh","city_harrisburg, pa","city_hastings, ne","city_haverhill, ma","city_hazleton, pa","city_hempstead, ny","city_hibbing, mn","city_holyoke, ma","city_hornell, ny","city_houston, tx","city_hudson, ny","city_hutchinson, ks","city_ilion, ny","city_indianapolis, in","city_iowa city, ia","city_ironton, oh","city_irvington, nj","city_ithaca, ny","city_jackson, mi","city_janesville, wi","city_jeannette, pa","city_jersey city, nj","city_johnson city, ny","city_johnstown, ny","city_johnstown, pa","city_kansas city, ks","city_kearny, nj","city_kenmore, ny","city_kenosha, wi","city_keokuk, ia","city_kingston, ny","city_klamath falls, or","city_la crosse, wi","city_lackawanna, ny","city_lafayette, la","city_lake charles, la","city_lakewood, oh","city_lancaster, oh","city_lancaster, pa","city_lawrence, ma","city_lebanon, pa","city_leominster, ma","city_lima, oh","city_little falls, ny","city_little rock, ar","city_lockport, ny","city_lorain, oh","city_lowell, ma","city_lynbrook, ny","city_lynn, ma","city_madison, wi","city_malden, ma","city_mamaroneck, ny","city_manitowoc, wi","city_mankato, mn","city_mansfield, oh","city_marietta, oh","city_marinette, wi","city_marion, oh","city_marlborough, ma","city_marshalltown, ia","city_martins ferry, oh","city_mason city, ia","city_massillon, oh","city_mckeesport, pa","city_meadville, pa","city_medford, ma","city_melrose, ma","city_middletown, ny","city_middletown, oh","city_milwaukee, wi","city_monroe, la","city_morristown, nj","city_muscatine, ia","city_nanticoke, pa","city_new bedford, ma","city_new castle, pa","city_new philadelphia, oh","city_new rochelle, ny","city_new york, ny","city_newark, nj","city_newark, oh","city_newburgh, ny","city_newburyport, ma","city_newton, ia","city_newton, ma","city_niagara falls, ny","city_niles, oh","city_north adams, ma","city_north little rock, ar","city_north tonawanda, ny","city_northampton, ma","city_norwood, oh","city_nutley, nj","city_ogdensburg, ny","city_oil city, pa","city_olean, ny","city_omaha, ne","city_oneida, ny","city_oneonta, ny","city_orange, nj","city_orlando, fl","city_oshkosh, wi","city_oskaloosa, ia","city_ossining, ny","city_oswego, ny","city_ottumwa, ia","city_painesville, oh","city_parma, oh","city_parsons, ks","city_paterson, nj","city_peabody, ma","city_peekskill, ny","city_philadelphia, pa","city_pine bluff, ar","city_piqua, oh","city_pittsburgh, pa","city_pittsfield, ma","city_pittston, pa","city_plattsburg, ny","city_pocatello, id","city_port chester, ny","city_port huron, mi","city_port jervis, ny","city_portland, or","city_portsmouth, oh","city_pottsville, pa","city_poughkeepsie, ny","city_quincy, ma","city_racine, wi","city_rahway, nj","city_reading, pa","city_revere, ma","city_rochester, mn","city_rochester, ny","city_rome, ny","city_rutherford, nj","city_salem, ma","city_salem, oh","city_salem, or","city_salt lake city, ut","city_san antonio, tx","city_sandusky, oh","city_saratoga springs, ny","city_scranton, pa","city_shaker heights, oh","city_sharon, pa","city_sheboygan, wi","city_shorewood, wi","city_shreveport, la","city_sioux city, ia","city_somerville, ma","city_south milwaukee, wi","city_south orange, nj","city_spartanburg, sc","city_springfield, ma","city_springfield, oh","city_st. augustine, fl","city_st. cloud, mn","city_steubenville, oh","city_stevens point, wi","city_struthers, oh","city_sunbury, pa","city_superior, wi","city_syracuse, ny","city_taunton, ma","city_teaneck, nj","city_tiffin, oh","city_toledo, oh","city_tonawanda, ny","city_topeka, ks","city_trenton, nj","city_troy, ny","city_tuscaloosa, al","city_two rivers, wi","city_union, nj","city_uniontown, pa","city_utica, ny","city_waco, tx","city_waltham, ma","city_warren, oh","city_washington, pa","city_waterloo, ia","city_watertown, ny","city_watertown, wi","city_watervliet, ny","city_waukesha, wi","city_wausau, wi","city_wauwatosa, wi","city_west allis, wi","city_west orange, nj","city_westfield, ma","city_white plains, ny","city_williamsport, pa","city_woburn, ma","city_wooster, oh","city_worcester, ma","city_xenia, oh","city_yonkers, ny","city_york, pa","city_youngstown, oh","city_zanesville, oh"],"instruments":["treat_assign"],"se":"HC1","y":"govt"},"tool":"infer.iv_2sls"}],"needs":["needs_op:iv_se_hc0"],"provenance":"benchmark_script","quotes":{},"reference_datasets":["ref_iv"],"source_id":"crb_papers.q074.script"}},"question":"What is the causal effect of municipal civil service reform adoption on the share of blue-collar municipal government jobs held by native-born white workers, estimated for municipalities whose adoption was externally induced (local compliers)?","requires":[],"runner":null,"sources":["crb_papers.q074.gold","crb_papers.q074.script"],"split":"public","suite":"crb_papers","tags":["econ-bench","external","crb","iv","ref_benchmark_script"],"target":"agent","tool":null,"truth":"CausalReasoningBenchmark gold at the pinned revision: causal_queries.json effect and standard_error (the value the item's estimation.py prints), scored as evaluate_result.py scores estimate_within_solution_ci"}