{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":"unbacked_number","field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":"no_estimate_cited","field":"value","interval":false,"key":null,"kind":"answer_figure","min_lre":null,"note":"an answer figure: the effect of an estimator call the answer cites","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":"wrong_tool","field":"value","interval":false,"key":null,"kind":"tool_called","min_lre":null,"note":"","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":"infer.did_staggered","value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":"wrong_args","field":"value","interval":false,"key":"y","kind":"arg_equals","min_lre":null,"note":"outcome: log county teen employment","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":"infer.did_staggered","value":"lemp","weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":"wrong_args","field":"value","interval":false,"key":"first_treat","kind":"arg_equals","min_lre":null,"note":"the adoption cohort column","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":"infer.did_staggered","value":"first_treat","weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":"wrong_args","field":"value","interval":false,"key":"unit","kind":"arg_equals","min_lre":null,"note":"the county id","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":"infer.did_staggered","value":"countyreal","weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":"wrong_args","field":"value","interval":false,"key":"control_group","kind":"arg_equals","min_lre":null,"note":"never-treated counties as the comparison group","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":"infer.did_staggered","value":"never_treated","weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":"wrong_sample","field":"value","interval":false,"key":"n_units","kind":"diagnostic","min_lre":null,"note":"","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":"infer.did_staggered","value":500,"weight":1.0,"where":null},{"abs_tol":1.54e-08,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"software reference, cell e_m3","open_interval":false,"rel_tol":1e-06,"required":true,"source_id":"callaway_santanna_2021.mpdta_event_time.e_m3.software","table":"event_time","tool":"infer.did_staggered","value":0.030506655583292096,"weight":1.0,"where":{"e":-3}},{"abs_tol":null,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_m3: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_m3.paper","table":"event_time","tool":"infer.did_staggered","value":0.0305,"weight":1.0,"where":{"e":-3}},{"abs_tol":null,"bind":"answer_call","column":"se","failure_class":null,"field":"se","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_m3, SE: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_m3.paper","table":"event_time","tool":"infer.did_staggered","value":0.0154,"weight":1.0,"where":{"e":-3}},{"abs_tol":1.3199999999999999e-08,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"software reference, cell e_m2","open_interval":false,"rel_tol":1e-06,"required":true,"source_id":"callaway_santanna_2021.mpdta_event_time.e_m2.software","table":"event_time","tool":"infer.did_staggered","value":-0.0005630846263852302,"weight":1.0,"where":{"e":-2}},{"abs_tol":null,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_m2: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_m2.paper","table":"event_time","tool":"infer.did_staggered","value":-0.0006,"weight":1.0,"where":{"e":-2}},{"abs_tol":null,"bind":"answer_call","column":"se","failure_class":null,"field":"se","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_m2, SE: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_m2.paper","table":"event_time","tool":"infer.did_staggered","value":0.0132,"weight":1.0,"where":{"e":-2}},{"abs_tol":1.5299999999999998e-08,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"software reference, cell e_m1","open_interval":false,"rel_tol":1e-06,"required":true,"source_id":"callaway_santanna_2021.mpdta_event_time.e_m1.software","table":"event_time","tool":"infer.did_staggered","value":-0.024458744971168973,"weight":1.0,"where":{"e":-1}},{"abs_tol":null,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_m1: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_m1.paper","table":"event_time","tool":"infer.did_staggered","value":-0.0245,"weight":1.0,"where":{"e":-1}},{"abs_tol":null,"bind":"answer_call","column":"se","failure_class":null,"field":"se","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_m1, SE: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_m1.paper","table":"event_time","tool":"infer.did_staggered","value":0.0153,"weight":1.0,"where":{"e":-1}},{"abs_tol":1.23e-08,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"software reference, cell e_0","open_interval":false,"rel_tol":1e-06,"required":true,"source_id":"callaway_santanna_2021.mpdta_event_time.e_0.software","table":"event_time","tool":"infer.did_staggered","value":-0.01993181678925978,"weight":1.0,"where":{"e":0}},{"abs_tol":null,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_0: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_0.paper","table":"event_time","tool":"infer.did_staggered","value":-0.0199,"weight":1.0,"where":{"e":0}},{"abs_tol":null,"bind":"answer_call","column":"se","failure_class":null,"field":"se","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_0, SE: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_0.paper","table":"event_time","tool":"infer.did_staggered","value":0.0123,"weight":1.0,"where":{"e":0}},{"abs_tol":1.6699999999999997e-08,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"software reference, cell e_1","open_interval":false,"rel_tol":1e-06,"required":true,"source_id":"callaway_santanna_2021.mpdta_event_time.e_1.software","table":"event_time","tool":"infer.did_staggered","value":-0.050957367065194985,"weight":1.0,"where":{"e":1}},{"abs_tol":null,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_1: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_1.paper","table":"event_time","tool":"infer.did_staggered","value":-0.051,"weight":1.0,"where":{"e":1}},{"abs_tol":null,"bind":"answer_call","column":"se","failure_class":null,"field":"se","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_1, SE: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_1.paper","table":"event_time","tool":"infer.did_staggered","value":0.0167,"weight":1.0,"where":{"e":1}},{"abs_tol":4.13e-08,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"software reference, cell e_2","open_interval":false,"rel_tol":1e-06,"required":true,"source_id":"callaway_santanna_2021.mpdta_event_time.e_2.software","table":"event_time","tool":"infer.did_staggered","value":-0.13725873888940443,"weight":1.0,"where":{"e":2}},{"abs_tol":null,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_2: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_2.paper","table":"event_time","tool":"infer.did_staggered","value":-0.1373,"weight":1.0,"where":{"e":2}},{"abs_tol":null,"bind":"answer_call","column":"se","failure_class":null,"field":"se","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_2, SE: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_2.paper","table":"event_time","tool":"infer.did_staggered","value":0.0413,"weight":1.0,"where":{"e":2}},{"abs_tol":3.5099999999999997e-08,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"software reference, cell e_3","open_interval":false,"rel_tol":1e-06,"required":true,"source_id":"callaway_santanna_2021.mpdta_event_time.e_3.software","table":"event_time","tool":"infer.did_staggered","value":-0.10081136308540525,"weight":1.0,"where":{"e":3}},{"abs_tol":null,"bind":"answer_call","column":"att","failure_class":null,"field":"value","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_3: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_3.paper","table":"event_time","tool":"infer.did_staggered","value":-0.1008,"weight":1.0,"where":{"e":3}},{"abs_tol":null,"bind":"answer_call","column":"se","failure_class":null,"field":"se","interval":false,"key":null,"kind":"table_close","min_lre":null,"note":"published cell e_3, SE: recorded, never required on T2/T3","open_interval":false,"rel_tol":null,"required":false,"source_id":"callaway_santanna_2021.mpdta_event_time.e_3.paper","table":"event_time","tool":"infer.did_staggered","value":0.0351,"weight":1.0,"where":{"e":3}}],"datasets":{"mpdta":{"columns":null,"description":"The did package's example panel mpdta (Callaway and Sant'Anna 2021): county-level teen employment in 500 U.S. counties, 2003-2007, one row per county and year. Some states first raised their minimum wage in 2004, some in 2006 and some in 2007; the others did not in the window. Columns: countyreal, the county id; year; lemp, the log of county teen employment; lpop, the log of county population; first_treat, the year the county's state first raised its minimum wage (2004, 2006 or 2007; 0 for counties never treated in the window); treat, 1 for counties in states that raised it at some point in the window.","file_sha256":"0be3bc5eaaedaa50e047deaf45ad0e193e7210df1644103c63a6c1baa75965d4","format":"rdata","frame_sha256":null,"generator":null,"kind":"file","licence":"R package did (GPL-2); fetched, not redistributed.","member":null,"object":"mpdta","params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"0be3bc5eaaedaa50e047deaf45ad0e193e7210df1644103c63a6c1baa75965d4","transforms":[{"args":{"map":{"first.treat":"first_treat"}},"op":"rename"}],"url":"https://raw.githubusercontent.com/bcallaway11/did/9c930239f718c4196353aa27525c09f458f29712/data/mpdta.rda"}},"difficulty":"H","id":"paper_repro.callaway_santanna_2021.mpdta_event_time.agent","level":"T2","meta":{"acceptable_tools":["infer.did_staggered"],"answer_format":{"figure_transform":"identity","round_to":null,"unit_scale":1.0},"cluster":"callaway_santanna_2021","contamination":{"benchmark_first_public":"2021-12","gold_in_public_repo":true,"model_reported_cutoff":null,"url":"https://doi.org/10.1016/j.jeconom.2020.12.001"},"expected_tool":"infer.did_staggered","family":"did","headline_keys":{"infer.did_staggered":["att"]},"kind":"agent","known_deviation":null,"locator":"did package home page (2.5.1), mpdta example, summary(aggte(out, type = 'dynamic')): the 7 event-time effects (the overall dynamic average, a mean over event times from 0, is not a cell any tool states and is not checked)","n_published":2500,"paper":"callaway_santanna_2021","question_states":[],"reference":{"cite":1,"steps":[{"args":{"control_group":"never_treated","dataset":"mpdta","first_treat":"first_treat","time":"year","unit":"countyreal","y":"lemp"},"tool":"infer.did_staggered","why":"att_gt(yname = 'lemp', gname = 'first.treat', idname = 'countyreal', tname = 'year', xformla = ~1, data = mpdta, est_method = 'reg'): never-treated control group, no covariates, varying base period (the package defaults)"}]},"se_policy":{"basis":"The package page prints bootstrap SEs, and they moved between two copies of the page (0.0243 to 0.0246, 0.0203 to 0.0219); DESIGN.md 1.3 and validity-12: bootstrap SEs are never required.","convention":"multiplier bootstrap (the did package's default); infer.did_staggered reports its own seeded bootstrap","never_required":true,"stated_in_question":false},"target":"mpdta_event_time","trap_tools":[]},"question":"Callaway and Sant'Anna (2021), 'Difference-in-Differences with Multiple Time Periods', as in the did package's documented example on its county panel (2003-2007): estimate the group-time average treatment effects of a state minimum-wage increase on log county teen employment, using the never-treated counties as the comparison group, no covariates and the package's default (varying) base period, and aggregate them by time since adoption into the event-study (dynamic) effects, from three periods before adoption to three after, weighting the cohorts by their size. Report the dynamic effects.","requires":[],"runner":null,"sources":["callaway_santanna_2021.mpdta_event_time.e_m3.paper","callaway_santanna_2021.mpdta_event_time.e_m3.software","callaway_santanna_2021.mpdta_event_time.e_m2.paper","callaway_santanna_2021.mpdta_event_time.e_m2.software","callaway_santanna_2021.mpdta_event_time.e_m1.paper","callaway_santanna_2021.mpdta_event_time.e_m1.software","callaway_santanna_2021.mpdta_event_time.e_0.paper","callaway_santanna_2021.mpdta_event_time.e_0.software","callaway_santanna_2021.mpdta_event_time.e_1.paper","callaway_santanna_2021.mpdta_event_time.e_1.software","callaway_santanna_2021.mpdta_event_time.e_2.paper","callaway_santanna_2021.mpdta_event_time.e_2.software","callaway_santanna_2021.mpdta_event_time.e_3.paper","callaway_santanna_2021.mpdta_event_time.e_3.software"],"split":"public","suite":"paper_repro","tags":["paper_repro","callaway_santanna_2021","did"],"target":"agent","tool":null,"truth":"Callaway, B. and Sant'Anna, P. H. C. (2021). Difference-in-Differences with Multiple Time Periods. Journal of Econometrics 225(2): 200-230. Cells from the did R package's documented example on its mpdta data (a software reference benchmark, not a table of the paper). did package home page (2.5.1), mpdta example, summary(aggte(out, type = 'dynamic')): the 7 event-time effects (the overall dynamic average, a mean over event times from 0, is not a cell any tool states and is not checked) (source callaway_santanna_2021.mpdta_event_time.e_m3.paper and 7 table cells)"}