{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number traced to a tool call or the question","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":true,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"CRB estimate_within_solution_ci: the answer figure inside the gold 95% interval effect \u00b1 1.96\u00b7|SE| of source crb_papers.q017.gold (closed, evaluate_result.py; no --auto-rescale)","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":[-1.0028823886598044,3.5429155415073774],"weight":1.0,"where":null},{"abs_tol":1.159642329124281e-06,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold effect to rel 1e-6 (abs floor 1e-6\u00b7SE); never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q017.gold","table":null,"tool":null,"value":1.2700165764237865,"weight":1.0,"where":null},{"abs_tol":1.159642329124281e-06,"bind":"answer_call","column":null,"failure_class":null,"field":"se","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold SE to rel 1e-6; never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q017.gold","table":null,"tool":null,"value":1.159642329124281,"weight":1.0,"where":null}],"datasets":{"ds_0009":{"columns":null,"description":"| Variable Name | Description |\n| --- | --- |\n| actor.id | Stable, URI-style identifier for a public Twitter account belonging to a Venezuelan opposition figure (e.g., legislators, mayors, activists, or journalists). Uniquely identifies the account across all months in the panel. |\n| month | Month start date (YYYY-MM-01) indicating the calendar month over which tweets were aggregated. Coverage runs from 2013-01-01 through 2020-05-01. |\n| perc_foreign_policy | Percentage of an account\u2019s tweets in the given month that reference foreign-led policy topics (e.g., diplomacy, sanctions, international pressure, or military action). Constructed via dictionary-based text classification informed by word-embedding similarity trained on the corpus; reported as a percent (0\u2013100), with fractional values possible (e.g., 0.78 means 0.78%). |\n| tweeted_exile | Indicator (0/1) that the individual associated with the account was residing outside Venezuela during the month. Coded from publicly available reporting and biographical sources; months are flagged once a departure is documented and until return, where known. |\n| log_tweets | Natural logarithm of the total number of public tweets posted by the account in the month, providing a scale-stable measure of activity volume. |\n| actor | Text identifier for the account (often the same string as actor.id), included for readability and consistency with external sources. |\n| months | Sequential month index starting at 1 for the first month in the dataset (January 2013) and increasing by one each calendar month. Useful for aligning time across accounts. |\n| actorid | Numeric panel identifier for the Twitter account, facilitating joins and fixed account indexing across files or analyses. |\n\nDataset Context:\nThis dataset assembles a monthly panel of Twitter activity from prominent Venezuelan opposition figures across a period of intense political and economic turbulence, spanning January 2013 to May 2020. The roster includes elected officials and well-known public voices who were active on the platform. Researchers identified accounts manually and used Twitter\u2019s full-archive access to retrieve complete public tweet histories. Tweets were then aggregated to the account\u2013month level to create consistent longitudinal records of online discourse.\n\nTo track whether individuals were inside or outside the country over time, residency status was coded from publicly available sources such as news reports, official statements, and biographical materials. Where exact dates were uncertain, the earliest documented indication of living abroad was used to anchor timing. The monthly topic measure included here\u2014the share of tweets about foreign-led policy actions\u2014was built using dictionaries expanded from seed terms via word-embedding similarity trained on the study corpus, capturing Venezuela-specific vocabulary and multilingual usage. Monthly tweet counts were also recorded and log-transformed to summarize posting volume.\n\nThe observation window covers major developments in Venezuela\u2019s political landscape, including leadership transitions, contested elections, large protest cycles, and deepening economic and humanitarian crises. During these years, traditional media faced heavy constraints, and social media\u2014particularly Twitter\u2014served as a key venue for public communication, political messaging, and engagement with domestic and international audiences. Many opposition figures maintained active online presences throughout the period, and some relocated abroad; the panel structure allows their communication patterns to be followed continuously month by month. All data derive from public digital traces and public-domain biographical information, and the panel includes only months in which an account posted at least one tweet.","file_sha256":"3128de21a97807498f279bcb7159973e89621cc6bd4a98c8f319813649f925a8","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"3128de21a97807498f279bcb7159973e89621cc6bd4a98c8f319813649f925a8","transforms":[],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0009/data/data.csv"},"ref_first":{"columns":null,"description":"Each actor's first month with tweeted_exile == 1 (g)","file_sha256":"3128de21a97807498f279bcb7159973e89621cc6bd4a98c8f319813649f925a8","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"3128de21a97807498f279bcb7159973e89621cc6bd4a98c8f319813649f925a8","transforms":[{"args":{"columns":["actorid","months","tweeted_exile","perc_foreign_policy","log_tweets"]},"op":"select"},{"args":{"columns":["perc_foreign_policy","log_tweets","months"]},"op":"dropna"},{"args":{"query":"tweeted_exile == 1"},"op":"filter"},{"args":{"aggs":[{"column":"months","fn":"min","name":"g"}],"by":["actorid"]},"op":"collapse"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0009/data/data.csv"},"ref_obs":{"columns":null,"description":"Per actor over months 54 and 56: how many of the two months it is observed in, and its largest tweeted_exile there","file_sha256":"3128de21a97807498f279bcb7159973e89621cc6bd4a98c8f319813649f925a8","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"3128de21a97807498f279bcb7159973e89621cc6bd4a98c8f319813649f925a8","transforms":[{"args":{"columns":["actorid","months","tweeted_exile","perc_foreign_policy","log_tweets"]},"op":"select"},{"args":{"columns":["perc_foreign_policy","log_tweets","months"]},"op":"dropna"},{"args":{"query":"months in [54, 56]"},"op":"filter"},{"args":{"aggs":[{"column":"perc_foreign_policy","fn":"count","name":"n"},{"column":"tweeted_exile","fn":"max","name":"t"}],"by":["actorid","months"]},"op":"collapse"},{"args":{"aggs":[{"column":"n","fn":"count","name":"nboth"},{"column":"t","fn":"max","name":"tmax"}],"by":["actorid"]},"op":"collapse"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0009/data/data.csv"},"ref_panel":{"columns":null,"description":"Rows of months 54 and 56: the cohort first exiled at 55 and never or not-yet (after 56) exiled actors with no exile tweet in either month, each observed in both months","file_sha256":"3128de21a97807498f279bcb7159973e89621cc6bd4a98c8f319813649f925a8","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"3128de21a97807498f279bcb7159973e89621cc6bd4a98c8f319813649f925a8","transforms":[{"args":{"columns":["actorid","months","tweeted_exile","perc_foreign_policy","log_tweets"]},"op":"select"},{"args":{"columns":["perc_foreign_policy","log_tweets","months"]},"op":"dropna"},{"args":{"query":"months in [54, 56]"},"op":"filter"},{"args":{"how":"left","on":["actorid"],"with":"ref_first"},"op":"merge"},{"args":{"how":"left","on":["actorid"],"with":"ref_obs"},"op":"merge"},{"args":{"query":"nboth == 2 and (g == 55 or ((g != g or g > 56) and tmax == 0))"},"op":"filter"},{"args":{"args":{"cmp":"eq","column":"g","value":55},"name":"treated","op":"indicator"},"op":"derive"},{"args":{"args":{"column":"treated","default":0,"keep_unmapped":false,"mapping":[{"source":1,"target":1}]},"name":"treated","op":"map"},"op":"derive"},{"args":{"args":{"cmp":"eq","column":"months","value":56},"name":"post","op":"indicator"},"op":"derive"},{"args":{"args":{"columns":["treated","post"]},"name":"treated_post","op":"interaction"},"op":"derive"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0009/data/data.csv"}},"difficulty":"M","id":"crb_papers.q017","level":"T2","meta":{"agent_reach":{"blocked_by":["reference_frame:ref_first","reference_frame:ref_obs","reference_frame:ref_panel"],"dtypes":{"ds_0009":{"perc_foreign_policy":"float64","tweeted_exile":"int64"}},"named":{"outcome":["perc_foreign_policy"],"treatment":["tweeted_exile"]},"reachable":false,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":null,"unit_scale":1.0},"cluster":"ds_0009","crb":{"data_path":"data/ds_0009/data/data.csv","dataset_id":"ds_0009","gold":{"ci95":[-1.0028823886598044,3.5429155415073774],"ci_rule":"effect \u00b1 1.96\u00b7|standard_error| (evaluate_result.ci_bounds)","effect":1.2700165764237865,"standard_error":1.159642329124281},"identification":{"causal_quantity":"ATT","controls":["log_tweets"],"group_variable":"actorid","minimal_controlling_set":["log_tweets"],"outcomes":["perc_foreign_policy"],"reason_for_minimal_controlling_set":"\u201cX0 is the (log) number of tweets per month, accounting for variation in rates of tweeting.\u201d (Data and Methods \u2192 Empirical Strategy: Fixed Effects Models, p. 1368). The authors also note \u201cExile has no effect on tweet volume, and results hold when including these months (Appendix B1).\u201d (footnote to Empirical Strategy), indicating log tweet volume is not post\u2011treatment in this setting.","strategy":"Difference-in-Differences","time_variable":"months/month","treatments":["tweeted_exile"]},"identification_strategy":"Difference-in-Differences"},"headline_keys":{"infer.ols":["coef.treated_post"]},"naive":{"dataset":"ds_0009","treatment":"tweeted_exile","y":"perc_foreign_policy"},"native_id":"17","reference":{"cite":1,"note":"Reference path transcribed from the item's own estimation script.","steps":[{"args":{"cluster":"actorid","controls":["treated","post","log_tweets"],"dataset":"ref_panel","x":["treated_post"],"y":"perc_foreign_policy"},"tool":"infer.ols","why":"sm.OLS(y, [const, treated, post, treated_post, log_tweets]).fit(cov_type='cluster', groups=actorid) (CR1 errors with statsmodels' default G/(G-1)\u00b7(N-1)/(N-K) correction)"}]},"reference_path":{"calls":[{"args":{"cluster":"actorid","controls":["treated","post","log_tweets"],"dataset":"ref_panel","x":["treated_post"],"y":"perc_foreign_policy"},"tool":"infer.ols"}],"needs":[],"provenance":"benchmark_script","quotes":{},"reference_datasets":["ref_first","ref_obs","ref_panel"],"source_id":"crb_papers.q017.script"}},"question":"Among activists who go into exile in month index 55, how does their monthly share of tweets referencing foreign-led policy topics change in the month following exile relative to the month prior, compared to activists who remain in-country?","requires":[],"runner":null,"sources":["crb_papers.q017.gold","crb_papers.q017.script"],"split":"public","suite":"crb_papers","tags":["econ-bench","external","crb","did","ref_benchmark_script"],"target":"agent","tool":null,"truth":"CausalReasoningBenchmark gold at the pinned revision: causal_queries.json effect and standard_error (the value the item's estimation.py prints), scored as evaluate_result.py scores estimate_within_solution_ci"}