{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number traced to a tool call or the question","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":true,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"CRB estimate_within_solution_ci: the answer figure inside the gold 95% interval effect \u00b1 1.96\u00b7|SE| of source crb_papers.q041.gold (closed, evaluate_result.py; no --auto-rescale)","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":[-0.011605549900999091,0.0457113543604841],"weight":1.0,"where":null},{"abs_tol":1.4621659250378366e-08,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold effect to rel 1e-6 (abs floor 1e-6\u00b7SE); never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q041.gold","table":null,"tool":null,"value":0.017052902229742505,"weight":1.0,"where":null},{"abs_tol":1.4621659250378366e-08,"bind":"answer_call","column":null,"failure_class":null,"field":"se","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold SE to rel 1e-6; never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q041.gold","table":null,"tool":null,"value":0.014621659250378366,"weight":1.0,"where":null}],"datasets":{"ds_0021":{"columns":null,"description":"| Variable Name | Description |\n| --- | --- |\n| year_f | Four-digit calendar year covered by the company report (e.g., 2007\u20132017). |\n| company | Numeric company identifier originating from external listings used to build the sampling frame (e.g., Orbis or stock exchange records). Stable within firm across years but not necessarily consecutive. |\n| quota | Policy-period indicator: equals 1 for Italian firm-years from 2011 onward (the year a national law regulating the gender composition of boards was enacted); equals 0 for Italian firm-years before 2011 and for all Greek firm-years in 2007\u20132017. Reflects the adoption date, not firm-specific compliance dates. |\n| sustain | Report type flag: 1 if the document is a sustainability/social responsibility report; 0 if it is an annual/financial report. When both report types existed for a firm-year, the sustainability report was retained. |\n| pct_rev_change | Year-over-year percentage change in the firm\u2019s year-end revenue, computed as 100 \u00d7 [(Revenue_t \u2212 Revenue_{t\u22121}) / Revenue_{t\u22121}]. May be missing where consecutive-year revenue figures were unavailable. |\n| prop_attn | Proportion of a report\u2019s text devoted to gender-equality themes, measured as the share of tokens matching a curated dictionary (e.g., leadership representation, pay gaps, family-care policies, discrimination/harassment) divided by total tokens. Values range from 0 to 1. Texts were standardized to English (machine translation used when needed) before tokenization. |\n| companyid | Study-assigned numeric identifier for each company, providing a consistent panel unit across years (distinct from the source \u201ccompany\u201d code). |\n| year | Integer time index for analysis (1 = 2007, \u2026, 11 = 2017), aligned to the study period and uniform across firms. |\n\nDataset Context:\nThis firm-year panel assembles corporate reporting from large publicly listed companies in Italy and Greece during 2007\u20132017, a period that spans a major corporate governance reform in Italy affecting the gender composition of boards (enacted in 2011 with a phased compliance schedule and sanctions). Greece did not introduce an equivalent requirement for private listed firms within this window. The two-country setting provides multiple years of corporate disclosures before and after the reform\u2019s introduction in Italy.\n\nSampling began with high-revenue listed firms on the Italian and Greek exchanges, identified through commercial listings. For each firm-year, publicly posted annual reports and sustainability/social responsibility reports were downloaded from company websites. When both report types were available, the sustainability report was retained because it typically includes more detail on workforce practices and social initiatives. This process yielded 962 reports for 96 firms (52 in Italy and 44 in Greece), with most companies observed repeatedly over the 11-year span.\n\nPDFs were converted to text and, where necessary, machine-translated to English to maintain a consistent analysis language. A dictionary-based approach quantified how much of each report focused on workplace gender-equality topics by counting dictionary token matches and dividing by the total number of tokens in the document. The dictionary was built from relevant policy and workplace terminology and includes terms related to leadership representation, pay equity, family-care and work\u2013life balance provisions, and discrimination/harassment. Because report length and formatting vary, the measure is constructed as a proportion to facilitate comparisons across documents and years.\n\nFinancial information extracted from reports or public filings was used to compute year-over-year revenue changes. Identifiers enable linkage across years: companyid provides a stable panel key, year is an internal time index, and year_f records the calendar year. Users should be aware that measurement can be influenced by factors such as document length, translation quality, and differences between annual and sustainability reporting formats; the sustain flag indicates report type to help account for such variation. Overall, the dataset offers a structured view of how major listed firms in two Southern European economies communicated about workplace gender-equality topics across a decade that included a salient corporate governance reform in one country.","file_sha256":"e7a25901cae1d1a00a86567bbc1c285ed2ab23166c1bfdc4423b3134e05ea474","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"e7a25901cae1d1a00a86567bbc1c285ed2ab23166c1bfdc4423b3134e05ea474","transforms":[],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0021/data/data.csv"},"ref_ever":{"columns":null,"description":"Each company's max of quota over its complete rows (ever_treated)","file_sha256":"e7a25901cae1d1a00a86567bbc1c285ed2ab23166c1bfdc4423b3134e05ea474","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"e7a25901cae1d1a00a86567bbc1c285ed2ab23166c1bfdc4423b3134e05ea474","transforms":[{"args":{"columns":["quota","prop_attn","companyid","sustain","pct_rev_change","year_f"]},"op":"select"},{"args":{"map":{"year_f":"year"}},"op":"rename"},{"args":{"columns":["quota","prop_attn","companyid","sustain","pct_rev_change","year"]},"op":"dropna"},{"args":{"aggs":[{"column":"quota","fn":"max","name":"ever_treated"}],"by":["companyid"]},"op":"collapse"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0021/data/data.csv"},"ref_panel":{"columns":null,"description":"Complete company rows of 2010 and 2012","file_sha256":"e7a25901cae1d1a00a86567bbc1c285ed2ab23166c1bfdc4423b3134e05ea474","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"e7a25901cae1d1a00a86567bbc1c285ed2ab23166c1bfdc4423b3134e05ea474","transforms":[{"args":{"columns":["quota","prop_attn","companyid","sustain","pct_rev_change","year_f"]},"op":"select"},{"args":{"map":{"year_f":"year"}},"op":"rename"},{"args":{"columns":["quota","prop_attn","companyid","sustain","pct_rev_change","year"]},"op":"dropna"},{"args":{"how":"inner","on":["companyid"],"with":"ref_ever"},"op":"merge"},{"args":{"query":"year in [2010, 2012]"},"op":"filter"},{"args":{"args":{"cmp":"eq","column":"year","value":2012},"name":"post","op":"indicator"},"op":"derive"},{"args":{"args":{"columns":["ever_treated","post"]},"name":"post_ever","op":"interaction"},"op":"derive"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0021/data/data.csv"}},"difficulty":"M","id":"crb_papers.q041","level":"T2","meta":{"agent_reach":{"blocked_by":["reference_frame:ref_ever","reference_frame:ref_panel"],"dtypes":{"ds_0021":{"prop_attn":"float64","quota":"float64"}},"named":{"outcome":["prop_attn"],"treatment":["quota"]},"reachable":false,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":null,"unit_scale":1.0},"cluster":"ds_0021","crb":{"data_path":"data/ds_0021/data/data.csv","dataset_id":"ds_0021","gold":{"ci95":[-0.011605549900999091,0.0457113543604841],"ci_rule":"effect \u00b1 1.96\u00b7|standard_error| (evaluate_result.ci_bounds)","effect":0.017052902229742505,"standard_error":0.014621659250378366},"identification":{"causal_quantity":"ATT","controls":["sustain"],"group_variable":"companyid","minimal_controlling_set":["sustain"],"outcomes":["prop_attn"],"reason_for_minimal_controlling_set":"Among provided covariates, sustain (report type) is the minimal adjustment that addresses systematic differences in how the outcome is measured across report types. The authors note that \u201csustainability reports are more likely to include in-depth discussions of employee-related matters. We control for this in our statistical analysis\u201d (Data section, p. 612). Identification in their DiD relies on parallel trends, not on time-varying financial controls; they also show robustness \u201cincluding models with no controls\u2026 Our results are robust to these analyses\u201d (Results, p. 616).","strategy":"Difference-in-Differences","time_variable":"year/year_f","treatments":["quota"]},"identification_strategy":"Difference-in-Differences"},"headline_keys":{"infer.ols":["coef.post_ever"]},"naive":{"dataset":"ds_0021","treatment":"quota","y":"prop_attn"},"native_id":"41","reference":{"cite":1,"note":"Reference path transcribed from the item's own estimation script.","steps":[{"args":{"cluster":"companyid","controls":["post","ever_treated","sustain","pct_rev_change"],"dataset":"ref_panel","x":["post_ever"],"y":"prop_attn"},"tool":"infer.ols","why":"smf.ols('prop_attn ~ post*ever_treated + sustain + pct_rev_change').fit(cov_type='cluster', groups=companyid) (CR1 errors with statsmodels' default G/(G-1)\u00b7(N-1)/(N-K) correction)"}]},"reference_path":{"calls":[{"args":{"cluster":"companyid","controls":["post","ever_treated","sustain","pct_rev_change"],"dataset":"ref_panel","x":["post_ever"],"y":"prop_attn"},"tool":"infer.ols"}],"needs":[],"provenance":"benchmark_script","quotes":{},"reference_datasets":["ref_ever","ref_panel"],"source_id":"crb_papers.q041.script"}},"question":"By how much did the proportion of report text devoted to gender-equality topics increase in Italian company-years after the 2011 quota versus Greek company-years, using 2012 as the post period and 2010 as the baseline?","requires":[],"runner":null,"sources":["crb_papers.q041.gold","crb_papers.q041.script"],"split":"public","suite":"crb_papers","tags":["econ-bench","external","crb","did","ref_benchmark_script"],"target":"agent","tool":null,"truth":"CausalReasoningBenchmark gold at the pinned revision: causal_queries.json effect and standard_error (the value the item's estimation.py prints), scored as evaluate_result.py scores estimate_within_solution_ci"}