{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number traced to a tool call or the question","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":true,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"CRB estimate_within_solution_ci: the answer figure inside the gold 95% interval effect \u00b1 1.96\u00b7|SE| of source crb_papers.q077.gold (closed, evaluate_result.py; no --auto-rescale)","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":[0.12858109340881937,0.5011710651477705],"weight":1.0,"where":null},{"abs_tol":9.504846217830386e-08,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold effect to rel 1e-6 (abs floor 1e-6\u00b7SE); never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q077.gold","table":null,"tool":null,"value":0.314876079278295,"weight":1.0,"where":null},{"abs_tol":9.504846217830386e-08,"bind":"answer_call","column":null,"failure_class":null,"field":"se","interval":false,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"replication cell: the gold SE to rel 1e-6; never required","open_interval":false,"rel_tol":1e-06,"required":false,"source_id":"crb_papers.q077.gold","table":null,"tool":null,"value":0.09504846217830387,"weight":1.0,"where":null}],"datasets":{"ds_0047":{"columns":null,"description":"| Variable Name        | Description                                                                                                      |\n|----------------------|------------------------------------------------------------------------------------------------------------------|\n| opened | Email-platform indicator of the recorded image-display action used as a proxy for closely reading the assigned invitation: 1 if recorded, 0 otherwise. Absence of a recorded action does not establish that the email was unread. |\n| attended | Recorded event participation: 1 if the recipient collected a raffle ticket at the event entrance using the invitation's code; 0 otherwise. Names/email addresses and hand stamps supported attendance recording and prevented duplicate collection. |\n| intended | Recorded intention to participate: 1 for an Evite RSVP of \"yes\"; 0 otherwise, including recipients without a \"yes\" RSVP. |\n| esteem | Pooled recognition-message assignment: 1 for either the newsletter or Facebook invitation, 0 for the information-only invitation; equals newsletter + facebook for records with assignment data. |\n| female | Pre-experiment female indicator inferred from subscriber information/email addresses (1/0). |\n| newsletter | Assignment to the invitation offering to list attendees' names in the organization's newsletter so other members could celebrate their participation (1/0). |\n| facebook | Assignment to the invitation encouraging attendees to post event photos on the organization's Facebook page for other members to \"like\" (1/0). |\n| openednewsletter | Recorded image-display action for a newsletter-assigned recipient; equals opened \u00d7 newsletter for records with assignment data. |\n| openedfacebook | Recorded image-display action for a Facebook-assigned recipient; equals opened \u00d7 facebook for records with assignment data. |\n| openedesteem | Recorded image-display action for a recipient assigned either recognition-message variant; equals opened \u00d7 esteem, or openednewsletter + openedfacebook, for records with assignment data. |\n| control | Same information-only assignment indicator as information. |\n| information | Assignment to the information-only invitation containing the event's purpose and logistics without the additional promise of public recognition (1/0). |\n| fivemiles            | Indicator for residing within five miles of the event location                                                    |\n| yearshpccpreexp      | Number of years affiliated with the cooperating organization (duration of membership/engagement, in years)         |\n| fiveyears            | -                                                                                                                |\n| fouryears            | -                                                                                                                |\n| surveyrespondent     | Indicator for responding to the post-event (follow-up) survey                                                     |\n| reported_attended    | Self-reported attendance on the post-event survey                                                                 |\n| previous_attend      | Indicator for prior attendance at a coordinating organization pride event                                         |\n| groupwarmth          | Survey-scale measure of perceived warmth/trust toward the organization or group (self-reported)                   |\n| received_email | Email-receipt response recorded for post-event survey respondents (1/0); not the full-sample email-platform delivery log. |\n| used_website         | Indicator for using the organization's website to obtain information about the event                              |\n| received_other_email | Indicator for having received forwarded or other organizations' emails about the event                            |\n| printedmaterial      | Indicator for learning about the event via printed materials                                                      |\n| wordofmouth          | Indicator for learning about the event via word-of-mouth                                                         |\n| forwarded_hpcc       | -                                                                                                                |\n| otherlgbt_org        | Indicator for involvement in other LGBT organizations                                                             |\n| sports_team          | Indicator for membership/involvement with a sports team (organizational affiliation)                              |\n| professional_org     | Indicator for membership in a professional organization                                                           |\n| dramaclub            | Indicator for membership in a drama/theater club                                                                  |\n| religious_group      | Indicator for membership or affiliation with a religious group                                                    |\n| school_board         | Indicator for involvement with a school board or similar civic body                                               |\n| nonprofit            | Indicator for involvement with a nonprofit organization                                                           |\n| otherorg_none        | Indicator for reporting none of the listed organizational affiliations                                            |\n| attended_meeting     | Indicator for past attendance at community or organizational meetings                                             |\n| attended_protest     | Indicator for past attendance at protests                                                                         |\n| complained_govt      | Indicator for having complained to government officials (past civic action)                                       |\n| signed_petition      | Indicator for having signed petitions (past civic action)                                                         |\n| wrote_letter         | Indicator for having written letters to officials (past civic action)                                             |\n| financial_donation   | Indicator for having made financial donations (past civic action)                                                 |\n| vote_08              | Indicator for voting in the 2008 election                                                                          |\n| vote_09              | Indicator for voting in the 2009 election                                                                          |\n| vote_10              | Indicator for voting in the 2010 election                                                                          |\n| party                | Party identification (coded categorical variable)                                                                 |\n| politicalinterest    | Self-reported level of interest in politics (survey scale)                                                         |\n| age                  | Respondent age in years                                                                                            |\n| young                | Indicator for belonging to a younger age cohort (study-specific cutoff)                                            |\n| education            | Education level (coded categorical or ordinal)                                                                     |\n| transgendered        | Indicator for respondent identifying as transgender                                                               |\n| retired              | Indicator for respondent reporting retired status                                                                 |\n| military             | Indicator for prior or current military service                                                                   |\n| race                 | Race/ethnicity coded variable                                                                                     |\n| unemployed           | Indicator for reporting unemployed status                                                                         |\n| njresident           | Indicator for residing in New Jersey                                                                              |\n| forleadership        | Self-reported reason: attended to pursue or signal interest in a leadership position                               |\n| forimportant         | Self-reported reason: attended because it was an important event for the organizing group                          |\n| foradmiration        | Self-reported reason: attended because community members would think highly of attendees (esteem-related reason)    |\n| forcause             | Self-reported reason: attended to support a cause they care about                                                  |\n| forsocial            | Self-reported reason: attended for social reasons (camaraderie, meeting people)                                    |\n| fortime              | Self-reported reason: attended because timing/location made it easy to attend                                      |\n| forfun               | Self-reported reason: attended to have fun                                                                        |\n| democrat             | Indicator for identifying as or leaning toward the Democratic party                                                |\n\nData Description: The observations concern individual email-list subscribers of an LGBT advocacy and support organization in New Jersey around a rally and march in October 2011. After duplicate addresses and entries associated with businesses or organizations were removed, 3,651 individual email addresses were randomly assigned across three invitation variants. Invitations were sent one week before the event. Four undeliverable addresses were excluded, leaving 3,647 recipients with assignment and behavioral records: 1,217 in the newsletter variant, 1,213 in the Facebook variant, and 1,217 in the information-only variant.\n\nAll variants supplied the event's purpose, timing, and location. The newsletter variant additionally promised public recognition by listing attendees' names in an organizational newsletter. The Facebook variant invited attendees to post event photos that other group members could \"like.\" These two variants are pooled by esteem. Recipients in all three groups were also offered raffle tickets at the event entrance using a code in their invitation and were asked to RSVP through a corresponding Evite page.\n\nEmail-platform logs recorded an image-display action used as a proxy for closely reading an invitation. It is an imperfect measure: email-client settings and text-only viewing can prevent a reader from generating the recorded action. opened measures this action for any assigned variant; openedesteem measures it only for recipients assigned one of the two recognition-message variants. Message assignment and the recorded opening action are therefore different fields.\n\nThe RSVP outcome intended records an Evite \"yes,\" rather than attendance. Event attendance was recorded separately through raffle-ticket collection and identifying information. An online survey sent one week after the event collected self-reported attendance, motivations, information sources, and background characteristics. Survey measures are observed only for respondents and are distinct from the assignment, RSVP, opening-log, and attendance records.\n\nThe supplied CSV has 3,855 rows, but 208 have no assignment or behavioral data and contain only the fiveyears field set to zero; these are not additional documented experimental participants. The 3,647 rows with assignment data are the records described above.\n","file_sha256":"ed2268d2501c0a6ac20dbc95f11fa8bbf7f6bb2bcb0333889d8b2ffea1746282","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"ed2268d2501c0a6ac20dbc95f11fa8bbf7f6bb2bcb0333889d8b2ffea1746282","transforms":[],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0047/data/data.csv"},"ref_iv":{"columns":null,"description":"Complete rows of intended, openedesteem and esteem","file_sha256":"ed2268d2501c0a6ac20dbc95f11fa8bbf7f6bb2bcb0333889d8b2ffea1746282","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"redistributed by CausalReasoningBenchmark (MIT) under the original replication data's terms","member":null,"object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"reference","sha256":"ed2268d2501c0a6ac20dbc95f11fa8bbf7f6bb2bcb0333889d8b2ffea1746282","transforms":[{"args":{"columns":["intended","openedesteem","esteem"]},"op":"select"},{"args":{"columns":["intended","openedesteem","esteem"]},"op":"dropna"}],"url":"https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark/resolve/edd0fe1c1d1c5c0a4f0e3e67a50481ac679fb888/data/ds_0047/data/data.csv"}},"difficulty":"M","id":"crb_papers.q077","level":"T2","meta":{"agent_reach":{"blocked_by":["reference_frame:ref_iv"],"dtypes":{"ds_0047":{"intended":"float64","openedesteem":"float64"}},"named":{"outcome":["intended"],"treatment":["openedesteem"]},"reachable":false,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":null,"unit_scale":1.0},"cluster":"ds_0047","crb":{"data_path":"data/ds_0047/data/data.csv","dataset_id":"ds_0047","gold":{"ci95":[0.12858109340881937,0.5011710651477705],"ci_rule":"effect \u00b1 1.96\u00b7|standard_error| (evaluate_result.ci_bounds)","effect":0.314876079278295,"standard_error":0.09504846217830387},"identification":{"bad_controls":["opened","openednewsletter","openedfacebook","surveyrespondent","reported_attended","attended","used_website","received_other_email","printedmaterial","wordofmouth","forleadership","forimportant","foradmiration","forcause","forsocial","fortime","forfun","groupwarmth"],"causal_quantity":"LATE","instrument":["esteem"],"is_encouragement_design":true,"outcomes":["intended"],"reason_for_bad_controls":"These variables are post-treatment or downstream of the treatment openedesteem and thus constitute bad controls that can induce over-control or collider/selection bias. In particular: opened, openednewsletter, openedfacebook are post-treatment measures of email-opening behavior (mediators/colliders) and conditioning on them blocks or distorts the causal path from the instrument to the outcome. surveyrespondent is a post-event selection indicator; conditioning on survey response risks Heckman/collider bias. reported_attended and attended are downstream outcomes; controlling for them when the outcome is intended would condition on a later outcome, biasing effects. used_website, received_other_email, printedmaterial, wordofmouth, and the motivation items forleadership/forimportant/foradmiration/forcause/forsocial/fortime/forfun, as well as groupwarmth, were collected in the post-event survey and reflect post-treatment perceptions or information channels; adjusting for them conditions on variables affected by the treatment, inducing over-control or collider bias (see Chernozhukov et al., Deeper Dive into DAGs, Sections 11.3\u201311.4: post-treatment variables, mediators, and selection; figures 11.9\u201311.12).","strategy":"Instrumental Variable","treatments":["openedesteem"],"variant":"encouragement design (fuzzy)"},"identification_strategy":"Instrumental Variable"},"headline_keys":{"infer.iv_2sls":["coef.openedesteem"]},"naive":{"dataset":"ds_0047","treatment":"openedesteem","y":"intended"},"native_id":"77","reference":{"cite":1,"note":"Reference path transcribed from the item's own estimation script.","steps":[{"args":{"dataset":"ref_iv","endog":["openedesteem"],"exog":[],"instruments":["esteem"],"se":"classical","y":"intended"},"tool":"infer.iv_2sls","why":"2SLS by matrices with sigma2 = u'u/(n - k)"}]},"reference_path":{"calls":[{"args":{"dataset":"ref_iv","endog":["openedesteem"],"exog":[],"instruments":["esteem"],"se":"classical","y":"intended"},"tool":"infer.iv_2sls"}],"needs":[],"provenance":"benchmark_script","quotes":{},"reference_datasets":["ref_iv"],"source_id":"crb_papers.q077.script"}},"question":"What is the causal effect of closely reading/opening the esteem-promising email on the probability of RSVPing 'yes' to the event among recipients whose opening behavior was responsive to the email variant they received?","requires":[],"runner":null,"sources":["crb_papers.q077.gold","crb_papers.q077.script"],"split":"public","suite":"crb_papers","tags":["econ-bench","external","crb","iv","ref_benchmark_script"],"target":"agent","tool":null,"truth":"CausalReasoningBenchmark gold at the pinned revision: causal_queries.json effect and standard_error (the value the item's estimation.py prints), scored as evaluate_result.py scores estimate_within_solution_ci"}