{"args":{},"checks":[{"abs_tol":null,"bind":"any","column":null,"failure_class":null,"field":"value","interval":false,"key":null,"kind":"answer_backed","min_lre":null,"note":"every number of the answer is stated by a tool call","open_interval":false,"rel_tol":null,"required":true,"source_id":null,"table":null,"tool":null,"value":null,"weight":1.0,"where":null},{"abs_tol":null,"bind":"answer_call","column":null,"failure_class":null,"field":"value","interval":true,"key":null,"kind":"answer_figure_close","min_lre":null,"note":"QRData eval.py: strictly inside gold \u00b1 3% (a '%' gold divided by 100), on the answer figure after meta.answer_format","open_interval":true,"rel_tol":null,"required":true,"source_id":"qrdata.gold.356","table":null,"tool":null,"value":[4.2970999999999995,4.5629],"weight":1.0,"where":null}],"datasets":{"collections_email":{"columns":null,"description":"In a study to determine the causal effect of sending an email reminder on the repayment of debts, a fintech company conducted a random test involving 5000 customers who were late on their payments. Each customer was randomly assigned to either receive an email about negotiating their debt or to be part of a control group that did not receive the email. Data was collected on the amounts paid by the late customers after this intervention. The dataset collections_email.csv contains variables including the amount paid (`payments`), whether the email was sent (`email`), whether the email was opened (`opened`), whether the customer contacted the collections department to negotiate their debt after having received the email (`agreement`), the customer's credit limit before being late (`credit_limit`), and the customer's risk score prior to the delivery of the email (`risk_score`).","file_sha256":"3c3a97cca6fdd96f6856f7856b5f8bc786a31d1a9be78c9f610ef3070783e970","format":"csv","frame_sha256":null,"generator":null,"kind":"file","licence":"CC BY-NC 4.0","member":"data/collections_email.csv","object":null,"params":{},"path":null,"read_kw":{},"records":null,"role":"agent","sha256":"3c3a97cca6fdd96f6856f7856b5f8bc786a31d1a9be78c9f610ef3070783e970","transforms":[],"url":"https://raw.githubusercontent.com/xxxiaol/QRData/de450af45ff7101b328bb064c6b475f73414a7ed/benchmark/data.zip"}},"difficulty":"M","id":"qrdata_socsci.356","level":"T2","meta":{"agent_reach":{"blocked_by":[],"dtypes":{"collections_email":{"credit_limit":"float64","email":"int64","payments":"int64","risk_score":"float64"}},"named":{"outcome":["payments"],"treatment":["email"]},"reachable":true,"rule":"magi_bench.reach/1"},"answer_format":{"figure_transform":"identity","round_to":2,"unit_scale":1.0},"answer_format_from":{"figure_transform":null,"round_to":"nearest hundredth","unit_scale":"1.0: an estimator's figure is on the gold's own scale"},"cluster":"collections_email.csv","expected_tool":"infer.ols","family":"numerical","headline_keys":{"infer.ols":["coef.email"]},"naive":{"dataset":"collections_email","treatment":"email","y":"payments"},"native":{"answer":"4.43","keywords":["Causality","Randomized experiment","Regression analysis","Average treatment effect","Confounder selection"],"multiple_choices":null,"question_type":"numerical","reference":"Causal Inference for the Brave and True 7"},"native_id":356,"reference":{"cite":1,"note":"Caminho de refer\u00eancia inferido \u00e0 m\u00e3o.","steps":[{"args":{"controls":["credit_limit","risk_score"],"dataset":"collections_email","x":["email"],"y":"payments"},"tool":"infer.ols","why":"OLS of payments on the randomised email with the pre-treatment covariates that predict the outcome; opened and agreement are post-treatment and left out."}]},"reference_path":{"calls":[{"args":{"controls":["credit_limit","risk_score"],"dataset":"collections_email","x":["email"],"y":"payments"},"tool":"infer.ols"}],"needs":[],"provenance":"hand_inferred","quotes":{},"reference_datasets":[],"source_id":null},"reference_why":"OLS of payments on the randomised email with the pre-treatment covariates that predict the outcome; opened and agreement are post-treatment and left out."},"question":"What is the average treatment effect (ATE) on payments from sending the email reminder to late-paying customers? Please choose the variables to adjust for, conduct linear regression, and round your answer to the nearest hundredth.","requires":[],"runner":null,"sources":["qrdata.gold.356","qrdata.eval_py"],"split":"public","suite":"qrdata_socsci","tags":["econ-bench","external","qrdata","brave_and_true","numerical","reference_path","agent_reachable","naive"],"target":"agent","tool":null,"truth":"QRData gold: benchmark/QRData.json item 356 at de450af45ff7101b328bb064c6b475f73414a7ed"}