Question

The residential division of Prism's high-speed Internet service uses one advertising agency while its commercial division uses another. Two analysts, Andy and Brad, are asked to test evaluate the effectiveness of the two agencies. Andy proposes an A/B test that compares the click-through rates per ad for the two agencies. Brad proposes a difference-in-difference test in which the budgets for both agencies are increased by 50% and the precent change in click-through rates are compared. What might be the source of selection bias for the two proposals? Which is likely to be smaller?

Answer 1