Module 4

• Experiments from planned designs, design data, and causal about changing outcomes. Experiments and comparison. All analytics rest on comparison.

1. Is this the most important business outcome?
2. How will we know? What are the pre-test outcome metrics
3. Experimental units (they shouldn’t communicate, buffer between control and experiment)
4. Statistically valid, “power calculation”; how much am i willing to invest to understand?

What’s the purpose of our experiment?

• Success / efficacy in testing required re-designing the site

• Testing the test required education on the new scoring process, which did not slow down the process but quickly conveyed info.

• The experiment on the left violates probabilistic equivalence because once you receive name and address from the two groups, you now know something about those two groups: One is interested in RED and the other has a low bar for giving up PII for money