His Intro
- Purchases are downstream of behavior and the behavior influences the repayment of loans.
Cobra
- Randomization + observational equivalence
- slicing
- Business problem + outcomes
Randomization Check
- Confound check: If you don’t have randomization, then you can’t isolate a variable. If all high income are in your test group, the high income could affect
- Large p-values are good when it comes to randomization.
- The high p value signals that it can’t distinguish from the two
- He says that biz vs stats significance is v. different and that the absolute percentage, so is something between a 1 and 5% difference between populations matter?
Slice
- Create segments;
- Randomize in the large population, then gamer represents measure and instrument everything. Matryoshka doll example
- Slicing shows us the base rate of control vs. test and it then shows the comparative life for each group.
Business analysis
- Expected spend per customer that I contact = Spend x response
- so $75 x #.##%
- If you offer a discount of $75 to $25 (1/3) then you need an increase of 3x (remember that 200% is 3x)
- understand the assumptions of our models: Lifetime contract value (if you change that, your multipliers)
- What are the assumptions underpinning how you think about lifetime contract value and
- A deeper question being we can’t disentangle the ad from the incentive it offered
- Set-up your scoreboard before the experiment is ran; simulate the experiment first so that you can look at the results together
Digital Advertising Effects
- ROAS = total Revenue / Spend on ad
- iROAS = incremental dollars that otherwise wouldn’t have happened
- how to get them to switch: Who in the network should you contact; how to enlist expertise; maybe chang ethe terminology? Reject hte premise
- Velvetta cheese example: It’s not just velvetta cheese… and they’re a part of Kraft… and so Kraft cheese and all Kraft in… So ask what’s in the revenue number
