- Requires org + incentive changes
- More than technical discussion. Requires consideration and organizational intent
- Haha, it took 15 minutes for executives to say that this says nothing about whether or not this graph says anything about google analytics conversion. We would need to know these populations, run an RCT, etc.
- So if you lumped these together, you would see a +1 in usage, which would belie the reality that the machines do make people faster. So how do you go from -9 and -6 to +1?
You get that because the novices who are slower use the machine. And so your overall sample size is, for novices who are slower of 85%, they may be getting faster, but they are still overall slower. Compare that to if, let's just say the experienced technicians are 50%, but 90% of them are not using it.
Then what does that say? That says that you are taking the people who are comparatively slower in their new usage sample and the people that are getting somewhat faster in the experience group are only 10% of that total.
Orange = don’t use on the bottom and green = use
Organizational and incentive changes
See the Harrah’s slides
- Total rewards and incentivize origination, so you won’t hoard
- So the problem that they were trying to change involved Harrah's going up against the Mirage and Steve Wynn. And so they had to understand All right, if our customers engage in large amounts of cross-market play, But Harrah's only receives 36% of the total gaming dollars. How do we increase? We have to first identify the customers across the markets, determine what a potentially good customer is, and create incentives for those customers. In order to do that, we have to have data.
- Their structure was not set up like that. People didn't want to share data. They thought there was fiefdoms. And in order to do that, they restructured. So they gave a new organizational form, meaning the property management, The IT integration and behavior tracking centralize the data, and they align the incentives so that somebody didn't think of them as their client, but as Hera's client.
- And this enabled them to understand a woman like Sarah, who they previously may have saw as only spending $100 on the wheel. As a client that they could win back and that could help them to get from 36% to 42% when that 1% of improvement equated to $40 million. So how do we see Sarah as a new and incentivize her behavior? to capture more market share. Well, in order to do that, to start with, we have to be able to see that, and that's where the data comes in.
Data must be problem-driven —> So analytics must be PLANNED (Do we have the right people in the room)
Back-up and before you even gather data, what problem are you trying to solve? So your data without an intentional system… well it will tell you what the systems were set up to do. We might not know what that is, but we know it likely won’t be what we want answer
So before doing the overhaul the
The CEO's 50 questions should have been sat down and they should have thought through with the systems that we currently have, can we even answer these? And if we can't, and we can't cobble together the data to provide any insight, then let's ask, should we restructure and are these answers we want moving forward?
And rather than let's restructure so we can understand the data, it is what problems are we trying to solve? What questions do we want to answer? Then without investing a great deal of money, is there data available today that can help us with that? If there's not, then well, those are sunken costs and let's move on.
Who is and is not in the room
- Often, a BU problem sits in a silo and the ?s that VPs want to answer are too scoped and need to be considered in the broader context.
They used to model their model (on the right) w/ the model on the left
New Model and the terms
Exploratory replaces descriptive: Describe outcomes, understand how data was generated, and explain variability in the data will inform predictive and causal models.
Predictive stays (anticipate): Data that you have to predict an outcome you don’t know; e.g., sales probability, inventory levels, etc.
Causal replaces prescriptive: Using data you have or create to test an outcome you predict your action will influence. Examples: How a maintenance call improve uptime, impact of a sales call on conversion, proactive churn interventions
Prediction + Cause = Optimization
- If we know what to anticipate and believe we can connect that anticipation to change an outcome, then we should optimize a business problem/solution toward that end.
Balloon Effect 🎈
- Start w/ business priorities —> Business outcomes of interest and then ask how analytics can create value
Exploratory Analytics
Predictive Analytics
Analyst vs. data driven models: The advantage of analyst-driven models is that it is easy to explain your predictions. There is a clear mapping from the data to the prediction. For example, one can quickly learn how age is related to length of stay. The advantage of data-driven models is that they will provide the best prediction, but they are generally difficult to interpret. In some cases, it's impossible. Hence, if you care about why, you will tend to use analyst-driven models, but if you simply want to predict, a data-driven model can be superior.
Define the biz objective and then fit a business outcomes model to that objective such that you can align your data + EIE approach.