Companies need better Cultures not Better Data

For the last decade and a half my line of work has been in data. As a way of earning your daily bread, it was mostly ok: data comes in, insights come out. The life of a mercenary statistician had a certain appeal—I thought that there were far worse jobs I could do. For instance, I work in the food space. I don’t grind babies to dust so that I can eat. You could say that the work was morally neutral, but I’d dispute that. Delivering groceries direct to customers cuts the total carbon in the entire supply chain by half. It felt like I was part of something bigger, though typing it out now it’s apparent how needlessly technocratic my motivations were. It wasn’t to help people as people qua people, but to minimise some abstract number.

The passage of the previous paragraph mirrors my experience: from a simple statement of my expectations (‘I turn data into rent money’) through to disillusionment with the ethical basis of what I was doing. I wish that the journey had occurred due to my own enlightenment—that there was some event which totally altered my perception of my work. But there was no moment where it became apparent I had misjudged my vocation, instead I just got bored and frustrated with a few projects.

You see, I was a specialist in A/B testing. What that means is that I helped analysts and product owners at my company design and run experiments on our customer base, mostly in the hope that we can sell people more of our product. One of the truisms of A/B testing is that over 90% of your experiments do nothing. Which means that your current business model really is as good as it gets. The other truism is that experiments are radically uncertain, and all practitioners have to cultivate a level of epistemic openness regarding this. If you truly knew the outcome of an experiment before you ran it, then why did you need to run it?

Alas in the age of machine learning, everything is up for grabs. This includes building ML models which can predict the outcome of experiments. Ok, that was a bit loose. What was claimed about the ML model is that it could predict the outcome metric of an experiment with a ceratin accuracy. As a predictive model it was really good. If we had been using these models to predict the outcome metric they would be fine models. But the idea was to predict the difference in the outcome metric between the test group and the control group in an experiment. This won’t be a good prediction of the actual effect of the intervention because, if the intervention actually has an effect, the test group will be systematically different from the control group in a way which isn’t captured in the training data. Technically, there is a covariate shift, and your model won’t predict well. In other words, you can’t predict the outcome of an experiment. If you want to know what happens you need to run the experiment.

Anyway, this idea consumed three years of my life. Once a powerful VP was convinced that they could predictably shorten experiment run times, then there was no hope of opposing it. It didn’t matter that there were no papers on the subject. It didn’t matter that we’d need 50% more subjects in the experiment to account for the extra uncertainty induced by the ML model. It didn’t matter that it made experiments more difficult to interpret (what do you do if the predictive metric and the underlying metric disagree?) We needed to predict the outcome of experiments before they had concluded.

All this is to say: problems in organisations are political, not technical. They need a political solution. My problem was that powerful executives deemed the time it took to complete an experiment as too long. The business needed to move faster. There were many technical people who saw it as their job to say ‘yes’ to this impulse, and systematically ignore anything which got in the way. Which is to say, companies don’t need better data or engineering. They need better cultures.