Does the world revolve around you? Modeling bias of only the data you can control

Psychologists say when things happen to us, good or bad, we often believe it is due to our actions.  We look to our choices and decisions and wonder how we can improve for the future.  We know for a fact that any result has a multitude of factors – of which one’s own actions are a small piece

This self-blame bias seems to exist in the modeling world as well.  In fact, we are taught to use this approach in most business schools that say marketing alone drives sales.   I am often asked by my clients “How can my sub-organization increase revenue?”.  It is all too frequent that they choose approach such as the one pictured where they do a simple model using only economic factors, pricing and marketing spend to predict revenue.  But yet – if you ask that same person what impacts revenue sales or brand value – most people will list a plethora of levers.  Take revenue for example – it’s influenced by pricing, sales headcount, marketing investment, customer service reputation, product offerings etc….  It’s a complete driver set.

Looking at the attached chart, logical people given the information above will conclude that the marketing campaign was a failure because it would assume that marketing dollars alone drive revenue and would not consider the other factors.   Marketing afterall did increase and revenue remained flat despite that increase.  Without a multivariate model, it is impossible to articulate what the impact of Sales reductions and sales efficiency reductions had on revenue –  and whether those losses were in fact mitigated by the marcom spend increases – as only one example…

Considering only the business levers that you control or that are easy to measure can lead you to the wrong conclusions.  When I first joined the pricing team, we considered only our price, our competitors’ prices, and of course unit sales on the product in determining if we should move a price.  The model would recommend price moves and we would approve them.  In one instance, it recommended a price decline based on the fact that unit sales had declined significantly.   The next week, the model again recommended a price reduction as unit sales had declined even more so.   This perked my interest and as I pulled up the product online, I discovered the product was out of stock on backorder.  People are less likely to purchase products on back order so unit sales often decline significantly regardless of the price compared to the market.   Being hyper-focused on pricing as the only driver for model sales had resulted in me pulling on the pricing lever when instead I should have focused on working with our procurement organization to get the product back in stock.  At the very least, stock levels needed to be considered in our pricing recommendation model.

In my work, I’ve built a model that considers not only marketing measurements, but other business levers and how those work together to tell the full story of revenue and/or brand value.  A modeler could just focus on metrics that are easy to gather – like econometrics, but when difficult ones such as product innovation and competitive performance are just as important to include.

Modeling holistically can help you isolate influences from the difficult to capture metric influences such as social media expenditures and brand campaigns.   It also allows you to understand the interrelationship of the differing variables, their relationship with revenue and allows you to isolate the impacts of one business lever (marketing spend) in relation to all the other levers that impact revenue.  Using more advanced neural network modeling can even provide opportunities to include a greater number of business levers and better explain the complexity of what drives revenue.

My team has built a model using advanced neural networks to explain the drivers of revenue and how the input variables interrelate with one output shown below.


When a model is built with more business levers included, instead of a direct regression model, you’ll find that some marketing variables – what I call direct contact variables like direct mail, coupons, email -will have less of an impact than in the direct regression model.    Other variables such as overall brand campaigns, corporate social responsibility investments, and social media metrics value will often show an increase in impact with the more advanced multivariate approach.   Companies that use a direct regression model are often undervaluing their brand campaigns and social media investments in favor of direct mail, email etc – the levers that are easily tied to a sale.

This even allows you to even consider tradeoff decisions in terms of which business levers you should invest in as part of a Financial Strategy conversation.

As modeling infrastructure expands to allow for more variables, models will become more robust and more explanative of all the impacts that drive revenue – resulting in better decisions for businesses!