How to be more successful with analytics

Hasn’t everyone these days already heard about how analytics has improved a business; making it more nimble, more efficient, more able to customize to consumer needs?  So, why are there still unsuccessful analytics projects?  In large part, analytics has definitely piqued most everyone’s interest. 3904325807_d64fcbf5dc_oThe majority of businesses want analytics as part of their decision making.  The problem that arises is that the nontechnical business decision maker doesn’t always have the vision — or the background — to dream up a role where analytics can participate… or gives analytics too large of a role.  The analysts at first will not fully understand the business needs or problems that the company is struggling with, and how to translate a model into a recommendation for that business decision.  In addition, the analysts may not know how to communicate the limitations of their analysis or struggles of their role.

As I look across my career in analytics, I start to see some trends between the projects I THOUGHT would be successful and the ones that actually were successful and it left me with some thoughts on what makes an analytics project successful.

Meet a strategic business need:  There are so many projects that envisioned that they would forever change the company.  The problem is, as an analyst,  we are rarely in control of those decisions.   The “build it and they will come” strategy has not traditionally worked well for me.   Instead, I get a project where I didn’t have funding for it and the business is disinterested in my really cool model.   I recommend instead finding a buyer BEFORE you build it.  Find someone who is energetic and interested in using analytics to help make their decisions, and make sure that what you’re building is something that could have an impact on THEIR success and is also in line with the company’s overall goals.  This is easy to say, but sometimes so hard to do…  especially when you’re the analyst, and you see a potential project that could be so cool, but the decision makers just aren’t yet open to allowing the change, or it isn’t in line with where the company is going.  Those are the projects you should bypass, and instead go after the (sometimes less sexy) ones that will get more investment in terms of resources and business attention.

Explain analytics in a nontechnical way:  If the analysis cannot be explained or you don’t have the capability on your team to help the business decision makers understand HOW your analysis can impact their decision, acceptance will be rare, and very difficult.  This is especially true as you get into more advanced modeling or more strategic (aka bigger $$ impact) decisions.  I’ve spent much of my career coming up with creative ways to explain advanced modeling to business people.  It can be a great deal of fun too.  For example, I’ve explained elasticity as the different between adding flour versus pepper to a soup.  One has a much bigger impact to the taste.   I still believe this is very important, but as analytics plays more of a life in everyday consumer lives, this is also easier.  I can always talk about the everyday analytics I’ve used myself – “It’s like when Netflix knows what movies you want…”  or “It’s like UPS estimating when your package will arrive.”

Demonstrate ROI:  The very last thing that a project must do is actually impact ROI.  I’ve had several projects that, while able to increase predictability of an outcome, were 17123251389_9f5db13fec_onot able to do it with enough of an increase over the regular business decision making to warrant an investment in the model.  For example, I completed a project to attempt to identify customers who were likely to be detractors in terms of Net Promoter Score (NPS), with the idea that the team would reach out to them and attempt to correct their concerns proactively.  The model, while successful — able to slightly increase the attitude of likely detractors — did not provide enough of an additional gain versus looking at three main variables that the team was already using. So, the project was discontinued.  Being realistic about the actual impact of what your team’s modeling could accomplish will help you select the work that will have the greatest impact.

So how can you as an analyst become more successful?  Here’s a few thoughts…

Walk a mile in their shoes:  First way for an analyst to be successful at a company is to understand the decision well.  How is it made today?  Why has it been decided in certain ways in the past?  What does the business decision maker consider when 4394284246_5028b65d39_omaking the decision?   The analyst needs to think of the pressures on the business decision maker.  What are their goals?  How can data help make them successful?  What would the business decision maker like to tell his or her boss or board?  If the analyst is able to think of the world from the business decision maker’s perspective, the analyst will be much more successful at generating analyses that will actually be used, and those successes will generate trust and further funding for more advanced analyses.

Run presentation past a nontechnical supporter:  Nothing will get someone without a background in analytics to roll their eyes as much as discussing p values or detailed modeling approaches.   You need to tailor your presentation for the skillset that your business decision maker has.  They want to trust you, and believe that you have done the right hold out value and made the best assumptions.  I’ve coached many team members to remove the two pages worth of assumptions from their deck.  Highlight one or two of the large ones if appropriate, but you aren’t meeting in your stats class now.  This is business and explaining what you’ve done and why you’ve done it in an easy to comprehend way is vastly more important.  How do I do this?  I think of trying to explain my presentation to my ten year old niece.  How would she respond to my explanation of what I’m doing?  Or perhaps to my mother… (assuming your mom isn’t a stats professor of course.)  Then if possible, I run my presentation past someone outside of analytics and get their feedback and see what questions they raise.  Often when you are dealing with the details day in and day out, one might not even realize that you are using jargon that others might not understand. Doesn’t everybody know what segmentation is? Actually, no.

Create proof points for your techniques:   Why should the business decision maker use your new model?  In the world of business, showing the strength of your model alone might not be enough.  Are other people in the company using a similar model to impact a different decision?  I used this approach explaining how a model around company investment used a very widely accepted marketing investment approach.  Are there other executives in the company that are already using models that you have built?  Your own reputation is a great way to help build trust in your model.  If one executive says to another – “I’m not sure how her team does it, but they doubled our impact with a model they built.”  that can greatly assist with your growth. Do they have trusted technical team members? Finding their trusted analytics person and meeting with them to explain the details of your analytics in great detail and convincing them to accept the approach can make the official business acceptance a mere formality.  Even better, you can incorporate their respected analytics person into the design process — they likely understand the business problems better than you and will make your first version even stronger.   If you don’t have any of those, but are using a modeling approach that has been used in the past in other industries to assist with a problem, and your approach is similar, highlighting that! Still too cutting edge in your approach or these proof points aren’t enough: Can it be tested on a small subset?  If so, what is the business decision maker risking? A dubious decision maker is much more likely to greenlight a limited test than a full-scale roll-out.

All in all – Analytics is a growing field and there are more than enough ideas of where to make improvements to the model.  It’s an exciting place to be and with a few small changes in prioritizing and implementation, you can be much more impactful as an analyst.

Revolutionizing decision making with analytics

Big data will change the way that business is done forever.  It will revolutionize marketing, product management, shipping, pricing, and even customer service.  The sheer amount of data coming in will provide companies with more predictability and customization power than ever before.

Where did big data start?  Big data started with the internet and the tracking of people on the internet through web analytics.  Web analysts were the trailblazers of big data and the first to start to explore how to use the power of data to predicting people’s behavior, desires and needs.

I’ve started to notice in my career a similar progression for any business decision as it slowly flows from a non-analytical business owner making ‘gut calls’ to completely data-driven decision making.  This transformation is widespread, occurring everywhere from marketing investment decisions to HR recruitment analytics to predicting IT system failures.   For the purposes of simplicity, I’m going to talk about marketing campaign decisions as an example — but the process seems to hold true no matter the context.

Think of the data available to marketing analytics back in the 1960s.  If you’ve watched Mad Men, you know how much gut instinct went into decision making.   In one episode, an analyst gets dismissed by Don Draper as knowing very little and being of almost no help.  Those gurus, who may not have even been able to articulate why they were making decisions, were the trusted leaders.  But, suddenly, relevant data becomes available and people begin to dream what this data could provide.  The first step is reporting on trends.  As the trend reports come out, the business starts to see connections between some trends and business performance.  This then leads to Key unnamed (2)Performance Indicator reports where a metric or set of metrics has gained business acceptance.  At this stage in the analytics evolution of a business, a report that referrals are down this quarter might cause concern that sales will miss their target.  Why are they concerned?  Because there is a belief that the metric is connected with performance.  This is an important and necessary step to move into the next phase.

Business Intelligence or Data Exploration is where you start to really dive into the data and combine it with the knowhow of those who have been doing the work for 20 years — the experts of the company who have an instinctive feel for why one campaign will work and another will fail.  This stage can also be critical to helping the analyst understand and dissect some of the decision making and for the current experts to start trusting the analyst team to understand of the intricacies of the business.  Sometimes this phase is skipped… but in my experience, this can be extremely detrimental to the progression of analytic decision making in a company, resulting in a more adversarial relationship, with analytics fighting traditional business decision makers and the experts they’ve relied on in the past.

The most exciting phase for most analysts is once the business is ready to not just use data to figure out what’s going on now, but to also use it to ask ‘what if?’: predictive modeling.  This is the phase where most statistics majors get excited – where advanced modeling can be created to predict which campaign will perform best, which IT systems are likely to fail, which patient is most likely to develop diabetes in the next year.  As these models continue to be developed and prove their worth, the next phase comes into play: the data is collected real-time and decisions can be made automatically in  response with analysts only monitoring the system.  New data continues to come in, and results in an iterative approach.  As new or more detailed data sources come in, we still go through the phases (Finding Trends, KPIs, Data Extraction, Modeling), but the integration of the new data goes faster.

At each point in these phases, the acceptance of the analytics from the business is an important part of progression to the next level.  This has become easier as other companies and industries have gained public renown for their use of analytics, and provided pressure to follow suit… but still there can be resistance.

unnamed (1)Ownership of the business decision making also follows a phased progression eventually ending with the analytics team taking over ownership of the business decision. Let’s take a brief walk down history lane. Pricing analytics actually first started in the airline industry.  The idea of pricing based on plane fill rate and timing came up and some executive was willing to give it a test — the first phase of the movement towards analytics under ownership of the decision.  Assuming the test is successful — as it most definitely was in the airline business — the business decision maker is willing to pass along more control and suggests developing across all markets or lines of businesses, but isn’t ready to hand over the reins, and wants to stay in the loop as a decision approver.  Eventually, even this approval effort becomes a formality as the analytics increases in sophistication and the third phase is reached where the analyst has the approval and is expected to review all decisions made by the model.  Eventually the speed and quantity of incoming data becomes great enough, and model sophistication becomes strong enough, that no human intervention is required in the business decision, only monitoring to improve the models.

unnamedAs an exception that proves the rule, here’s an example of where algorithms designed on models to set pricing resulted in an out of print book soaring to $2 M dollars before the analysts noted it and modified their pricing model.  Quite fun!  It shows the exciting and somewhat silly decisions that we have to look forward to as more of our day to day decisions become machine and model driven.

I think it also shows the importance of having analysis define reasonable upper and lower bounds for the decisions that come out of their model — it’s the lack of these that cause things like flash crashes in the stock market.

Eventually there will be complete integration of the business decision making and the analytics.  We are already starting to see this in several areas including pricing and digital marketing.  Can you imagine someone being in digital marketing that doesn’t understand page views and referrals and feels comfortable looking at numbers and predicting future impacts? I cannot.  This shows how integrated analytics has become with business decision making in the digital marketing space.  I believe in the next 5 years, this blurring of analytics and business decision making will continue to take over other areas as well.  It’s a great time to be in analytics!

Guest-Blogging at the 2013 INFORMS Conference on Business Analytics & Operations Research

I’ve been invited to guest-blog during the 2013 INFORMS Conference on Business Analytics & Operations Research in San Antonio this Monday and Tuesday!

During the conference, I (as well as a number of other wonderful people) will be blogging at .  Come and check us out — and if you’re reading this and are going to the conference, drop me a line and let me know!

Conference Presentations

I’ve presented at several conferences recently and wanted to post my slides here in case anyone at the presentations wanted to see them.

Social Media ROI Measurement: The Good, the Bad & the Ugly – eMetrics Confererence 2013, Toronto

The Elusive Brand – How to Measure Brand and the Communications Focused on It – Predictive Analytics World Conference 2013, Toronto

Strategic Case Study: Investment Optimization for Executives using Big Data – Data Day Texas 2013, Austin

If you attended one of the presentations, I’d love to hear what you thought — just comment here or email me!

Does the world revolve around you? Modeling bias of only the data you can control

Psychologists say when things happen to us, good or bad, we often believe it is due to our actions.  We look to our choices and decisions and wonder how we can improve for the future.  We know for a fact that any result has a multitude of factors – of which one’s own actions are a small piece

This self-blame bias seems to exist in the modeling world as well.  In fact, we are taught to use this approach in most business schools that say marketing alone drives sales.   I am often asked by my clients “How can my sub-organization increase revenue?”.  It is all too frequent that they choose approach such as the one pictured where they do a simple model using only economic factors, pricing and marketing spend to predict revenue.  But yet – if you ask that same person what impacts revenue sales or brand value – most people will list a plethora of levers.  Take revenue for example – it’s influenced by pricing, sales headcount, marketing investment, customer service reputation, product offerings etc….  It’s a complete driver set.

Looking at the attached chart, logical people given the information above will conclude that the marketing campaign was a failure because it would assume that marketing dollars alone drive revenue and would not consider the other factors.   Marketing afterall did increase and revenue remained flat despite that increase.  Without a multivariate model, it is impossible to articulate what the impact of Sales reductions and sales efficiency reductions had on revenue –  and whether those losses were in fact mitigated by the marcom spend increases – as only one example…

Considering only the business levers that you control or that are easy to measure can lead you to the wrong conclusions.  When I first joined the pricing team, we considered only our price, our competitors’ prices, and of course unit sales on the product in determining if we should move a price.  The model would recommend price moves and we would approve them.  In one instance, it recommended a price decline based on the fact that unit sales had declined significantly.   The next week, the model again recommended a price reduction as unit sales had declined even more so.   This perked my interest and as I pulled up the product online, I discovered the product was out of stock on backorder.  People are less likely to purchase products on back order so unit sales often decline significantly regardless of the price compared to the market.   Being hyper-focused on pricing as the only driver for model sales had resulted in me pulling on the pricing lever when instead I should have focused on working with our procurement organization to get the product back in stock.  At the very least, stock levels needed to be considered in our pricing recommendation model.

In my work, I’ve built a model that considers not only marketing measurements, but other business levers and how those work together to tell the full story of revenue and/or brand value.  A modeler could just focus on metrics that are easy to gather – like econometrics, but when difficult ones such as product innovation and competitive performance are just as important to include.

Modeling holistically can help you isolate influences from the difficult to capture metric influences such as social media expenditures and brand campaigns.   It also allows you to understand the interrelationship of the differing variables, their relationship with revenue and allows you to isolate the impacts of one business lever (marketing spend) in relation to all the other levers that impact revenue.  Using more advanced neural network modeling can even provide opportunities to include a greater number of business levers and better explain the complexity of what drives revenue.

My team has built a model using advanced neural networks to explain the drivers of revenue and how the input variables interrelate with one output shown below.


When a model is built with more business levers included, instead of a direct regression model, you’ll find that some marketing variables – what I call direct contact variables like direct mail, coupons, email -will have less of an impact than in the direct regression model.    Other variables such as overall brand campaigns, corporate social responsibility investments, and social media metrics value will often show an increase in impact with the more advanced multivariate approach.   Companies that use a direct regression model are often undervaluing their brand campaigns and social media investments in favor of direct mail, email etc – the levers that are easily tied to a sale.

This even allows you to even consider tradeoff decisions in terms of which business levers you should invest in as part of a Financial Strategy conversation.

As modeling infrastructure expands to allow for more variables, models will become more robust and more explanative of all the impacts that drive revenue – resulting in better decisions for businesses!

NPS (Net Promoter Score): You Manage What You Measure

We have a new revolution in brand measurement mentality – making this decade forever known as the Net Promoter Score (NPS) era.  Are there any marketing professionals that haven’t heard of NPS (invented by Fred Reichheld of Bain Consulting)?  If you haven’t, it’s time to get educated:

NPS is a fantastic measurement for any company to use!  It is truly a revolutionary concept, consolidating lengthy brand survey results into a single trackable KPI that is easy to understand and intuitive.  Companies who focus on NPS improve the customer experience in order to enhance customer retention and increase brand reputation.  Because of this, NPS is an attractive metric for executives and helps our business decision makers quickly incorporate business analytics into their decision making.

NPS is Everywhere

One friend of mine told me his company even displayed NPS scores for the employee benefits hotline, separate from the NPS score for HR as a whole.  I see NPS-driven signs at hotels, at restaurants — even on the phone when calling my health insurance, they tell me that “a score of 8 is not good enough and if I can’t give them an 9 or higher, please contact their management so they can attempt to change my mind.”  NPS is absolutely everywhere!

So, if NPS is so wonderful, what’s the problem?  NPS is everywhere on everything, and has become synonymous with brand equity.  When having a conversation with people about brand health or brand reputation, NPS and brand are used interchangeably.  For example, take this question I got last week:  “Can you tell me what I can do to improve my business’s NPS?  I really want to improve the brand reputation this year.”

NPS is a great indicator of brand, but it is not actually brand – it’s simply one way of attempting to measure brand.   It is extremely popular due to its intuitive, straight-forward nature and how easy it is to track – but it is only one indicator of brand reputation and not brand itself.

Other Measurements of Brand

Before NPS and after NPS, there are hundreds of other ways of measuring a company’s brand – including many other types of survey methods that are still used today.  (Particularly noteworthy is that NPS is solely based on existing customers, where other metrics may capture potential customers who still can have a word-of-mouth influence on your brand — despite having never been your customer. These potential customers would be reflected in other brand awareness studies, but not in NPS.)

We can’t leave out the long-famous pricing test, discrete choice conjoint analysis — taking your product and put a competitor’s label on it, asking customers what they would pay for it versus the correctly labeled version.

There are even financial estimates of brand value, such as Top 100 Brands which attempts to quantify a company’s brand equity from financial evaluations.

Lastly, just like survey results, brand equity measurements can be surmised from social media analytics.

Brand Measurements

My point is that valuing a brand is difficult and complicated, and multiple approaches exist.  The multitude of methods may provide differing results at the same time.  No one single method of measuring brand is right — all are attempting to give a company indications of an underlying brand value that is very difficult to measure — and NPS is just one of those indicators.  Understanding what is really happening is where skilled analysts come in, to help business decision makers get a clear picture.

For example, even companies with the exact same NPS score may have widely different brand values by other metrics — in 2008, Amazon and Google had the same NPS score (73), but in Interbrand’s Top 100 Brands for that year Google was ranked 10th and Amazon was way down the list at 58th.

Optimizing NPS is not optimizing Brand

Sales compensation is often an indication of revenue or profit margin numbers, yet sales compensation is not perfectly correlated with revenue or profit margin numbers — as most sales managers know all too well.  Changes in sales compensation plans are managed very closely because individuals can game the system,  using unintended means of maximizing their sales compensation number that don’t result in a matching increase in sales for the company.

This can also be applied to NPS — there are methods of optimizing NPS scores that may not increase the overall brand equity of a company.

When a window installer says, after finishing, “In a few days, you’ll be given a survey and if you can’t give me  9 or 10s on all the questions, would you allow me to change your mind now?” is he really going to make a difference to the company’s brand?  Do you think that hanging up a sign in your hotel or dry cleaners that says “Giving us a 8 on our survey is not good enough. Talk to our management if you can’t give me an 9 or better” is optimizing your brand?  Really?  How is this impacting your brand any more than hanging up a sign that says – “Not fully delighted with your experience? Talk to management so we can help”?  The first sign, which calls out survey numbers specifically, will indeed have a much greater impact on your NPS score — but it won’t impact your brand equity any more than the second sign.

In addition, NPS is more heavily weighted towards customer service metrics than other brand measurements. My research  showed that NPS is more heavily correlated to (and therefore likely influenced by) customer support than the other brand metrics measured.   If you choose to use NPS as your company’s only method of measuring brand equity, it could result in your emphasizing customer service over other factors.  If an upscale jewelry company only measures NPS , it may work to further improve its already fantastic customer service – but may overlook billing issues that are keeping some customers from returning.  Even more so, these billing issues could be relatively easy to mitigate, providing a larger ROI than improving the already great customer service.

Finally, looking only at NPS could also limit you to only considering your existing customer pool –which may be acceptable for some mature businesses that already own a large piece of the market,  but could be extremely limiting for others.

So, the plan to improve brand reputation may be different than the plan to improve NPS.  Don’t get me wrong – it is a good idea to attempt to improve NPS – just remember NPS isn’t the same thing as brand equity. NPS is just one of many ways to measure it.  

What is the Marketing World Coming To?

“What is the marketing world coming to?” exclaimed my friend at our lunch conversation after reading the article on Target’s modeling approaches that identified a teenage customer as pregnant before her father even knew.  He was disturbed at how much information companies would know about him and felt his privacy was being invaded.

“Isn’t it wonderful?”  I replied, honestly excited at the level of information that marketing scientists like myself now have access to and all that we could do with it.  I truly found the Target story exciting and encouraging to my profession, unlike my friend, who was disgusted.

We went on to have a very interesting debate about how both Amazon sending him book and video recommendations and Google returning more accurate results based on his preferences are helpful, but somehow, targeting for other things such as knowing you are considering a kitchen remodel or might soon be registering for a wedding was crossing some line for him.  I think since it’s such a new technology, most people don’t understand how it is done, and many are unaware that it could even be done in the first place – so consumers find it alarming.  As it becomes more common, it will eventually be expected and appreciated.

I didn’t even bring up that I believe that soon, colleges will look to personality and intelligence profiling from social media in evaluating college applications.  They could go so far as to perform facematching to see if you’re a party child or if you really did spend all year helping homeless puppies get adopted like you wrote on your essay.   Another scary thought is that life insurance companies could base premiums off of lifestyle details that customers don’t know they’re providing.  As one example, life insurance companies could use location information from your cell phone, tweets, or photos, upping your premiums if you’ve gone too often to that “bad-for-you” BBQ restaurant. Of course, the flipside is that people who were actually living healthily would have below-average insurance premiums costs… but we were eating lunch at one of those BBQ places.

My friend decided to swear off facebook, free email programs, turned off location service on his phone, quit his frequent purchaser clubs, and even removed his tolltag from his car – all just to avoid the prying eyes of us data scientists.

As I did with him, I beg you to not consider this some evil plan of the marketing industry.  All of these new modeling techniques could potentially be beneficial to you too!  Imagine walking into a new bakery for the first time, and they already know what dish you will like the best — and despite that they have an award-winning soup, they know you prefer spicier soups and steer you to a delicious sandwich, just by knowing what you usually buy at your local grocery store.

Think about when you are in the market for a new car and commercials come on about a car that you are considering – and how you actually stop and watch the commercial instead of walking out of the room or fast-forwarding through it.  You are in fact interested in a potential product and want to hear the messaging to see if it is worth purchasing.  In this new world, companies who sell cars will know you’re in the market and are concerned about gas efficiency, so you’ll see website ads that show you details on how this car’s MPG compares to other cars in this sector, giving you the information you need to make a decision, all without you having to spend time on research or go out of your way to let sellers know you’re in the market.  This saves you TIME.  It also means many of those meaningless annoying commercials will be replaced with ones that talk about purchases you’re actually considering.

Another potential use for this type of modeling would be to identify medical risk factors based on where you’ve been and conditions you’ve experienced. By considering this information as part of your medical history, doctors may be able to more quickly diagnose conditions, or alert you to potential health risks — offering more targeted testing, or telling you things to look out for.  Take this hypothetical:  Some very unusual but potentially lethal virus has shown up from a patient who was at the same mall as you on Saturday (which both you and the virus infested patient were tagged as having been at due to credit card purchases).  In order for this virus to take hold, it needs darkness and you tweeted about going to the movie theatre afterwards.  Some futuristic doctor may contact you and tell you to be on the lookout for certain symptoms as your location history indicates you are at a higher risk of getting this disease.

The real beneficiary of this modeling is of course the companies who are marketing.  Targeted marketing has always had the best return on investment.  Modeling improves a company’s targeting, thereby minimizing spend on people who are not interested — they don’t waste money on people who were not going to purchase in the first place.   The money saved by not advertising to these uninterested people alone will justify this type of modeling and ensure that it continues.

However, there are some other potential gains for companies.  Companies will benefit from more accurately valuing their customer in terms of where to focus sales and marketing efforts, as models will allow you to better weight customers who are more likely to purchase, will spend more money, or will have a greater influence via word of mouth

Eventually, companies will begin to change their messaging to more granular subsets of consumers – emphasizing specific features based on the customer’s main drivers in the purchase. Continuing the car example, one customer may be more likely to purchase if safety features are emphasized, while another customer will prefer to see the beauty of the design — and still another wants to know how the product compares to competitors in terms of maintenance and warranty.  Each of these are different customers who may all be considering the same product for different reasons and will need their value system to be catered to, to encourage the purchase.  If customer desires are modeled properly, this will deepen the return of the marketing dollars.

We could even go a step further: instead of different messages about the product, differentiate the message delivery as well.  A company may know that one customer likes slapstick humor and serves up ads that will resonate with that customer, while a different customer prefers to hear a deep-voiced man describe the awards the product has received – and another is compelled with emotional imagery that help him feel connected with the product.

Like it or not, target marketing based off of the information you purposely or unintentionally provide is happening right now and will advance significantly in our lifetimes.  For those of you like my friend, who finds it creepy, companies will just need to be a bit more discreet about why we are targeting you.  ;)