The Quant Revolution in Marketing: the Past, Present, and Future
In case you haven’t noticed, for us marketers, there is a revolution going on.
What revolution?! Some may ask. Well, it is the same great quant revolution that, like a perfect storm, has forever changed many other industries - professional sports, investment, and etc.
In the investment business, data has always been very important. Especially for value investors, it is critical to access as much data as possible for an investment target and then try to determine whether it is over- or under-valued or just-about-right. Value investing as a strategy involves a fundamental judgement: is a stock trading at significant discount against its intrinsic value?
The quant investing revolution started in the late 1970s and early 1980s, but major breakthroughs were not seen until the1990s. In contrast to value investing, for the most part quant investment do not bother with fundamental judgement on intrinsic value, but instead try to exploit market inefficiencies or to act on predictions on short-term trading movement.
Perhaps not surprisingly, marketing has been following a rather similar trajectory, yet some twenty years behind quant investing. How has the quant revolution altered our industry? What are the advantages and limitations this revolution has brought? Where is the revolution leading to?
To answer the above questions, it is necessary to divide the quantitative revolution in marketing into three stages, each corresponding to the past, the present and the future (see Table).
Stage 1: Pre-Revolution
The Era of Market Research
The invention of market research in the 1910s for the first time allowed companies to make informed marketing decisions, it also infused to a limited degree consumer centricity. Instead of selling what one produces, sell what consumers want. This simple common sense was a major breakthrough in thinking in the age of “any customers can have a Ford any color he wants, as long as it’s black”.
While informed-decisions tend to be better than otherwise, the role of data in this stage was to serve as either the knowledge base or validation for human decisions. Human intelligence, with all of its glories, also comes with many flaws. Often, market research data was cherry-picked or entirely brush aside when it contradicts with decision maker’s thinking, or reduced to rubber stamp to “validate” whatever decisions desired.
Furthermore, two major limitations in market research data cast long shadows on how much impact data can exert: data scope and granularity. Most survey nowadays lasts some 20 minutes and on hundreds of samples. The limited amount of data that can be collected means rather narrow and shallow data input; and the fact that findings from small samples cannot be reliably extrapolated out to the broader population (partly because of lack of big data) confine any conclusions to PPT slides while attempts to execute them tend to be “lost in translation”.
Hence there has never been a shortage of jokes on market research.
Stage 2: The Dawn of Quant Revolution
Clearly, given the limitations of market research data, it takes a new type of data in order to summon the dawn of a new era. Yes, it had to be big data.
And in China, likely any marketing historians, if any, would agree that Alibaba’s launch of the Uni-Marketing framework in 2018 marked the beginning of this stage. With DataBank and StrategyCenter, marketers for the first time got their hands on a wide array of data on real consumer e-commerce behavior. Suddenly, insight extractions such as consumer profiling and segmentation can be done with a wide range of real behavioral data which yield fresh insight; and better yet, the resulted profiling or segmentation can now become more than just a theory on a PPT slide: target audiences can be tagged with unprecedented accuracy in the real world (albeit in a database); and perhaps the best yet, with the insight and accurate tagging, precise messaging and media plans can be tested in the real world against all kinds of KPI including purchase conversion to identify the more effective combinations. In no time, suddenly marketers were thrilled by the magic power of increasing ROI by many folds, waste and fraud reduced, GMV happened, …
For the first time in history, marketers are able to extract insight with huge amount of data both in width and depth, and directly translate strategy into testable and optimizable executions with less “loss in translation”. And also for the first time marketers are able to link specific strategy and campaigns to short-term GMV and therefore ROI becomes a plausible and measurable KPI.
While big data allowed for unprecedentedly powerful solutions, the limitations in the big data and data platforms currently available again got into the way.
First of all, most of the big data available is behavioral data. While the data provides more granularity and allowed for tagging on real consumers, it has one major drawback: behavioral data can only tell how consumers behave, not how they make decisions - in other words, we know “what” but not “why”. As a result, identifying target groups with higher ROI based on correlations or A/B testing is the norm. In rare occasions, some analysts would dig out “why” using traditional market research and then use the insight to guide big data analysis, but that’s rather the exception than the norm. Many probably will ask, didn’t some big data guru state that correlation is everything? Well, if you are just predicting stock movement in the next 72 hours, yes, correlation is sufficient. But if you are trying to figure out what is going to be the next blockbuster product or where the consumer taste is heading to or if there is a killer selling message to be had, correlation is utterly insufficient.
Another major limitation has to do with the level of access existing marketing data platforms offer. Almost invariably, the platforms do not allow apps to be embedded which can execute the analysis automatically based on existing algorithms; nor can the A/B testing, tagging, media placement be carried out seamlessly without tons of human interventions. All these means that human is still the thinker and worker who makes the decision and pushes the buttons. Marketing automation in this sense is simply beyond reach.
Stage 3: Paradigm Shift
Changing the Game beyond Human Recognition
If there is ever a sci-fi writer who, out of utter boredom, wants to write about the future of marketing, most likely it will look like this:
● Much of the routine marketing decisions, especially the executional level ones such as media spot plans, target selections, incremental improvement in communications, etc., will be carried out and optimized automatically based on self-improving algorithms; some of these are already a reality today.
● Most advertising, social media contents and so on will be largely generated by AI engines. Already, AI can compose a piece of Bach-like music to pass as a real one to even the trained ears. Similarly, machine learning can be deployed to roam through existing creative contents and the resulting black-box algorithms can be used to generate new contents with little human intervention. There are already some companies striving down this road. It is just a matter of time when some of them can churn out AI contents that can pass as the fruits of a decent copywriter.
● Similarly, machine learning will comb through consumer journey data to devise individualized consumer journey intervention plan to pinpoint the best timing or occasions to reach out to certain consumers with the most effective messages/activities in order to serve the marketing objectives, be it seeding, brand building, purchase conversion, retention, or fission.
● Aside from routine deployment of AI and algorithms, Stage 3 will demonstrate a major departure compared with Stage 2: Stage 2, for the most part, is about incremental efficiency gain working within the existing consumer behavioral and decision patterns, Stage 3 will proactively reshape these patterns by re-engineering consumer journey and restructuring consumer decision-making. Here machine learning coupled with historical data alone won’t be sufficient. Experimental design guided by behavioral sciences will shed light into the new frontiers in marketing and reveal new insight and solutions to nudge consumer decisions.
To fully realize the potential that Stage 3 has to offer, it requires data that can depict the entire consumer journey as complete as possible. Today’s data silos (e.g. TMall’s databank, etc.) are very powerful but each of them alone can only cover a small corner of the vast digital universe. Given the transforming value a full data set possesses, it is just a matter of time before someone figures out a viable offering.
With such a magic dataset available, the profile of marketing professionals will increasingly look like that of a quant fund: less will business students but more will be mathematicians, programmers and behavior scientists.
Marketing in this stage will be full of excitement, wonders, and magic powers. In a sense, marketing, armed with data and algorithms, will be more powerful than ever in shaping consumer behaviors. But will consumers and governments tolerate that? Will consumers turn against marketing for pushing unwanted products to them like they have been with Facebook and etc? Will the governments become weary to the industry for manipulating the mass like the they have been with Cambridge Analytica and its incarnations?
Clearly marketers at the forefront of the quant revolution need to adhere to two principles: 1. Utmost respect for consumer privacy; and 2. Nudging consumer behavior only when it adds value to the consumers - enabling them to make fully-informed, wise, and sustainable consumption decisions. After all, if one day a marketer wakes up and realizes that she possesses this magic power to shape consumer decisions, she better asks herself this question: do I want to live in a world where marketers freely deploy data and algorithms and manipulate my decisions with little consideration for my own personal wellbeing?
With the stage set for the great quant revolution, what are the implications for today’s marketers? All great shifts in an industry inevitably bring unprecedented opportunities. At Illuminera, we believe that these opportunities will be made available in this revolution:
● Data Asset: data is the new oil. Any unique data sets that can complement or, better yet, replace existing data silos will be of great value.
● Algorithms and Applications: algorithms, simple or complex, that can make decisions on behalf of marketers will be embedded in automation apps. In the very near future, these apps will be making daily targeting and messaging decisions. Given more time, they would make strategic marketing decisions better than their human counterparts.
● Inter-system Automation Systems: integration systems that can link and facilitate the collaboration among various data silos, algorithms, digital media outlets, e-commerce platforms, digitalized offline touch points, and other stakeholders.
● Behavioral Sciences and Social/Cultural Forces: unlike quant investing, arbitraging market inefficiency will not make brand owners a lot of money. As such, pure statistical approach is not sufficient. We need profound insight on how consumers make decisions in order to learn how to nudge their decisions. Behavioral sciences, especially behavioral economics, coupled with social and cultural forces that sway how consumers think, will serve as the beacon when data scientists race to build the killer algorithms and apps that can nudge better than others.
Certainly the jury is still out on whether or how the third stage will turn out. However, at this moment, I would kill to hear well-thought out answers on what the Stage 4 might look like.