QuantifiedNation #2: It's not you, it's the model
Quantified Nation brings you Marketing Measurement Magic through the fingertips of Jesús Martín and Pablo Pérez. Subscribe so you do not miss future editions.
Food for thought
The application of modeling to marketing measurement has been with us for a long time. The invention of the audimeter paved the way in US for the Nielsen Radio Index in 1942 and the Nielsen Television Index in 1950. These innovations revolutionized marketing measurement by introducing listening devices in select households and employing statistical models to analyze sample data and derive metrics from the population. Since then, we can have a more accurate understanding how many people watched a TV program and by extension, the ads within them. Further improvements would allow access to such information in less than a day with greater granularity.
Technology has been key in marketing measurement and the great revolution came in the context of digital marketing. When measuring digital advertising, available technology allows you to access many observed metrics which do not require modeling. For example, the number of times the ad was shown, measured by the metric impressions (or viewed impressions, if possible).
These observed metrics have somehow received lots of praise and attention, technology has allowed us to observe them, avoiding modeling techniques. However, in case you want to measure how many people has been impacted by an ad or a campaign (reach) and how many times (frequency), that leads us towards a completely different story which cannot be directly observed. You will need assumptions, estimations, models. Moreover, if you want to measure combined reach across different channels, you will also need a blind faith in the modeling ability of your measurement provider, if any.
Impressions are lazy, reach and frequency are for measurement magicians.
Leaving reach measurement aside, a similar situation occurs when measuring marketing outcomes or effectiveness. Experimentation, including A/B, incrementality testing and other variations attempt to find a causal relationship between a marketing activity and a business outcome assuming a distribution, using models. Since the 60s, Marketing Mix Models have been used to understand how different marketing activities generated sales. Panels, brand or ad recall surveys rely on models to extrapolate results from a sample.
When digital marketing started to become prominent, the availability of observed behavioral data exploded. In terms of outcomes, there are metrics reporting if the user had done something in my website after seeing or clicking on an ad, over time. Having such amount of metrics for your digital marketing campaigns, why would you care about modeling? The accuracy of such data has been always low, especially for mid/upper funnel media due to incomplete path to conversions, cross device behavior or different standards per marketing channel to name a few. However
Furthermore, recent technological and regulatory changes have further restricted access to observed data within digital ecosystems, making it impossible to rely solely on that data for optimal decision making.
Despite this, many companies continue to treat this information as if it were accurate or even more dangerous, considering it is the ground truth. Some even employ last-interaction attribution (or a variant) in their cross-channel marketing evaluation including ROI calculations. This is particularly true for reporting marketing results to finance teams, where observed metrics are still preferred.
While this data is essential to fuel Artificial Intelligence solutions in digital marketing, this approach is no longer sustainable in the current landscape for a proper evaluation of marketing channels effectiveness.
Observed last click attribution is lazy, modeled data driven attribution is for measurement magicians.
It could be the way you have always done it. Or perhaps this is the way how you have learned from others. Maybe you are avoiding adopting other models due to a valid concern about bias. You may think the time of accuracy in digital measurement will have a come back. However, all signals point the opposite.
In case you do not take modeling seriously, integrate it into your marketing operations, including understanding and validating the models, ensuring best practices, and enjoying the benefits, your marketing will fall far from optimized.
Hot Takes
Click for a teaser of the main ideas Econometrics and the C-suite (available for a small fee). A very diverse set of experts, including effectiveness legend Les Binet, share share their views on econometrics.
Interview (click to see) to Les Binet and Peter Field on the 10th anniversary of their renowned “The Long and the Short of it”.
Ridhima Kumar from Aryma Labs recently shared their approach for adding granularity to MMM by using a variation Shapley Additive Explanations (click for details of their approach). This is an alternative so the probably most usual approach of running a multivariate regression with detailed marketing inputs that explains the overall variable that lacks granularity. Of course the best way to solve the granularity problem in MMM is to collect more data (store-level, region-level…) but that is not always possible. Great to see innovative approaches to extract the most information out of a limited dataset.
Kantar just launched “Campaign Audience ValidationTM” in Spain (press note), which is their proposal for cross media post-evaluation of reach and frequency of advertisement. The product is also available on Argentina, Brazil, Chile, Colombia and Vietnam. This solution aims to provide a relevant coverage of cross media reach combining Kantar proprietary data with additional sources such as Ads Data Hub for YouTube. However, the information provided is somehow obscure. Why can’t we make press notes clearer and explain products in more earthly terms?
One more for Kantar, which released their “Media Trends & Predictions 2024”. Click here to download.
Seems that the Ehrenberg Bass Institute is up to something soon. Looking forward to it! A new book maybe? We’ll follow the news.
Industry updates and upcoming events
The MMA is organizing a virtual session on “Bridging the Finance / Marketing Gap” on december 13th. Click here for more details and registration information.
The NYU/ARF “Certificate in Marketing Optimization and Insights” is open for enrolment in December for the cohort to start on January 2024. Click for more details.
Chart of the day
The searches of “popular brand” have recently overtaken the ones of “cheap brand”. Interesting to see 2008-2012 hump of “cheap brand” during the global crisis and its later steady decline.
Source: Google Trends
Classics revisited
Today we want you to read this already classic Harvard article from March 2023 which discusses why Marketing Mix Modeling (MMM) is making a comeback and why it is essential today to calibrate the MMM using experiments, with recommendations on how to do it. Tell us what you think about it!
Thank you for reaching the end of the newsletter. We hope to see you again in the next edition. Until then, share your thoughts, let us know what you think in comments, reply to this e-mail or get in touch with us (Pablo and Jesús) on LinkedIn.