Granularity is a critical component of any measurement approach. It is the level of detail – the breakout of KPIs and media variables. Too much granularity results in fragmentation and noise. Too little and measurement outcomes are overly general and lack the ability to optimize. The trick is to find the correct balance between accuracy and precision that produces the most actionable results. Leavened recommends clients first think about how they plan media when considering what level of granularity makes the most sense. Things that are planned together and have similar impacts on consumers are most likely able to be grouped together.
Marketing analyses try to answer one thing: how did doing X impact the consumer to do Y? When considering the best level of granularity, consider which marketing efforts should have a similar impact. If there are too many unlike things together then the question of ‘how did X impact Y’ begins to break down. If part of X had a large impact but another part had a smaller impact, that gets lost when rolling the variables up together. Likewise, if X1, X2, and X3 all are known to have similar impacts then breaking those out is unnecessary. The goal is to find like things that are likely to have like outcomes and group them together.
Macro-Grouping of Media
Media granularity is the breakout of marketing data into macro sub-groups like channel, partner, strategy, and format and micro sub-groups like tactics, audience, and creative. Generally, Marketing Mix Modeling (MMM) measurement solutions should consider the macro granularity to get insights at the channel and strategy level. It is very likely that all video on Facebook impacts the consumer similarly. It is also likely that all TV impacts the consumer similarly (and differently than Facebook). This ensures insights that impact the overall media mix. Should we invest more in Facebook? Is OTT working for our brand? What is the optimal frequency of TV? These are the types of questions that should be answered at this level of granularity.
Sometimes when spends are either low, or don’t have enough time in market, it is necessary to roll up some channel variables into higher groupings. When rolling-up variables, it is important to consider the media strategy. Rolling up all OTT partners into a single variable might make sense because while it won’t offer insights into which OTT partner is working best, it does inform whether OTT as a category is an appropriate approach. This assumes that all OTT partners impact the consumer similarly. While it may be preferable to get insight at a partner level, sometimes there just isn’t data available to provide that insight. The answer is either roll up the data for channel insight or get no insights at all.
Micro-Breakout of Media
The more detail the better, right? This isn’t always the case. Endlessly breaking up variables into their sub-components can create too much noise. This comes from the infinite permutations that lead to decision paralysis. It can also be difficult to find statistically significant results with too many variables. There are many different measurement solutions, and each is best served at different levels of granularity. Market mix modeling (MMMs) is best at understanding the marketing mix (we’re not very creative in naming things). Many MMMs begin to break down when trying to understand too many tactical things like creative copy. Copy testing is great at understanding which ad copy works best but cannot inform a media mix. One way to solve both macro and micro levels of granularity is to build nested measurement solutions, Leavened does this with AdImpact.
Nesting analyses are one way to add tactical nuance to a strategic model. Once a primary set of models are developed and an overall optimal media mix understood, run new analyses on each media channel to get more tactical insights into what drove that channel’s contribution. This can be either a multi-stage regression (as in AdImpact) or a nested model.
Multi-stage regressions help understand how each tactical breakout impacted the category contribution indexed against each other. This informs TV media buyers, for instance, that primetime NBC performed twice as well (200 index) as the average station/daypart. It can inform a social media team that targeting gamers underperformed targeting home chefs. This all happens in two steps. First completing a MMM at a strategic macro-granularity and then running an additional analysis based on one specific variable’s contribution. Now the media teams not only know what the optimal media mix is but also how to optimize within that mix.
Another type of nested analysis is nested models. Instead of building multi-stage regressions, a contribution from one MMM is used as a dependent variable of another. This attributes a portion (or all) of the contribution of one media channel to other variables. This can be necessary when there are too many different channels or tactics. When this happens, it may be difficult or impossible to build a stable model. Rolling up like channels, like Brand Media vs Sales Activation, make it easier to build an accurate picture of how media is impacting sales. While it is great to know the ROI of Brand Media, the media team needs more granularity to optimize the business. So nested models are built. The contribution of Brand Media in the direct model is used as the dependent variable within a nested model. The channels within Brand Media are then split out again and used as the variables for that model. This reattributes the Brand Media contribution to the media channels and gives media teams more granularity to work from.
Finding the Right Balance
Marketers are always looking for the silver bullet solution that provides insight at all levels. While we may not have a single solution, marketing can build one by combining several solutions together. Too much granularity can lead to analysis paralysis at best and untrustworthy insights at worse. It is important to lean into the benefits of each measurement solution and measure at the right abstraction. Leavened prefers to build a top-down approach. First accurately understand how a media mix impacts business and then break out channels into sub-models. While precision is important, it is imperative to not sacrifice accuracy for it.