One of the challenges in regression analysis is to account for the possibility that the relationship between the predictor variables and the response variable may change over time.
Time and again data scientists come across scenarios where business has assumptions and our models must incorporate the possibility that they’re right.
Vanilla regression models are great at estimating E[Y|X]. The coefficient of Xᵢ (or βᵢ) can be interpreted as the incremental change in Y when Xᵢ is increased by 1. In econometrics, it is often hypothesized that βᵢ varies with time and we could call it βᵢₜ. What we are really trying to estimate then, is E[Y|X,Time]. Here’s a chart for the 2-year revenue of a company — it is a seasonal business and there’s no reason to believe that the β’s are also time-dependent.
In this post, we will introduce the concept of time varying coefficients in regression and show how it can be applied to a marketing mix model. If you haven’t already, do check out my Medium series on building a MMM from scratch and all the elements that go into it. Just to recap, a Marketing Mix Model is a statistical tool that helps marketers measure the impact of different marketing activities (such as advertising, pricing, promotions, etc.) on sales or other outcomes. A typical marketing mix model can be expressed as:
where yₜ is the outcome variable (such as sales) at time t, xᵢₜ are the predictor variables (such as advertising spending, price, etc.) at time t, βᵢ are the coefficients that measure the effect of each predictor variable on the outcome variable, and Ɛₜ is the error term.
However, this model assumes that the coefficients are constant over time, which may not be realistic in some cases. For example, the effect of advertising on sales may vary depending on the seasonality, the product life cycle, the competitive environment, etc. To capture this variation, we can use time varying coefficients in regression, which allow the coefficients to change over time as a function of some other variables. For example, we can write:
where βᵢ(t) are now functions of time or some other variables that capture the dynamics of the coefficients. There are different ways to specify these functions, such as using splines, polynomials, random effects, etc. The choice of the function depends on the nature of the data and the research question.
When the effectiveness of media changes over time, it can be captured by a time-varying function as the coefficient. One of the most popular choices is a Gaussian Random Walk.
You’ll be interested to know that this process is used to model many data in finance such as stocks or gold.
A Gaussian random walk is a stochastic process that models the cumulative effect of independent and identically distributed random variables. It is also known as a normal random walk or a Wiener process. A Gaussian random walk can be represented by a sequence of random variables X₁, X₂, …, Xₙ, where each Xᵢ follows a normal distribution with mean 0 and variance 1, and the sum of the first 3 variables
is the position of the random walker at time n. A Gaussian random walk has some important properties, such as being a Markov process, having stationary increments, and being continuous in probability.
Here’s an upgraded version of the PyMC model I built in this post. I’ve added code to vary coefficients by week, by quarter, by semester or by year. This form of regression is also commonly called a Rolling Regression.
Let’s look at what’s changed in the code.
First, you will notice that this model is now wrapped in a function BayesianMMM. This is for convenience, so we can run it with different arguments. This example takes one argument called splits, which decides how granular is your definition of time — weekly (W), quarterly (Q), half yearly (H) or year-over-year (YoY).
You will also notice that the PyMC model itself takes an argument called coords which is essentially your time variable. If you choose to vary coefficients every week, the time variable would be 1…104. If you choose quarterly, it would be 1,1,1…,2,2,2…,3,3,3…4,4,4…,1,1,1… (remember, this is 2-year data.
Next, our coefficient is now a GaussianRandomWalk variable instead of TruncatedNormal. Just like before, we initialize it at the location prior (if you don’t recall this, go back here). In this setup, every time period has its own coefficient but the coefficients share information across consecutive time periods. This is the biggest strength of a rolling regression.
Now let’s take a look at three case studies.
Upper funnel marketing refers to the activities that aim to generate awareness and interest among potential customers who are not yet ready to buy. It is often difficult to measure because it does not directly lead to conversions or sales, but rather influences the customer journey in a subtle and long-term way. However, it helps to know which channel is more effective and the trends over time.
Very useful knowledge has come out of this. Around the start of the time period, Direct Mail was the least performing channel for a long time before it surpassed TV. Anecdotal evidence confirms that with the advent of CTV, the effectiveness of TV has diminished over time.
Another observation would be CTV climbing higher than Email over time. These are interesting observations, but we cannot draw conclusions without A/B tests.
Marketers should not assume that paid search effectiveness is constant throughout the year, but rather adjust their campaigns according to the quarterly fluctuations and opportunities. However, some general trends can be observed especially in retail and e-commerce.
Paid search effectiveness tends to be higher in the fourth quarter, especially during the holiday season, when online shopping activity increases and consumers are more likely to click on ads that offer discounts, deals, or gift ideas. Conversely, paid search effectiveness may decline in the first quarter, when consumer demand drops and advertisers face lower conversion rates and higher costs per click. The second and third quarters may show more stable or moderate results, depending on the specific industry and market conditions.
We find that Paid Search effectiveness is expectedly highest in 4th quarter.
Year-over-year trends are changes in key performance indicators (KPIs) that occur from one year to the next. They can help marketers set realistic goals, adjust their budgets, and optimize their tactics based on what works and what doesn’t.
By analyzing historical data, conducting market research, and testing different strategies, marketers can optimize their paid search effectiveness and achieve their goals in every quarter. Publisher guidance and in-house A/B tests can also identify times when the ROI is highest. Lastly, all of this helps calibrate our MMM.
We see that YoY Paid Search effectiveness has improved by roughly 2%. Trends like these are reported quite often and attributed to the success of marketing departments.
If you’ve followed my MMM series, I’ve stressed on how important it is for your MMM (or any other measurement) to be consumable by stakeholders. Time-varying coefficients help answer a lot of questions that may be thrown at a vanilla MMM. With Bayesian optimization techniques, we can make small adjustments into our model code so it can inherently answer those questions.
One of the advantages of using time varying coefficients in regression is that it can provide more accurate and flexible estimates of the effects of predictor variables on the outcome variable over time. It can also help identify when and how these effects change and what factors drive these changes. This can help marketers design more effective and efficient marketing strategies and optimize their marketing mix.
Thanks for reading 😊