# Estimating Time-Varying Coefficients with Rolling Regression | by Abhijeet Talaulikar | Jul, 2023

## Gaussian Random Walk as priors for regression coefficients

One of the challenges in regression analysis is to account for the possibility that the relationship between the predictor variables and the response variable may change over time.

Time and again data scientists come across scenarios where business has assumptions and our models must incorporate the possibility that they’re right.

Vanilla regression models are great at estimating E[Y|X]. The coefficient of Xᵢ (or βᵢ) can be interpreted as the incremental change in Y when Xᵢ is increased by 1. In econometrics, it is often hypothesized that βᵢ varies with time and we could call it βᵢₜ. What we are really trying to estimate then, is E[Y|X,Time]. Here’s a chart for the 2-year revenue of a company — it is a seasonal business and there’s no reason to believe that the β’s are also time-dependent.

In this post, we will introduce the concept of time varying coefficients in regression and show how it can be applied to a marketing mix model. If you haven’t already, do check out my Medium series on building a MMM from scratch and all the elements that go into it. Just to recap, a Marketing Mix Model is a statistical tool that helps marketers measure the impact of different marketing activities (such as advertising, pricing, promotions, etc.) on sales or other outcomes. A typical marketing mix model can be expressed as:

where yₜ is the outcome variable (such as sales) at time t, xᵢₜ are the predictor variables (such as advertising spending, price, etc.) at time t, βᵢ are the coefficients that measure the effect of each predictor variable on the outcome variable, and Ɛₜ is the error term.

However, this model assumes that the coefficients are constant over time, which may not be realistic in some cases. For example, the effect of advertising on sales may vary depending on the seasonality, the product life cycle, the competitive environment, etc. To capture this variation, we can use time varying coefficients in regression, which allow the coefficients to change over time as a function of some other variables. For example, we can write:

where βᵢ(t) are now functions of time or some other variables that capture the dynamics of the coefficients. There are different ways to specify these functions, such as using splines, polynomials, random effects, etc. The choice of the function depends on the nature of the data and the research question.

When the effectiveness of media changes over time, it can be captured by a time-varying function as the coefficient. One of the most popular choices is a Gaussian Random Walk.

You’ll be interested to know that this process is used to model many data in finance such as stocks or gold.

A Gaussian random walk is a stochastic process that models the cumulative effect of independent and identically distributed random variables. It is also known as a normal random walk or a Wiener process. A Gaussian random walk can be represented by a sequence of random variables X₁, X₂, …, Xₙ, where each Xᵢ follows a normal distribution with mean 0 and variance 1, and the sum of the first 3 variables

is the position of the random walker at time n. A Gaussian random walk has some important properties, such as being a Markov process, having stationary increments, and being continuous in probability.

Here’s an upgraded version of the PyMC model I built in this post. I’ve added code to vary coefficients by week, by quarter, by semester or by year. This form of regression is also commonly called a Rolling Regression.