Senior Data Scientist, CtrlShift
Predicting Campaign Performance
The prediction project sought to provide better forecasting of impressions, clicks and cost values for campaigns, which could then be used to derive potential CPM, CPC and CTR metrics.
The fundamental assumption is that there exists a relationship between campaign settings and campaign performance. In addition, campaign performance appears to possess periodical patterns (daily, weekly, monthly). With these assumptions on how campaign performance manifests, it stands to reason that campaign performance can be predicted with a consistent degree of accuracy.
As an outsider to media trading, I needed to have multiple of meetings and discussions with the traders and the engineering team to understand finer details and pain points of media trading.
Working closely with them, we first distilled the process of setting up and optimising campaigns. We then built models to “mimic” the process and incorporated machine-learning decision-making modules. To build the time-series machine learning model, historical campaign data was used, and the model refreshed daily to incorporate newly captured campaign data.
The Lessons Learnt:
The complexity of the business problem surprised me in the early stages of this R&D project. The early projects did not perform very well which resulted in multiple iterations of learning and testing. We were able to overcome these early obstacles with the introduction of high-quality log level data from DSPs and by tackling the problem from different angles. For example, we began by building one generic model using all historical data so that we could directly predict campaign performance given campaign parameters and key dates while avoiding the problem of cold-starts.
However, this model was too general and resulted in poor performance. We then shifted to a campaign-level time-series approach. This allowed each campaign to have its own time-series model able to represent a more specific campaign pattern.
The model was verified by comparison with the conventional moving average model that is currently widely used by the media industry. We found that the machine learning model we built performed to a higher degree of accuracy.
In terms of predicting impressions, on average, our AI module reaches 96% accuracy and it is 0.6% higher than the moving average model in a 7-day ahead prediction. Similarly, for cost prediction, our approach is 0.5% more accurate than the moving average model.
What this means is that our AI approach can, unequivocally, provide meaningful predictions for the traders. The jump in accuracy may seem incremental or minor, but every cent is crucial when working with programmatically driven campaigns at scale.
For traders, the ability to know with an almost 100% degree of accuracy what a campaign will cost on day one, will enable them to better advise their clients quickly on spend expectations and offer the option of budget reallocation or strategy revision.
When scaled over multiple campaigns deploying millions of dollars, it can make a difference in how hundreds of dollars is ultimately invested – whether they optimized or wasted on non-performing strategies. The Lab’s campaign prediction module will soon be a released on The Hub, to further support traders in optimising and ensuring the best possible outcome for their clients’ campaigns.
This feature can also be scaled across the various integrations we have with media buying platforms such as DV 360, Search 360, Facebook, MediaMath, AppNexus and iPinYou. Another step forward in our ongoing mission of aggregating and building high-quality innovation and automation through our enterprise programmatic advertising platform – The Hub.
Liked what you just read? Schedule a demo to find out more about The Hub, CtrlShift’s one-of-a-kind enterprise programmatic advertising platform for unified media planning, activation and insights. It aggregates platforms (social, search, RTB, native) to streamline digital media management and help brands and agencies achieve more.