Programmatic Marketing High-Level Architecture. v.1.

What is programmatic marketing?

In digital marketing, programmatic marketing campaigns are automatically triggered by any type of event and deployed according to a set of rules applied by software and algorithms. Human skills are still needed in programmatic campaigns as the campaigns and rules are planned beforehand and established by marketers. (wiki)

Another good article is here.

Let’s imaging you would like to develop a programmatic marketing platform (PMP) that dramatically reduces or eliminates completely human interaction.

What components have PMP got?

Our platform is going to utilise online and offline machine learning algorithms to make decision in real-time how to bid for ad exchanges. And in order to function our machine learning algorithms need data from many different sources (click stream, analytics, offline, CRM etc) as well as ad exchanges.

So our system has 4 high level main components:

1. Data Management Platform (DMP).
2. Demand Site Platform (DMS).
3. Core component that glues DMP and DSP based on Big Data and real-time analytics.
4. User interface to setup, control and monitor the components of the system.

High-level architecture of Programmatic Marketing platform

Data Management Platform

Main functions of data management platform are:

1. Support data intake activities from multiple sources
2. Normalize and prune the data
3. Support batch processing
4. High availability due to real-time high load nature of this component
5. Allow to “reply” data intake operations (Optional)
6. Provide possibilities to a user to control, setup and monitor DMP

There many solutions for DMP platforms, paid or not, simple and not. However, there are not many of the support our purposes. And for this exercise we omit vendor selection or any other not-architecting activities.

Below you will find super high level architecture with a few cloud specifics.

Sample Architecture of Data Management Platform for Programmatic Marketing

As you see we are using SOA here with messing and other distributed technologies. Sorry for being so simple and high-level, but more details to follow later.

Demand Site Platform

DSPs are used to fulfil demand of a 3rd party service providers, in our case those are Advertisement Exchanges like Facebook AX, iAd from Apple, DoubleClick from Google etc. Those platforms ask interested parties in real-time to bid for an ad placement.

What are the main requirements to a DSP:

1. Performance. Normally, DSP (or it’s RTB, which stands for Real-Time Bidding system) must reply in restricted amount of time. For example, Google’s DoubleClick requires to have response for a bidding request in less that 100ms in 98% of the time.
2. Extensibility. DSP may support annormous amount of Ad Exchanges and other platforms to integrate with in an easy and robust fashion.
3. High availability and elasticity. Do you know how much bidding request you may get for a particular interest? I don’t…
4. Control and monitoring. Yes, this is the default one, but here it is worth to mention as we need to be in full control.
5. And in our case it must provide back data to training our machine learning algorithms further to perfection.

I believe you always can add something to the list above, but let’s assume that you need to put top 5. What would you put? Please comment.

Demand Site Platform for Programmatic Marketing platform

As you see, we again simplify many bits and peaces. So, but I want to give you an overview and not full solution this time.

In order to make DSP you may use many things that were made before, like open-source RTBs, Client SDKs from Ad Exchanges and data feeding possibilities to train our models further.

The Black Box of Data Science

The black box this time I will leave as is and may open it just a bit to make you curios to come back.

The main idea of this black box it to minimise CPI (Cost Per Install) or CPC or CPA in the circumstances that allow this.

The Learning Curve of a Machine Learning approach to minimase CPI

Our black box consists from 2 big parts:

1. Offline models preparation.
2. Online/Real-time decision making and model adaptation/personalisation.

For offline bit you need to use supervised machine learning technics for which you will need to obtain data with input features and validated output (CPI/CPA/CPC), so that you can produce/train your models and than validate them (70/20/10 data split is recommended here). I also see that you will need to produce a set of models that perform better in some conditions and when mixed produce acceptable results.

However, there is another part of this puzzle is to personalize advertisement in order to increase conversation even further, which will require real-time matching algorithms to implement optimal bidding strategies.

Sorry for not telling or opening this subject, but it is not possible to do this in a post or even series, because you need books for this ;)

I just would like to point out that use may use a few production-ready machining learning tools like Google Predictive API for offline model calculations.

Conclusion

Programmatic marketing has huge potential and huge demand in the platform that can support marketing in all channels to develop brand awareness and generate better ROI. And this should not be a surprise for you that Programmatic Marketing Platform is very complex and demanding platform which requires significant investments.

Yours,

@distributedLeo