Online shoppers are no doubt familiar with the pattern: they’re browsing for the perfect pair of running shoes only to have numerous running shoe brands follow them around the rest of the Internet as they browse news sites and social networks.

They suddenly see ads for local races and foot deodorizers, too. What they likely don’t know is that behind these well-timed, highly-relevant ads is an increasingly sophisticated array of technologies to help these ads land like precision-guided enticements in their browser.

Welcome to advertising in the age of artificial intelligence.

A brief overview on artificial intelligence in CNBC

Like so many industries awash in information, advertising is being consumed and transformed by technology. The buying and selling of ads used to be mediated by humans. With the rise of programmatic advertising, it’s increasingly mediated by machines. Advertising has become Adtech.

Thanks to software powered by artificial intelligence, these machines are smarter than ever - helping marketers reach the right customer, at the right time with the right ad.


So What Is Programmatic Advertising?

Programmatic advertising is essentially a technology infrastructure that arbitrates the buying and selling of digital advertising. What used to be handled by humans with contracts and insertions orders is now transacted by computers talking to each other. This advertising can land on your laptop, your mobile phone, digital billboards dotting a highway and even smart TVs. Though not very old, programmatic advertising has grown rapidly into the dominant form of digital advertising.

Don’t take our word for it.

Since 2013, the percentage of marketers who have allocated over 50 percent of their budget to programmatic campaigns has grown from 7 percent to 32 percent, according to eMarketer. A separate eMarketer study estimated that just about four of every five US digital display dollars, some $32.56 billion worth, will transact programmatically in 2017. The share of ads transacted programmatically will only grow. eMarketer sees programmatic transactions reaching 84 percent of all digital display ad spending by 2019.

According to ZenithOptimedia’s Programmatic Marketing Forecasts, programmatic advertising will notch a 31 percent climb in 2017 outpacing current ad darling - social media (25 percent) and online video (20 percent). Meanwhile, Dun and Bradstreet predicts a 70 percent growth in programmatic ad spending by business-to-business advertisers this year.

As programmatic advertising has grown, so has a subset of the technology called Real-Time Bidding (RTB). As the name suggests, RTB involves the automated buying and selling of ad impressions through auctions in real time. (And when we say “real time” we mean it. Sigmoidal’s advertising clients require RTB transactions to unfold in 2 milliseconds--that’s a fraction of the time it takes the human brain to recognize an image.)

RTB works like this: When a visitor lands on a web page, the browser sends a request to an ad server, which then places that ad request on an exchange where software can bid on those impressions. Prospective advertisers (or more accurately, their software) analyze the impressions and decide how they want to bid, if at all. The exchange collects all the bids and awards the ad spot to the winner. This all happens in the time it takes a web page to load.

Like programmatic advertising, RTB is experiencing rapid growth. This year, 44 percent of digital display ads will be bought via RTB, according to eMarketer.


How Can You Compete?

This marketing migration to programmatic advertising and RTB has paved the way for powerful artificial intelligence (AI) software to transform how impressions are bid on, how ads are created and how they’re served to consumers, unlocking a significant growth potential for the companies who embrace it.

The key is data.

That shoe-shopping customer who is getting served ads for running sneakers and insoles as she travels around the web is carrying with her valuable virtual data. Everything from her browser, the time of day she’s surfing, what she’s clicked on, where she lives, what she’s “liked” on social media can be collected to create a profile. The more data that can be collected about an individual, the more accurate this profile becomes. Data can be sourced from the marketers themselves, from browser cookies and from third-party aggregators.

AI software can group or cluster this data to create aggregated user profiles. It can detect patterns that link attributes together in ways humans might miss. You can take the entire online universe and segment it into groups of people--some more willing to buy your product or fill out an online form on down to those who would be completely uninterested in doing either. Armed with this data, AI software can enter an RTB auction with incredibly predictive power--it knows which customers on average are going to be the valuable ones it wants to get an ad in front of, and which ones to avoid.

Simply having data isn’t enough, of course. You need intelligent software to rapidly analyze it and act on it. Many marketers compete in RTB bids with algorithms based on rules to place bids when the appropriate conditions are met.

While rules-based algorithms can execute rapid bids, they’re not as powerful as software that can predict consumer behavior and make millisecond decisions on what bids to place and for how much. For that, you need software based on AI techniques such as reinforcement learning.

With reinforcement learning, an algorithm automatically determines what an ideal behavior should be in a given environment to maximize a given goal. In this case, the environment is a real-time bid for ad impressions and the goal can be generating a lead or a sale or any customer-derived performance indicator for an ad campaign. Humans tell the software the budget, the goals and then set it loose on the auction to do its thing.

If the algorithm sees an audience it judges to have a low probability of purchasing a product or generating a lead (based on that data profile we noted above), it can avoid bidding on those impressions in an auction. It can also serve ads to those customers less frequently. Both approaches save marketers money, since they’re not wasting dollars chasing customers they’re less likely to win.

If the algorithm sees an audience with high probability of clicking on an ad it can automatically change the bid to decrease the average cost while still winning the vast majority of auctions.

As the name suggests, reinforcement learning is a process of iterative improvement. The software judges the results of its performance. Strategies that proved successful towards the goal set by human marketers get pursued again, while those that were less than optimal get discarded. Over time, this generates more effective bid strategies resulting in a more effective, more personalized ad campaign that uses the profile data of customers as its fuel. This ability to personalize and target ads is a powerful force. One survey found that 78 percent of consumers were more likely to make a purchase when presented with personally relevant advertising.

What’s Reinforcement Learning?

Reinforcement Learning is a very specific part of AI. It differs from standard Machine Learning problems (i.e. Classification and Regression) mainly because there is no ground truth for the model to predict.

For example, in image classification, a human can recognize whether there is a cat or dog in the picture. Similarly for house price prediction. In Reinforcement Learning on the other hand there is no "teacher" to tell the model what to aim for. In chess or Go, the only way to determine what action to take is to experiment with acting somewhat randomly and observe which moves led to a win and which were losing ones.

So what does RTB have to do with Reinforcement Learning? In RTB, the problem we try to solve is to find a minimal winning bid for a particular impression i.e., a bid that will win the auction, but anything lower would lose. Obviously, such a value is never observed. All the bidder knows is whether or not his bid was high enough to win the auction. This makes it a Reinforcement Learning problem, because the Model has to take an action (value of the bid) and gets an immediate reward (profit). With time, Model adapts to maximize expected profit conditioned on a provided bid.

In the real world, AI-powered ad software is already flexing its muscles, with impressive results. It helped this New York-based Harley Davidson dealer grow its lead generation by 2,930 percent after just three months. In all, that dealership tripled its sales for the year after implementing AI adtech.

The lingerie brand Cosabella enjoyed an eye-popping 336 percent increase in return on ad spend in just three months after it deployed AI-powered software to manage its advertising. Revenues increased 155 percent in the fourth quarter alone.

Many marketers are already taking note. According to Juniper Research more efficient RTB networks powered by machine learning will generate $42 billion in ad spending in 2021. In 2016, that number was pegged at just $3.5 billion.

AI-Powered Creativity

Beyond making the the most effective bids, AI can also maximize the effectiveness of your creative assets. Marketers can pool their creative assets - ad copy, images, color schemes, video--into a template that software can then automatically assemble and display to customers groups it thinks will be most receptive.

This Dynamic Creative Optimization, as it’s called, doesn’t just produce a single static ad. It can mix-and-match this assemblage of creative assets based on the data it has on the potential customer to create the most effective pitch for a given customer at a given moment. It can continually test these combinations against internal metrics to improve the effectiveness of a given ad campaign. Maybe ads with “buy now!” perform better than ads with “call now!” in the tagline. Or maybe ads with puppy pictures outperform ads with kittens (hard to believe, I know).

Real-life case study example

One of our clients is an RTB platform. We were asked to design and build a Reinforcement Learning Model, which will be used as a bidding advisor. The main problem with developing an RL model is the fact that it learns "on the fly." It means that typical solutions are an initialization of a random policy, implementation into the environment and letting the model take actions and learn from them. This raises an issue, because redirecting part of traffic to the model, which at first bids randomly and it is difficult to estimate how long it will take the model to learn to produce sensible bids.

With that in mind, we decided to create a model that makes use of historical data to produce a bidding strategy. The method we came up with, enables us to deploy a model with already learnt strategy, but it is never possible to fully evaluate your model without deploying it. This is because of the nature of RL problems. For example, using historical winning bids, we know that higher bids would have also won, but we don't know whether a lower bid would have won or lost.

In a nutshell, our proposed algorithm assumes that there exists a random variable which represents a minimum winning bid and that it depends on the particular website and user. Using historical data, we were able to build a function that approximates the conditional distribution of that random variable. Using that approximation, we then determine expected profit as a function of a bid and directly optimize it. The bid that maximizes the expected profit is a bid returned by the model.

Once implemented, our solution managed to increase the company's average profit by 70%.

Bottom Line

For modern marketers, leveraging mountains of customer data in a real-time environment requires intelligent software backed by some serious computer power. Right now, Google and Facebook are devouring ad dollars because they have both. But the tools of AI-powered advertising aren’t out of reach to other large and mid-size companies.

Sigmoidal’s team of data scientists, developers and AI experts have built AI adtech solutions that have saved their customers thousands. They can do the same for you.