For Google's AdWords, relevance takes time

Google's AdWords system often isn't fast enough in ranking relevant ads during a sudden
surge in search query volume.

(Credit: Screenshot by Tom Krazit/CNET)

Dodge's Challenger is a modern muscle car. The Challenger explosion 25 years ago was a tragic moment. Other than the name they don't have much in common, but for several hours Friday morning, Google's AdWords system considered them linked.
That's just one example of a weak spot in Google's famous AdWords system, which turned an interesting Stanford science project into the world's most powerful Internet company. Simply put, it takes some time for the AdWords system to determine whether an ad triggered by a search query is truly relevant to that query, meaning that in times of breaking news or a sudden spike for certain queries Google often serves completely irrelevant ads, such as the one promoting the Challenger's Hemi engine above news stories about the 25th anniversary of the Challenger space shuttle disaster.
A week of study of Google's "hot searches" as measured by Google Trends--a compilation of search terms whose query volume is disproportionately rising at a given hour compared to their usual frequency--provided numerous examples of how AdWords can require at least several hours to obtain enough feedback to properly rank ads.
Breaking news stories about the death of fitness guru Jack LaLanne triggered an ad for The Cord Bug, an accessory for car owners in cold climates that need to keep their engines warm overnight, in the most prominent slot. After a five-foot long monitor lizard was discovered wandering around a Southern California condo complex and showcased on morning news shows Wednesday, Google News served computer-monitor ads for several hours alongside search results.
This is probably not an issue on incoming CEO Larry Page's immediate to-do list, as Google continues to make quite a bit of money from relevant ads on the majority of searches. But it does speak to the thorny problem of determining relevancy in real time: it's not just a search problem, it's an ad problem too.
Your Quality Score is important to us
Ad rankings on Google for search keywords are determined by two main factors: the maximum cost an advertiser is willing to pay per click, and an ad's "quality score," which is a measure of how relevant the ad's copy is to the desired keyword, among other things. Even if an advertiser is willing to spend a lot of money per click, if their ad scores poorly on quality, it will likely appear below ads from advertisers that weren't willing to pay as much but scored higher on quality.
However, it takes time for Google to determine the quality score for a new ad. It needs to measure how often users are clicking on the ad as compared to other ads, as well as whether users are staying on the landing page behind the ad as opposed to returning immediately to Google.
How long does this take? Google won't say, but it's at least several hours in many cases.

Morning news reports on the discovery of a monitor lizard in Southern California sent
people to Google looking for more information, and the ads they were served during the
early spike in those queries weren't particularly relevant. Why serve an ad at all?

(Credit: Screenshot by Tom Krazit/CNET)

Google would only offer a statement on the issue. "Google's advertising system determines the quality of an ad based on how users are responding to that ad. This process can take a brief amount of time, especially if it's a fast-rising query that is newly popular," it said. Your definition of "brief," of course, may vary.
The gap is important for a few reasons. First of all, Google's top priority is to serve relevant content to its users, and it has long considered ads to be useful content so long as they are relevant to one's query.
Also, the gap allows advertisers to piggyback on search queries in Google Trends much the same way news organizations latch onto those reports in hopes of directing some of that search spike their way. An advertiser could get a decent amount of traffic relatively cheaply if they are quick to jump on a trending keyword that not many other people have purchased, taking what they can get before the quality score calculations take place and kick them off the page.
One prominent advertiser on trending topics in Google throughout the whole week was Ask.com, which confirmed that as part of its ongoing traffic-acquisition strategy it frequently purchases Google ads linked to trending search terms that direct clicks back to Ask.com's pages on that topic. (Yahoo and Microsoft's Bing employ similar strategies.) Those ads actually fare well in the quality score calculation since it's clear what type of content Ask.com is advertising, but it's not hard for others with less-relevant content to employ the same fast-mover strategy and settle for the second, third, or fourth spot on the search-results page until the calculations take effect.
AdWords showing its age?
But perhaps more troubling for Google is the notion that the system that generates an amazing amount of cash is a bit too creaky for a Web that publishes content at a speed which Google never could have anticipated 10 years ago when the system was first designed.
Expectations of how content should be delivered on the Internet are changing as news publishers and consumers focus on speed: just look at the demand for information following reports that Michael Jackson had died in the summer of 2009. There is an opportunity to serve relevant ads alongside that content in Google News or Realtime search that the company is simply missing because of the delay in determining relevancy.
That doesn't bode well for its chances of using the current incarnation of AdWords to monetize real-time content on Google. Irrelevant ads aren't good for anyone in Google's system: users don't want to see ads perceived as spam, advertisers want to target likely buyers, and Google won't make money from ads that receive few or no clicks. That's not to mention any institutional embarrassment from missing the mark when it comes to relevancy.
As Web usage shifts more and more toward the real-time consumption of content, Google will need to develop a strong system for ranking both the relevancy of the content as well as the ads. Somewhere, someone is working on this extremely difficult computer science problem. If they're not inside of Google already, the company might want to find them.