It’s been nearly seven months since comScore first announced the introduction of Media Metrix 360, our new panel-centric Unified Measurement of digital audiences. Our stated premise behind this initiative was to bring the digital media industry a solution which integrates server-side web analytics which do a good job of measuring total page views (if properly filtered for non user-requested traffic and counted correctly as one beacon per page) and panel-based audience measurement which provides insights into the behavior of individual people, as opposed to cookies or machines. The response to this initiative has been overwhelmingly positive, as evidenced by the high level of participation among top publishers – approximately 75% of the top 50 publishers in the U.S. are either fully reportable under this new methodology or in the process of doing so – as well as the reaction we’ve gotten from agencies and other industry stakeholders. This new methodology becomes even more important when considering the evolution of digital media, including the emergence of new channels for media consumption (mobile devices, tablets, e-readers, etc.) and the increasing fragmentation of the content landscape.

In short, our industry requires a digital media measurement infrastructure equipped to handle the realities of the next decade and beyond, and comScore has risen to the challenge. As with any major undertaking, there have been some hurdles and challenges we’ve encountered and overcome along the way. But in the course of these seven months, we have learned an extraordinary amount that we believe is enabling us to quickly vault our industry years into the future.

Through this undertaking, we’ve accumulated a new understanding of the highly complex and fragmented digital media landscape. To say that measuring this environment is complicated would be a severe understatement. But we have rapidly unearthed many intricacies and nuances which lead to a more accurate and harmonious measurement landscape.

• One of the earliest – and perhaps most obvious – findings along the way is that we’ve seen ample evidence of the foibles of server-side analytics for measuring the number of unique visitors (i.e. people) who visit a site. Due to many inflationary factors, including cookie deletion and rejection, bot and spider traffic, and site visitation from multiple locations, we’ve found clear and direct evidence that web site servers routinely overstate actual people counts by a factor of two and higher. In particular, the inflationary impact of cookie deletion is consistent with independent research from a variety of other research companies, including Forrester, Belden, Jupiter and Nielsen. The inflation in server data has also now become apparent to the IAB (see their Audience Reach Measurement Guidelines) and academics such as Max Fomitchev, an assistant professor of Computer Science & Engineering at Pennsylvania State University who conducted an exhaustive study and concluded:

  “Cookies are about just as inaccurate in estimating unique visitors as unique network addresses. This is the new and unrealized fact in the industry that has a direct impact on Internet advertising as currently reported unique visitor / core audience size numbers tend to overestimate the true audience size by a large factor (7-30, depending on the visitation frequency and the sampling period).”

As we have reached critical mass among websites participating in 360 programs, we have found some astounding statistics. In the month of December alone, we have seen over 1.5 Billion unique cookies just in the US. For certain portals we have observed significantly more cookies than there are Internet users in the U.S. These realities indicate that it is imperative to develop sophisticated methods to remove inflation caused by cookie deletion, rejection, multiple browsers, multiple devices and multiple locations of access such as home and work.

We’ve also discovered instances that illuminate some of the limitations of panel-based measurement. While we have numerous instances where a panel performs extraordinarily well in measuring audiences, there are some types of sites that can be more difficult to measure with a panel, including those visited by heavy Internet users from work computers and certain niche audiences. Huffington Post is one site that falls into both these categories, which has led to a substantial increase in its unique audience counts under the hybrid methodology. On the other hand, we’ve found that other sites show minimal changes under the 360 methodology.

A few people are under the impression that all we do is simply re-publish a Web site’s server data and charge for it. The statistics we publish are based on integrated panel / server methodology and typically show usage levels substantially lower than what one would obtain from raw server data alone. It is important to keep in mind that we are focused on people, which is what is needed for media planning. When an agency plans a campaign, it is usually looking for unduplicated reach across sites to assess the net reach a campaign will deliver. This can only be done by using a panel to track the same user across sites, using a unique and persistent identifier (which, because of cookie deletion and rejection, cannot be a cookie) to calculate the degree of audience overlap and the allocation of impressions between reach and frequency. It’s also important to identify the specific individual using the computer at any point along with their demos. There is no way to do this in an accurate manner without using a panel. Hence, the need for a panel, and the reason why the Media Metrix 360 methodology is based on a “panel-centric” model.

In addition, our position as an unbiased third-party audience measurement standard creates a burden of continuously and closely auditing any server data source that we receive to ensure it fairly represents the relevant audience being reported. Server data are notoriously complex and require significantly greater rigor than simply slapping a beacon on a page and reporting the calls. We need to filter out non-user requested traffic (pop-ups, i-frames, promotional servers, robotic traffic, etc.) and sort traffic according to its point of origin (geography and then home, work, public computer, mobile device, etc.). It is very important that we count everyone applying the same rules and standards. Furthermore, we have to take great pains to make sure that no company is attempting to game the system to try to get higher audience estimates. We also have to diligently apply edit rules along with branding and ownership standards that are strictly applied and rely on the use of our panel for verification. This particular role that we play is akin to the big 4 audit firms auditing a company’s financial statement by verifying the company’s own data, or the Audit Bureau of Circulation, which audits circulation of newspapers and magazines.

While it’s clear that there is widespread consensus that building Media Metrix 360 is the right approach, a small minority have questioned our decision to charge non-clients $5,000 to set up their site’s reporting using the Media Metrix 360 methodology. We believe this is a very reasonable price to charge for six months of reporting back to the client. If at the end of that period, the client does not want to continue subscribing (no-one has yet done so), we will still continue to implement the 360 methodology free of charge as long as the web site continues to maintain our beacons / tags. While we certainly understand that not every site can afford to purchase our data, the simple reality is that we incur substantial incremental costs associated with auditing, processing, editing and storing terabytes of server data and then integrating it with our panel data using our proprietary statistical methodologies. Companies willingly pay just for audit services -- and at a significantly higher rate than we charge. We believe it is only fair that we charge for the work we have to do and the use of the comScore assets we bring to bear on providing the service.

Some have argued that collecting data in this manner should be a cost of doing business and that some other services offer beacon-based measurement approaches for free. However, this argument neglects to account for how those other services monetize their approach – which is by selling publishers’ audiences (using their own cookies) to advertisers (and occasionally to their competitors), sometimes with the owner of the cookies not even being aware that this “Trojan Horse” strategy is happening. comScore is not in the business of selling a client’s cookies and that is part of the reason why our industry views us as the leading unbiased yardstick of Internet behavior. In life and business, there is no such thing as a free lunch, so Web site operators need to carefully consider the true cost of being beaconed by companies who are in the business of selling their cookies or using them for targeted advertising by leveraging information they got from those sites. In fact, we have observed instances where some Web sites were surprised and dismayed to discover that the “free” services they were using were, in fact, off selling their audience cookies on the online ad exchanges without their knowledge. As you can imagine, in such cases the beacons and tags were swiftly removed.

Ultimately, we both understand and expected some growing pains in undertaking such a major initiative as Media Metrix 360. Change, after all, is never easy. But it often reaps significant rewards down the road, and we’re delighted to find that our clients are moving us down this road faster than we ever anticipated. In the end, we believe the industry will be significantly better equipped for the future using our new reporting service and that it represents a more advanced, transparent and accountable measurement platform than has ever existed in the history of media measurement. And that is a very good thing.



Tags:

• Media Metrix 360

Last week comScore released our December 2009 U.S. Media Metrix data, which means of course that we can now look back at 2009 and take stock of what happened in the digital world during the past year. We’re in the process of summarizing these key stories right now and we’ll be releasing The 2009 U.S. Digital Year in Review report in the coming weeks. Stay tuned for more on that…

As we’ve been poring over the data for the year, the one story that continues to stand out is that of Facebook. Ever since it opened registration to the general public back in the fall of 2006, Facebook has seen considerable growth, so it’s not like this story is new by any stretch of the imagination.

And yet, even in its native market, Facebook continues to add to its audience at an incredible rate. In the past year alone, Facebook more than doubled its U.S. audience from 54.5 million visitors in December 2008 to 111.9 million visitors in December 2009. It went from being the #11 ranked property to the #4 ranked property. It now accounts for 7% of all time spent online in the U.S.

These numbers alone are enough to impress, but it’s when you dig deeper into the other metrics that Facebook’s performance really becomes illuminated. This past year saw Facebook grow substantially across nearly every performance metric reported by comScore. Unique visitors, page views, and total time spent all increased by at least double. Frequency metrics such as average minutes per usage day (up 6 percent) and average usage days per visitors (up 37 percent) also saw gains. In other words, more people are using Facebook more frequently to the point that the site accounts for three times as much total time spent online as it did last year. In fact, the only metric by which Facebook decreased was the average minutes per visit (down 11%), which makes sense when you consider the increasing frequency with which people are visiting the site.

To what can this success be attributed? I think there are a few prevailing factors at play. First, I think that Facebook reached critical mass in the U.S. a couple years ago at which point its growth began to feed on itself, allowing its momentum to vault it continually higher. Its growth also reflects in part the so-called “Zuckerberg’s Law.” At the Web 2.0 Summit in November 2008, Facebook founder & CEO Mark Zuckerberg famously remarked “I would expect that next year, people will share twice as much information as they share this year, and next year, they will be sharing twice as much as they did the year before. That means that people are using Facebook, and the applications and the ecosystem, more and more.” In other words, once the network is in place and people are active and engaged, the dynamics of the social interaction taking place incentivize participants to share information about themselves more regularly, which in turn solicits more engagement from others, creating a virtuous cycle of interaction. With increased interaction comes newer and fresher content, which helps feeds the addiction to consume information about what’s happening with the lives of people in one’s social network.

Anyway, Facebook’s continued ascent is just one of several big stories from 2009 in the U.S. digital media landscape. We hope you’ll check out the full report when we release it.

Last week I had the pleasure of teaching Advertising Effectiveness 101 as part of the IAB professional development series. I've taught a few of these before and every time I come away more pumped up about the opportunities ahead for our industry. If you aren't familar with the program the IAB gives certification to people new to the industry who complete 40 hours of training on online advertising. They do some of the training and a few other companies also offer courses. It's a great way for people new to the industry to quickly get a foundation and show they are serious about our business. Peter Cervieri from Scribe Media was there to ask some questions after the event as you'll see in the video below.

The event was sold out so it was full of smart folks excited to be there. Many came from traditional research backgrounds and were looking to integrate more online advertising measurement into their multi-channel plans. Others included new technology vendors and people primarily buying online advertising. We had a great discussion about how you need to start measuring online advertising effectiveness with the basics - person-based uniques, frequency and GRPs -- thereby building trust in our ability to deliver comparable metrics that make the conversation comprehensible when marketers look to online's capability. Of course once they start spending more online we can dazzle them with our amazing talents in measuring advertising impact attitudes, behaviors and sales too!

This post was originally published at new media age on January 7, 2010.

Many key developments in digital media stand poised to redefine the way we consume content across platforms in 2010. Among these will be the introduction of paid-for content, further blurring of traditional TV broadcasting and online video distribution, and a significant acceleration in mobile internet use. This last is particularly important, for while content will continue to converge under the digital umbrella, the mobile internet can bring it to a wider and more consistently engaged audience.

By October 2009 there were approximately 48m mobile subscribers aged 13 and over in the UK. Of these, 13.3m accessed the internet from a mobile browser and just under 17m used a mobile app. Driven largely by the success of the iPhone, smartphones already account for 17.5% of handsets in the UK. When you consider that 65% of smartphone users accessed the internet in October 2009 compared with just 28% of all UK mobile subscribers, the potential for these devices to drive mobile internet use becomes clear.

Apps are undoubtedly the poster boys of the smartphone revolution. With more than 100,000 of them now available, their diversity is clearly appealing to a broad audience. Of the 17m mobile subscribers who used an app in October 2009, maps were the most popular, followed by weather, social networking and search apps.

But beyond the current must-have status of apps, more pragmatically the mobile internet is fast becoming part of our daily lives. Some 3.3m people accessed news and information daily through their mobile in October 2009, with a further 4.3m doing so on at least once a week. Of all UK mobile internet users, 41% were aged over 35, proving that this phenomenon isn’t the preserve of the young.

In many ways the mobile internet will this year finish what the fixed internet started. One could argue that the internet has always been fundamentally flawed: the most up-to-the minute, real-time distribution mechanism in the history of media is constrained by access and hardware. The mobile internet looks set to rectify this, providing round-the-clock, immediately accessible news, information, entertainment and communications. At the risk of hailing the coming of yet another false dawn, expect the mobile internet to be big in 2010.

Happy New Year, everyone!

Yesterday, Fred Wilson posted some extremely interesting observations that relate to the problem of using clicks on referral links to measure attribution and assign credit to affiliates for referrals.

Affiliate Marketing Undervalues The Link

Fred’s conclusion is that using clicks fails to give affiliates full credit for the value of their referrals. As you can imagine, Fred’s post resonated with me because of comScore’s extensive research into online advertising effectiveness. We have come to the same conclusion as Fred. I responded to Fred with a comment that I hope you’ll find valuable.

Fred, you're correct that past comScore research has shown that search and display ads cause latent buying and offline buying that are not reflected in the click:

• http://www.comscore.com/Press_Events/Press_Releases/2006/03/Online_Impact_of_Offline_Buying
• http://www.comscore.com/Press_Events/Press_Releases/2008/11/Value_of_Online_Advertising

Essentially, it all comes back to the fact that the click does not reflect the "view thru" impact of ads (i.e. the impact of ads that are not clicked) -- and that even clicks don't accurately measure all the latent buying that occurs (because of cookie deletion).

Interestingly, this is not at all surprising to anyone who has had experience measuring the effectiveness of advertising in traditional media. Because there are no clicks to measure or cookies to track in TV, print or radio advertising, researchers use consumer panels and analytical designs that compare the behavior of people exposed to an ad campaign with the behavior of a balanced control group of people who were not exposed. The buying difference between the groups (after adjusting for any pre-campaign differences that might have existed -- and which are very small if the control group is carefully selected) can be attributed to the effects of the ad campaign. The same reliable analytical design is used in cinical trials by pharmaceutical companies to measure the efficacy of new drugs.

At comScore, we use our panel and the test vs. control group approach also on behalf of many of our publisher and advertiser clients to measure the effectiveness of online search and display ad campaigns in lifting sales -- both online and offline. We have also used it to measure the holisitc impact of referral programs. It really is the only way to accurately measure the complete impact of online ads and referral links in lifting sales of the advertised brand.

Here's a link to an example of the encouraging results we get when we use the approach to compare the effectiveness of online ad campaigns with TV advertising. It compares online and traditional advertising on an apples to apples basis:
http://www.comscore.com/Press_Events/Press_Releases/2009/8/comScore_dunnhumbyUSA_Research_Shows_Online_Advertising_on_Par_with_TV_Advertising_in_Growing_Retail_Sales_of_Consumer_Packaged_Goods_Brands

Your comments are welcome.