Consumer privacy tool, not so much

Consumer Reports has just published "66 Ways to Protect Your Privacy Right Now".

Many of those suggestions look good. But that should probably be more like 65 ways.

One piece of software linked to from the CR story is Adblock Plus, which would be a better fit for CR's "Selling it" feature on sneaky offers, weasel wording, and other examples of gray-hat marketing.

What's so bad about Adblock Plus?

When you first install Adblock Plus, the privacy option looks good. It says "Adblock Plus can do more than block ads." Scroll down to the bottom of the page, and you see

Browse privately by disabling tracking - hiding your tracks from ad companies that would track your every move.

Looks good, so flip the "Disable Tracking" switch.

disable tracking screenshot

Protected from tracking, right? Wrong.

It looks like you made the responsible choice, and now you're protected.

But start web-surfing with your browser's developer tools open, and you'll see third-party trackers from, for example, What's up with that? How could it be that even when you deliberately turn on "Disable Tracking" you still get trackers?

That's the tricky part. The company's "Acceptable Ads" whitelisting program actually overrides the other choices made by the user, including that nifty little "Disable Tracking" switch. Google and other companies pay Adblock Plus for "Acceptable Ads".

Want to make your decision to block trackers actually take effect? You'll find the other option that you need in order to protect yourself in a different dialog, cleverly but not helpfully labeled "Allow some non-intrusive advertising".

whitelisting screenshot

To really disable trackers, un-check a box that has a label that says nothing about trackers at all.

It is possible for a user to configure AdBlock Plus to block trackers. And fixing something by changing an option in one obvious place and again in a not-so-obvious place is not really that bad, by the standards of instructions for computer hobbyists.

But this isn't about a story in Puzzles for Computer Nerds Reports, it's about Consumer Reports, and "Consumers" probably expect things to be more a little more straightforward.

Next steps: There are better ways to deal with problem web ads, and sites can help recommend them to users.

Why the Coalition for Better Ads is a complete waste of time

Business Insider reports that Google is planning to "rid the web of bad ads" with yet another new industry organization.

On a website announcing the Coalition, the group says it will develop the criteria based on consumer research, which will look into the kind of online ads people love and hate. The first iteration of the scoring system is expected to be released in the fourth quarter of this year...

(read the whole thing)

Really? Big Data and algorithms to analyze why web ads are somehow crappier than print ads?

Web ads did not end up being so annoying because agencies designed them willy-nilly and had no idea of how much they were getting on people's nerves. Ad agencies employ humans who can tell a magazine-style ad apart from a crappy web ad as well as anyone else can. The people who make web ads already know when they're crappy. A lot of crappy ads just get clicks.

Web ads did not end up being so annoying because Data-Driven Marketing masters have failed to do some magickal research that would quantify user irritation. Any web editor who has ever moderated a comment section knows what the problem ads are. Web sites would start refusing those problem ads tomorrow if they had the market power to do it. The annoying ads persist because publishers don't have the market power to enforce quality standards, they way they can in print. If a high-reputation site won't run a marginally too-crappy ad, the ad agency can go buy (what is supposed to be) the same audience on a marginally lower-reputation site.

The publishers of sites with quality level "9" don't want to accept an ad of quality level "8" but they know they'll lose it to a lower-quality site if they don't run the ad and the third-party scripts that come along with it. And by accepting the third-party scripts, the publisher is giving up data and making it easier for the next, even crappier, ad to squeeze them even more.

So even if all the "9" and above sites could get together in some coalition against crappy ads, then the crappy ads—which both get clicks and provoke blocking—just go to non-coalition sites. And the incentives to defect from the coalition are obvious and powerful. The lowest-reputation site remaining in the coalition can always make more money right away by leaving.

The privacy nerd solution doesn't work either

By now the privacy nerds are popping up to propose the classic privacy nerd solution: high-reputation sites should just unilaterally stop running third-party tracking that leaks their audience data to low-reputation sites.

Unfortunately, that doesn't work. In today's web ad environment, where users are trackable from site to site, intermediaries have the power over high-reputation sites that they need to extract cooperation.

What high-reputation publishers need is some kind of "clerk cannot open safe" sign for audience data in the form of client-side tracking protection. The game does work out to winnable by the publisher if the cross-site tracking options are limited. A site has to be able to tell an ad agency, "Even if we did include that data-leakage-perpetrating, battery-sucking, fraud-enabling script you want us to include? Our users are tracking protected. Want to reach our audience? Do it our way. Without the crap."

Bonus links

Getting the whole story on ad blocking

Covering the "adblock wars" (as Doc Searls calls the situation) is hard.

It's like showing up at a peaceful protest where organized looters show up and mix with the protesters.

  • If you're reporting on the event and you cover the legit grievances of the protesters without mentioning looting that you know about, then store owners who get hit will say you're full of crap and on the looters' side.

  • If you cover the looting but not whatever the peaceful protesters are saying, then anyone who turned up to join the peaceful protest will say you're full of crap and putting the stores ahead of their important issue.

The Adblock Plus paid whitelisting program isn't just "controversial". It's actively wrong by any standard but the Silicon Valley "anything for network effects lol" standard. Adblock Plus is talking hippy-dippy community woo-woo while they shake down legit sites.

Can we please tell both stories?

Sure, there's a malware story, and an adfraud story, and the long-running paradox of why the most targetable ad media are the least valuable and the most blocked. Web ads need to be fixed.

But we can't lump the looters in with the peaceful protesters, or we start fooling ourselves on paid whitelisting the same way that the IAB keeps fooling themselves on malvertising and fraud.

This stuff matters. When Internet fair use advocates tolerated the Napster racket, our side drove a lot of legit working musicians over to the DRMers and maximalists.

The "adblock wars" will be won or lost based on choices made at legit web sites. The people who work hard to put news and cultural works on the web will have to decide what to do about ad blocking.

Sites will end up choosing different options from a spectrum. At one end is Johnny Ryan's "reinvention not reinsertion" concept, where you privacy-protect ads as part of making them resistant to blocking. At the other end is the hard-core option of using CFAA and DMCA against adblockers (with privacy tools as the by-catch.)

Web publishers are in a double crisis right now.

The IAB can afford to half-ass the fraud problem. Publishers can't. But publishers can't afford to ignore the paid whitelisting racket, either. If we want to make a reasonable case for next-generation advertising, it has to work for the people who write, shoot, edit, and publish our news and cultural works.

So enough with the "AdBlock Plus is controversial ¯\(ツ)/¯" already.

Next steps: Faster, cleaner ad blocker blocking

Haha n00b, learn to hack!

The web ad business is full of deception, according thought I was going to say Bob Hoffman, didn't you?

No, I'm going to cite no less an authority than Interactive Advertising Bureau CEO Randall Rothenberg, who advises that any brand that has to deal with companies in his industry had better hire someone with mad hacking skills.

The problem begins with the unwillingness of major marketers to insource significant, senior-level technical expertise.

At Marketing Week, Thomas Hobbs quotes Alessandra Di Lorenzo, chief commercial officer for advertising and partnerships at

Anyone who is a non-digital, native brand is probably less skilled in the inner workings of media and should be more on guard about what is really going on.

This is like needing a Chief Power Supply Officer because you buy PCs and you don't trust your vendor not to put in a bad power supply. In any other business, if you have a problem with deceptive sellers, a halfway decent trade organization would be all over that. "Tired of getting ripped off by crooks who sell other people's condos on Craigslist? Next time, call a Realtor® from the National Association of Realtors®." You know the drill. Every trade organization does this.

Except the adtech business. From adtech, brands get roughly the same message that a WordPress user with a security problem would get on a phpBB site tricked out with a black background and rotating ASCII-art skulls. haha pwned n00b! Better learn 2 h@ck nxt t1m3!!!1! The IAB puts an ideological committment to unlimited third-party tracking ahead of the interests of its honest members' customers.

It's not an "advertising" thing—magazine ads really get printed, bus ads really get stuck to the bus, radio stations really transmit. Adfraud is the unavoidable by-product of today's web ad system that allows ads to be targeted across high and low reputation sites. You don't need a fraud expert to buy magazine ads, but the web is another story. And that "senior-level technical expertise" that IAB wants you to get is not cheap. Software companies have enough trouble hiring people with hacking skills—and now any brand that wants to run a web ad is going to need one?

Judy Shapiro writes, "Marketers' trust in the ad-tech world is on the decline for lots of reasons: complexity, lack of transparency or standards."

I suppose that's a nice way of saying that people who are responsible for brands are sick of being told that deception is here to stay and it's up to you to learn to deal or hire a hacker who can.

Dr. Augustine Fou writes,

We are, after all, dealing with hackers who are very advanced in the use of technology AND who don’t play by the normal rules of engagement. The good guys are at a disadvantage before the race even starts. Something other than technology must be applied at the same time – like changing the financial motives or changing the metrics used to calculate ROI. For example, rather than use quantity metrics – such as number of impressions, traffic, and clicks – that are easily faked, if advertisers focused on actual sales or other ‘conversion events’ that only humans would do, they would be far better off, and less prone to fraud stealing their ad dollars.

But unless you have Dr. Fou or some other expert working for you, attribution is no solution. One thing everybody can learn from the Steelhouse/Criteo controversy is that attribution models are subject to gaming, and it's hard to work backward to see where the attribution snatching happened. Fraud can piggyback on a user's activity in order to let a fraudulent ad take credit for a real sale. This is even worse than straight-up bogus impressions, because it encourages you to move marketing money to places where it doesn't reach real users. Most of the people who really understand attribution models are fraud hackers.

Bob asks, Will The P&G Story Bring Down Ad Tech? Please?

(P&G is an interesting example for all the behavioral economists out there. This company is mostly selling products that you buy because Society expects you, as a sanitary human being, to use them. If you don't know that Society is seeing the same ad, why take the risk of overspending on making life less stinky for others when you don't have to?)

The answer to Bob's question is: no, this story won't have much impact all by itself. But it does make people think: how can we transform web advertising from a hacking game that brands can't win into a reputation game where brands have the advantage? Some more on behavioral economics and possible next steps.

Predicting the future

Doc Searls, on Twitter, links to some predictions for the future of marketing.

If you read that, it looks like the future will be...basically, copied from a white paper about retargeting.

Yawn slash eeew.

You can't just predict that the future is like today but more so. That would have gotten us "interactive TV" instead of the Web.

I'm not going to predict the future of Marketing in general, but since there's a lot of attention to one corner of it, here goes.

Consolidation. I don’t know if we’ll get all the way down to a two-logo Lumascape, but single digits, maybe. Two-sided markets tend to consolidate, because buyers go where the sellers are and vice versa. This isn’t happening in web advertising, because agencies have an interest in artificially complicating the online ad business, and venture capitalists have funded a lot of competing, minimally different, startups that will take time to settle out. But with more scrutiny on agencies because of the rebates problem, and less incoming VC money to adtech, it will be easier to see the consolidation happening.

More and harder math. As complicated as the math in online ads is now, it’s about to get more interesting and potentially way more important. Right now, online advertisers are playing a relatively simple level zero game of maximizing response rate given the available ability to target users in each medium. The next step is the level one game where brands and publishers re-shape the medium (and the ability of users to control it) in order to adjust how well the medium’s users can be targeted by a brand and its competitors.

An ad medium that facilitates collection of information from the user also limits transfer of information to the user, which is necessary for brand building (There’s no free lunch). Brands and publishers will need to adjust the balance of targetability and signal-carrying ability. That means that over the next several years, advertisers will have to solve level one problems in the areas of Behavioral Economics and Signaling.

More memorable ads. As a user, right now you’re seeing a lot of crappy ads, because the problem of measuring immediate response to a terrible ad is easier than the problem of measuring Brand Equity changes as the result of a signal-carrying ad. The terrible ad problem is temporary. Advertising is not a zero-sum struggle with math and technology on one side, and creative on the other. Better math will have the side effect of informing and justifying better creative.

Ad blockers fade to the background. Right now, the ad blocker is a threat to legit ads because new blocking development is sustained by the paid whitelisting model and because high-value and low-value ads are delivered the same way. High-value ads will beat ad blocking, possibly with a combination of

  • legal and regulatory attacks on paid whitelisting

  • front-ending the CMS with a proxy server that stitches ads into place and obfuscates IDs and classes

  • limiting third-party tracking that facilitates low-value and fraudulent ads

Ad blockers (and other privacy tools, as a side effect) will still catch the crap ads. And advertisers will still have to consider ad blocking, but in the background, much as email newsletter senders have to consider the spam filter. Go watch that Johnny Ryan video that I linked to last time.

(This post started as an answer on Quora so go upvote there if you do the Quora thing.)

Reinvention, not reinsertion

In this week's ad blocking news, the Adblock Plus people are all butthurt that Facebook is reinserting ads by editing HTML instead of paying for whitelisting. I'm not sure who's winning right now, but by the time I put this up the current state of the "cat and mouse game" (when have you ever seen a mouse buy stuff from a cat?) will be different. So go follow Lara O'Reilly and Jack Marshall in your news recommendation tool of choice if you really want to keep up.

Anyway, the web advertising debate is about Adblock Plus the way that the online music debate was about Napster. Can we turn down the long-playing drama and pay attention to the important stuff here, for 23 minutes and 36 seconds at least? What will web advertising look like when the ad blocker fades into the background, the way that spam filters are a background concern for email newsletter senders?

Dr. Johnny Ryan, speaking at the Advertising Research Foundation (watch the whole thing) has a good part of the answer.

There is a big question over whether it is a good idea to restore all of the tracking functionality that might infringe on the user's personal rights, and that actually might take away from the value of the ad.

It's not just the user's rights. When an ad medium makes it harder to target individual users, it gets better at signaling. Context matters. There's no free lunch. Everything we know about how ads work in context tells us that the average web user is a pretty good applied behavioral economist.

That's where reinvention, not reinsertion, comes in. What does web advertising look like when it works with the user's mental "reputation coprocessor" and not against it? When advertising turns into a game that you can win with a sustained signal of quality, not with targeting tricks that have to change every time the users figure them out?

Watch the whole thing.

Faster, cleaner ad blocker blocking

I'm still working on figuring out the best way to block browsers that have certain ad blockers running, without pushing costs onto users.

Paid whitelisting and other practices make conventional ad blockers bad for web sites. But I'm running into a couple of problems.

  • Ad blocker developers can easily see blocker-blocker scripts and work around them.

  • Blocker-blocker scripts waste bandwidth and energy for users who are doing things right.

  • Some blocker-blocker scripts also block the users of legit privacy tools.

What I really want to be able to do is run the blocker-blocker script only for users who I can confirm are part of the problem—blocking ads but allowing third-party tracking, as seen in the paid whitelisting racket. Paid whitelisting is a dark pattern.

So what I'm going to do is first, run a third-party tracking test, then if that shows the browser is vulnerable to third-party tracking, add the ad blocker detector script to the page.

  • Privacy software users will pass the third-party tracking test, so get no ad blocker detector.

  • Unprotected users will get the ad blocker detector, but it won't detect anything. They'll see the page (to which the tracking detection script can add a warning about vulnerability to third-party tracking).

  • Users participating in paid whitelisting will get blocked until they either fix their configuration or install a privacy tool on top of their ad blocker.

The whole thing depends on detecting third-party tracking accurately. There are potential false positives here.

In all of those cases the tracking protection detection script will load, but the user has still made the choice to get protected.

I want to encourage, not discourage, tracking protection experimentation by users (It's better for sites.) So I can't just check if Google Analytics can load on the page. Accurately determining if a user is trackable is what makes the Aloodo Project interesting.

Anyway, script. Reduce bandwidth consumption and battery suckage, get a more accurate result. Ideas welcome. (Yes, I'll stick a real license on it if anybody needs one.)

What do you mean, we?

Enough with the "we" stuff about fixing web advertising.

This is not a "we" problem. "We" can't promise to replace "ads that provoke blocking" with "better-performing ads", because ads that provoke blocking are the high-performing ads. As a web user, you're not seeing crap ads because the advertisers want to waste money and annoy you. You're seeing them because they test well.

Crappy, annoying, deceptive ads get clicks.

The terrible stuff on the web is there because it works.

Everyone agrees that "we" need to get rid of "bad" ads. Naturally, "we" is defined as "you" and "bad" is "not the ads that work for me." But because the same qualities that get response also provoke blocking, there's no equilibrium strategy here.

Imagine that all the right-thinking people agreed to L.E.A.N. or some other set of self-regulation terms. No auto-playing videos, no NSFW animations, no fake error dialogs.

The more that self-regulation limits crappy/click-getting practices, the more incentive for any advertiser who is willing to bend the rules and offer a little more money to run an ad that's a little bit creepier, a little more attention-getting or finger-fumble-attracting.

Incentives for bad practices are there because users can be tracked from site to site. That marginally extra-annoying advertiser will be able to find a publisher with marginal reputation, who claims to be able to reach the desired users and is willing to accept the ad. And self-regulation breaks down, or never really gets going in the first place. Cross-site tracking gives everyone an incentive to do advertising that gets clicks today and provokes ad blocking tomorrow.

So there's no "we" solution. The fixes for the web advertising problem will have to happen one user at a time. Every user who becomes harder to track from site to site helps give high-reputation sites a little more market power to enforce ad standards.

Publishers and brands need action from users

In today's web advertising, high-reputation and low-reputation publishers compete to reach the same users. And high-reputation brands are hard to tell apart from low-reputation ones.

High-reputation publishers and brands win when users get less trackable, but users have to be the ones to take the action.

So instead of putting everything in terms of "we", it's time to think about reciprocity and measuring the benefit from each additional tracking-protected user. Instead of hippy-dippy "we" stuff, relying on everyone to cooperate, let's talk exchange of value. Big Data is not just a tool to help with low-reputation strategies. Data-driven projects can help with high-reputation strategies, too.

Questions might include:

  • Which customers gain the most value to me when they're protected from tracking by low-reputation competitors? (For an HMO, what's the net present value of protecting a customer from quack diet ads? For a car insurer, how much is it worth to keep the most profitable customers from being picked off based on their social media usage?)

  • Which categories of readers are most valuable to the best advertisers on my site? How much does it cost me when adtech intermediaries can follow them elsewhere? What's a cost-effective tracking protection solution that I can offer them, to keep them from being reachable on low-value sites?

I'm not against "we need to work together" messages in situations where a cooperative solution is really workable or necessary. But for fixing web ads? Time to give it up.

ANSI standard ad-supported piracy?

The Trichordist blog started pointing out the ad-supported piracy problem quite a while ago, so let's have a quick look to see how well the adtech business has done at cleaning up its act.

Should no longer be a story, right? The Internet solves problems on Internet time, after all.

Here's the plan. I'll spend one minute doing a basic check, then go work on something else. It's not as if there isn't enough broken stuff on the Internet I could be figuring out.

So I'll do a web search for

[Michael Jackson MP3]

I'll make it easy for them by picking a well-known non-Creative-Commons recording artist. I'm expecting to come up dry here. (After all, why would any sensible Internet company send me to a pirate site when they could make some money by sending me to a legit music download site, or sell me some tracks themselves?)

ANSI ad on a pirate site

Ouch. Probably the most obvious copyrighted works in the world, and who's got their fingers in the pie?

  • Amazon
  • BuzzCity
  • Google, Google, Google
  • LinkShare
  • LiveInternet
  • OpenX

But turn off your banner blindness for a minute, and check out that banner ad.

It's an ad for The American National Standards Institute (ANSI).

Why is ANSI running an ad on a questionable an infriging site, when it could be buying ads on a legit site that covers engineering and science? Spewing ads into the web's less reputable corners just feeds the growing impression that "technology" is a rent-seeking, deluxe-bus-riding racket that's focused on diverting value from others instead of creating new wealth.

So here are a few questions for ANSI.

  • How did your ad end up on an infringing site? Can you retrace its steps?

  • What agencies or other intermediaries did you work with to place the ad? Did they make any guarantees about what kind of site it would show up on?

  • Have you received a refund for ad impressions on problem sites?

  • If you don't have the information to answer the first three questions, what is broken about the way you buy advertising?

I'll keep you posted on what I come up with.

The NAA just did privacy tools a big favor.

The Newspaper Association of America has filed a complaint (PDF) with the US Federal Trade Commission about four ad blocking practices. The NAA asks the FTC to:

  1. Require ad blockers engaged in “paid whitelisting” programs to end such programs or to cease misrepresenting the nature of their services to consumers.

  2. Require ad blockers to discontinue ad substitution practices.

  3. Require ad blockers claiming that they make publishers whole to cease making deceptive statements that mislead consumers.

  4. Prevent ad blockers from evading metered subscription services and paywalls.

(Washington Post story: Newspapers escalate their fight against ad blockers by Elizabeth Dwoskin)

If we clarify number 4 to include only deliberate paywall avoidance, and not privacy measures that accidentally reset the article count for "soft paywalls", then NAA has just done a huge favor for the developers of legit privacy tools.

The NAA has written a pretty good start for a code of conduct for privacy tool developers and users.

Legit privacy tools are in "compliance" with the NAA's rules already. If you look at the tracking protection tools page, everything we link to or recommend already avoids the four no-nos. It shouldn't be a problem for any tool to avoid all of these. Paid whitelisting is a naked protection racket, ad substitution is reputation-harming scribbling of unreviewed ads into a publisher's context (yes, adtech does it too, that's not the point) and deception and sneaking in without paying are just so obviously wrong that why am I even typing this?

It's possible that some privacy tools can have the result of resetting a soft paywall, but it's possible to protect a soft paywall from accidental resets, and I can get behind a code of conduct that bans specific functionality to get around paywalls.


The first reaction to the NAA complaint was disappointing. (Please, Twitter and Medium, copy this YouTube feature already.) A bunch of early comments were along the lines of "well, existing adtech is bad, too!"

Yes, we know. Third-party tracking is not just a privacy issue. The trackability of users from high-value to low-value sites causes data leakage, which results in lower revenue for publishers, and enables fraud. And adtech targeting breaks economic signaling, which means publishers aren't just getting a smaller piece of the pie, it's a smaller pie.

Today's adtech is a trash fire of fraud, malware, and low revenue. But that means privacy tools have the opportunity to be different, by avoiding publisher-hostile schemes. When software developers send a privacy message but then just set a competing trash fire, they're wasting that opportunity.

Legit privacy tools and high-reputation publishers, working together, can transform advertising on the web. Tools and sites can help users block low-value, cold-call-like targeted ads while permitting signal-carrying ads, the ones that respect users' choices not to be tracked.

High-reputation publishers have a responsibility to both educate readers about the problems of adtech as usual and hold tool vendors to high standards. The NAA is making some real progress here.

How C.H.E.D.D.A.R. is your browser?

(Update: switched to BlockAdBlock.)

Doc Searls writes,

To have a deal, both parties need to come to the table with terms the other can understand and accept. For example, we could come with a term that says, Just show me ads that aren’t based on tracking me. (In other words, Just show me the kind of advertising we’ve always had in the offline world — and in the online one before the surveillance-based “interactive” kind gave brain cancer to Madison Avenue.)

Read the whole thing.

"Just show me ads that aren’t based on tracking me" is a message that you can send and receive today. You can build C.H.E.D.D.A.R. ads if you go to the right ad network, or install the right ad server. You can run a C.H.E.D.D.A.R. browser today, if you install (for example) Privacy Badger and Self-Destructing Cookies on Firefox.

All the pieces of C.H.E.D.D.A.R. exist, but they're just not integrated, branded, or made easy to install everywhere.

Are you already running a C.H.E.D.D.A.R. browser? Let's find out.


So, now all the JavaScript programmers have done a "View Source" on this page, and you're all like, wtf, that's it? A tracking protection detector and a check for a first-party ad element?

Yes, that's it. You can always write more refined versions of these, but the point is that you can do C.H.E.D.D.A.R. on the client without waiting for any new code on the server side, and you can do C.H.E.D.D.A.R. on the server without waiting for any new code on the client side.

Wouldn't C.H.E.D.D.A.R. be better if we added an extra layer of protocol, or a special HTTP header, or something? No, because no server can tell how a client is configured. If a browser or extension sends some future new intent message, it doesn't reliably tell the site if the user is also running a conventional ad blocker. Considering that ad blockers are the most popular browser extensions, it's likely that many people who install a "we welcome ads not based on tracking" extension will also have tried an ad blocker, and might not even remember they left it on, or not know they have to turn it off.

Actually testing for the delivery of a legit ad, or fake ad element, is necessary. Combine that with DNT and tracking detection, and you get a reliable "Just show me ads that aren’t based on tracking me" message.

That doesn't mean that C.H.E.D.D.A.R. is anywhere near done. If you want to build software around it, there are a lot of potential projects.

  • Better tracking protection and tracking detection

  • New ways to test that a DNT-respecting ad has been delivered to a human user

  • DNT-respecting, fraud-resistant web analytics

  • A "Just show me ads that aren’t based on tracking me" button in privacy tools and ad blockers.

  • C.H.E.D.D.A.R. detection built into web content management systems

The problem with web ads, legit and otherwise, is much like the problem of opt-in email newsletters and email spam. Somehow the idea of a "spam-free replacement for SMTP" never really caught on. Instead, we got:

  • Legit email is the kind of email that makes it through existing spam filters.

  • A good spam filter is the kind of filter that lets legit email through (but blocks "spam")

The first spam filters got started before legit email senders had to be concerned about deliverability—but because of spam filters, deliverability is big business today.

If you write a new spam filter, or set up an email service, you have to let through the mail that people agree is legit. If you start a new service that sends mail, you have to pass the existing spam filters.

Different services have different ToSs, but we can send and receive email as ToSs change, because they all reflect a common set of norms around what is and isn't spam. And we never actually have to agree on a common definition of "spam".

We'll never get the web advertising problem nailed down in precise legal and technical terms. There will always be a mix of old and new clients and servers, a variety of laws and norms, and new inventions and business models. Whatever we come up with will have to be messy, imprecise, and resilient in order to stand a chance.

Service journalism and the web advertising problem

There's a toenail fungus photo in my morning news.

And it looks like it's an ad for some questionable toenail-fungus-treating multi-level-marketing scheme.

Yeech. How did that get on there? Pass the ad blocker already.

Forget tracking protection, forget new standards for responsible advertising, forget all that. Gross infected body parts and MLM ads before I have even had my coffee? Burn all this stuff down.

Terrible ads are a big reason why tracking protection seems like an incomplete solution to the problems of web advertising. Web users don't just block ads because people are good applied behavioral economists, seeking signal and filtering noise. A lot of web ads are just deceptive, annoying, gross, or all three. (Oh, right, some of them carry malware, too.)

Even if we could somehow combine the efficiency and depth of the web medium with the signaling power of print or TV, won't web ads still be crap? And won't people still block them?

It doesn't have to be that way.

Publisher standards

Print ads are less crappy than web ads. Why can't publishers enforce better standards on the web? How can a newspaper have memorable, well-designed ads in print, while the ads on the web site have users looking for the computer sanitizer?

It's hard for publishers to enforce standards when an original content site is in direct competition with bottom-feeder and fraud sites that claim to reach the same audience. And that competition is enabled by third-party tracking. As Aram Zucker-Scharff mentions in an interview on the Poynter Institute site, the number of third-party trackers on a site grows as new advertising deals bring new trackers along with them. All those third-party pixels and scripts—and a news site might have 50 to 70 of them—cause slowness and obvious user experience problems. But the deeper problem, data leakage, is harder to pick out. Any of those third parties could be leaking audience data into the dark corners of the Lumascape until it re-emerges, attached to a low-value or fraudulent site that can claim to reach the same audience as the original publisher.

Publishers can try to pin down their third parties with contractual restrictions, but it's prohibitively expensive for a publisher to figure out what any one tracker is up to. You know that sign at the corner store, "only two high school students in the store at a time"? If the storekeeper lets 50-70 kids in, he can't see who shoplifted the Snickers bar. The news site is in the same situation on third parties. Because any one publisher has contact with so many intermediaries, only the perpetrators can see where data is leaking.

A security point of view

Information security is hard. When you have to maintain software, you fix a bug when you can see that there's a bug. You don't wait until someone starts exploiting it. The earlier you fix it, the less it costs.

News sites work this way for some issues. If you found a bug in your site's content management system that would allow a remote user to log in as "editor" and change stories, you would fix it. Even if you had no evidence that random people were logging in, it's not worth taking the chance. Because it's so hard to catch data leakage in the act, it makes sense to apply the same bug-fixing principle. When there is an emergent bug in the combination of your site and the user's browser that allows for data leakage, then it is more effective to proactively limit it than to try to follow audience data through multiple third parties.

That doesn't mean just walking away from all third-party tracking. Henk Kox, Bas Straathof, and Gijsbert Zwart write, in Targeted advertising, platform competition and privacy

We find that more targeting increases competition and reduces the websites' profits, but yet in equilibrium websites choose maximum targeting as they cannot credibly commit to low targeting. [emphasis added] A privacy protection policy can be beneficial for both consumers and websites.


If websites could coordinate on targeting, proposition 1 suggests that they might want to agree to keep targeting to a minimum. However, we next show that individually, websites win by increasing the accuracy of targeting over that of their competitors, so that in the non- cooperative equilibrium, maximal targeting results.

When publishers lack market power, they have to play a game that's rigged against them.

Changing the game

So how to turn web advertising from a race to the bottom into a sustainable revenue source, like print or TV ads? How can the web work better for high-reputation brands that depend on costly signaling?

C.H.E.D.D.A.R is a basic set of technical choices that make web ads work in a signal-carrying way, and restore market power to news sites.

Some of the work has to happen on the user side, but tracking protection for users can start paying off for sites immediately. Every time a user gets protected from third-party tracking, a little bit of competing, problematic ad inventory goes away. For example, if a chain restaurant wants to advertise to people in your town, today they have a choice: support local content, or pay intermediaries who follow local users to low-value sites. When the users get protected from tracking, opportunites to reach them by tracking tend to go away, and market power returns to the local news site.

And users see a benefit when a site has market power, because the site can afford to enforce ad standards. (and pay copy editors, but that's another story.)

Service journalism

Users are already concerned and confused about web ads. That's an opportunity. The more that someone learns about how web advertising works, the more that he or she is motivated to get protected. A high-reputation publisher can win by getting users safely protected from tracking, and not caught up in publisher-hostile schemes such as paid whitelisting, ad injection, and fake ad blockers.

Here is a great start, on the New York Times site. Read the whole thing:

Free Tools to Keep Those Creepy Online Ads From Watching You by BRIAN X. CHEN and NATASHA SINGER

The next step is to make it more interactive. Use web analytics to pick out a reader who is

  • valuable as an audience member

  • vulnerable to third-party tracking

  • using a browser for which you know a good protection tool

and give that reader a nice "click here to get protected" button that goes to your tool of choice. There is JavaScript to do this.

Tracking protection for users means fewer ad impressions available at bottom-feeder and fraud sites, which means more market power for news sites, which means sites gain the ability to enforce standards. Put it all together, and no more toenail fungus ads before breakfast.

Markets for intent data?

This is an extended version of a question that I want to ask the participants at VRM Day 2016.

An idea that keeps coming up is the suggestion that prospective buyers should be able to sell purchase intent data to vendors directly. I'm having trouble thinking about how this would work.

Here's an offline example. It's a summer weekend, and I'm walking through an anchor store at the mall, looking at khaki trousers.

Here are two pieces of intent information.

  • "I'm cutting through the store on the way to buy something else. I wonder if there are any decent clothes on sale, since I could probably use some extras."

  • "I ripped my last pair of pants and I have a meeting on Monday morning. I have a lot of stuff to get done and I'm not leaving this store without a new pair."

On a hypothetical intent trading platform, what's my incentive to reveal which intent is the true one?

My intent information is worth something to me as confidential information going into a negotiation. The value could be low (I'm just looking) or high (I ripped my last pair). If a vendor is willing to pay me some price for my intent information, then in order for me to accept it, that price has to be greater than the value of the intent information to me plus the transaction costs to me of selling the information.

If I'm "just looking" and don't need the product right away, I'm willing to sell intent information for almost any price. But it's of little value to the vendor, because it just tells them that my intentions are to only accept an incredible bargain. I even have an incentive to spoof my intentions. If I can convince a vendor that I'm not interested, I might get a better deal. And a fraud perpetrator has an incentive to simulate a serious buyer. It seems that in any market for user intent data where the user gets paid, wrong data will be over-represented.

So where's the market for purchase intent data?

I can think of a few possibilities.

  • Consultative sales: Some sellers are willing to do valuable work for me if I'm serious. Taking my measurements at a clothing store, or supporting an evaluation for a business IT product.

  • Controlled circulation: The basic exchange here is an old magazine model. Give a free magazine to people who do something to prove that they're in the market for a certain product. On the Internet, you can make this as fine-grained as you want. Give the user the ability to share some attribute in exchange for some content. (For example, a local news site might let you read the music and theater section free, if you can share the fact that you recently bought a ticket for a show—then the site's ads can command a higher price because they reach known buyers.)

It's not rational for users to "leak" purchase intent data without compensation, and therefore it's rational to block or spoof any kind of asymmetrical data collection. But are there special cases where a trade for purchase intent data can happen?

  • In consultative sales, the sales person can set priorities based on purchase intent—the more the customer appears likely to spend, the more time and other value he or she can get. There's no up-front payment for intent data, but an ongoing exchange of value for data.

  • In controlled circulation, the marginal cost of adding a subscriber is small. The 1,001st subscriber costs much less than 0.1% of the total budget to serve. This is different from directly paying for intent data, where all 1001 prospects (or 1001 copies of the same fraudbot) cost the vendor the same.

So I guess I'm still a Big Data optimist. Remember, when email marketing started, most of the people who used email for marketing were spammers. Today, most of the marketing email in the world is still spam, but most senders are legit, opt-in email newsletters work, and we have a set of technologies and norms to separate the two. But pay-to-spam concepts never really worked out.

Are there any other examples of how a market for intent data can work?

Bonus links

Let's make an acronym.

The IAB has come up with "L.E.A.N." and "D.E.A.L." for strategies to face down the ad blocking problem. But if that's all we do, we would be wasting a crisis here. Worse, we have the adfraud crisis happening at the same time, so we would be wasting two crises.

The big problem from the web publisher point of view is:

The same content brings in an order of magnitude less ad revenue on the web than in print.

From the advertiser point of view, that looks like:

The web is a low-value advertising medium.

Making changes around the edges to try to slow down ad blocking won't help that. Web advertising is still on the downward slope of the peak advertising curve that any targetable ad medium goes through. For example. the "E" in "D.E.A.L." is a weak link. Explaining how web ads work today is likely to build more interest in blocking. The more targetable an ad is, the more rational it is to ignore, block, or regulate it. It's only good behavioral economics to pay attention to advertising when the ad medium can carry a hard-to-repudiate signal.

We can't get web ads out of the ad blocking rut, but there are ways to make the web work as a low-fraud, high-signal medium and get it off the peak advertising curve entirely. Doc Searls writes,

For example, we could come with a term that says, Just show me ads that aren’t based on tracking me.

Good idea. We can take the qualities that next-generation web advertising must have, and make them spell out a word. Best if the word makes it clear that we're working on the core problem. We don't have an "ad blocking problem" and an "ad fraud problem". We have one problem, and ad blocking and ad fraud are two symptoms.

So, acronym. Right. Let me take a whack at it.

CNAMEs: Ads, and other third-party resources such as analytics scripts, served from what looks to the browser like a subdomain of the publisher's domain, not from a third-party domain that appears on multiple publisher sites. This is a small change for third parties, but a big barrier to cookie licking fraud. And responsible privacy tools won't block a dedicated subdomain that can't be used to track users across sites.

HTML5: avoid the malvertising risks of vintage plugins by using web standards only. Maintaining a reasonably secure device on today's Internet is hard enough. Users can't maintain problematic software just to see the ads.

Encryption: Limit the ability of ISPs and other observers to gather user data that can be used for targeting later.

Data leakage protection: Many users are still unprotected from web tracking. When appropriate, notify them and offer incentives to get protected. (This is especially important for brands in data-sensitive categories such as health care, and for high-reputation brands that compete with low-reputation ones in categories such as financial services and travel.)

DNT: Respect user norms on tracking across sites. (Update 26 April 2016: Show that you do this by hosting a copy of the EFF DNT policy on the ad server.) Respecting DNT is better than tricking users into giving up information, because eventually, users figure out what they're uncomfortable with and take steps to protect themselves. Meet the users where they are instead of trying to move norms.

Accountability: accurate WHOIS info for everything. No anonymous registrations. Rob Leathern explains this better than I can. Malvertising and fraud are too easy otherwise. (Update: important for publisher sites too because of the brand-supported piracy problem. Any real solution to brand-supported piracy depends on cleaning up both third-party tracking, to protect users from being tracked to infringing sites, and contact info for any site where an ad can appear.)

Reciprocity: Now you have an ad medium that's worth something to both ends. It restores the essential bargain of advertising: an offer of signal from the advertiser for attention from the audience. The result is an ad system that's harder for scammers to defraud, valuable for the advertisers who pay for it, and rational for users to accept.

So does C.H.E.D.D.A.R. work for you? Let me know, or just get started.

New BlockAdBlock-based method to detect tracking

Here's a new track.js script, based on BlockAdBlock.

If you are already using BlockAdBlock to alert users of "dumb" ad blockers (which is a good idea, because the best-known ad blocker gives users a false sense of security by participating in covert tracking) you can now use Aloodo with an almost identical interface.

Include the script with:

<script src=""></script>

And set up your callbacks with:

if(typeof aloodo === 'object') {
    aloodo.setOption('debug', true);

The onLoad function gets called when the fake tracker iframe loads, and the onDetected function gets called when tracking is confirmed. The difference is because of the problem of an "untrained" Privacy Badger. If Privacy Badger is installed but has not learned to block, then the onLoad function will get called even though the user has protection.


  • Use onLoad to correctly alert more users of list-based protection. (In this case you will have to let Privacy Badger users know that they can take a test to check their results.)

  • Use onDetected to avoid alerting untrained Privacy Badger users. (You will fail to alert some vulnerable users of list-based protection.)

Because Aloodo has to use a third party and wait for the iframe to load, this script can't be as fast as BlockAdBlock.



View source here or check out the project on GitHub for more info or to report a bug.