The effects of online advertising 3 Apr 2007

"Pop-ups were [rated] 24% more intrusive than in-line ads and pop-unders were 33.1% more intrusive than in-line ads."

"Subjects who were not exposed to ads reported they were 11% more likely to return or recommend the site to others than those who were exposed to ads."

"Subjects exposed to in-line ads remembered 3.4% more of the material in the site than those exposed to pop-ups."

"Designers should realize the magnitude of ill effects caused by advertising... Reducing the likelihood of a person's return by 11% might be a cost that is too great for a site host to bear."

These findings come from "The effects of online advertising" - a study by Scott McCoy (College of William and Mary) and colleagues, published in the March issue of the Communications of the ACM (subscription required to access the paper).

Greg Linden wonders "if websites that continue to annoy with pop-ups and pop-unders are only measuring short-term gains (e.g. clicks on the pop-up ad) and not long-term costs (e.g. loss in retention, higher abandon rate, lower return rate)."

Important: McCoy et al.'s findings are based upon what users said, not what they did - so it's not necessarily 100% true. However, it's still interesting that these design elements provoked such a strong response from those participating in the study. (It's also worth noting that other studies - including this one by Jakob Nielsen - produced a very similar set of findings.)