A trade group for liquor retailers put out a press release last week with an alarming headline: "Millions of Kids Buy Internet Alcohol, Landmark Survey Reveals." Are millions of kids really buying booze online? To arrive at that jarring headline, the group used some questionable logic to pump up results from a survey that was already tilted in favor of finding a large number of online buyers.
The announcement, from the Wine and Spirits Wholesalers of America, received wide media attention. On NBC’s Today Show, Lea Thompson said, "According to a new online survey, one in 10 teenagers have an underage friend who has ordered beer, wine or liquor over the Internet. More than a third think they can easily do it, and nearly half think they won’t get caught." Several newspapers mentioned the study, including USA Today and the Record of New Jersey. The news even made Australia’s Gold Coast Bulletin.
For starters, consider the source. The trade group that commissioned the survey has long fought efforts to expand online sales of alcohol; its members are local distributors who compete with online liquor sellers. Some of the news coverage pointed out that conflict of interest, though reports didn’t delve more deeply into how the numbers were computed.
The Wine and Spirits Wholesalers of America hired Teenage Research Unlimited, a Northbrook, Ill., research company, to design the study. Teenage Research, in turn, hired San Diego polling firm Luth Research to put the questions to 1,001 people between the ages of 14 and 20 in an online survey. Luth gets people to participate in its surveys in part by advertising them online and offering small cash awards — typically less than $5 for short surveys.People who agree to participate in online surveys are, by definition, Internet users, something that not all teens are. (Also, people who actually take the time to complete such surveys may be more likely to be active, or heavy, Internet users.) It’s safe to say that kids who use the Internet regularly are more likely to shop online than those who don’t. Teenage Research Unlimited told me it weighted the survey results to adjust for age, sex, ethnicity and geography of respondents, but had no way to adjust for degree of Internet usage.
Regardless, the survey found that, after weighting, just 2.1% of the 1,001 respondents bought alcohol online — compared with 56% who had consumed alcohol. Making the questionable assumption that their sample was representative of all Americans aged 14 to 20 with access to the Internet — and not just those with the time and inclination to participate in online surveys — the researchers concluded that 551,000 were buying alcohol online.
But that falls far short of the reported "millions of kids." To justify that headline, the wholesalers’ group focused on another part of the survey that asked respondents if they knew a teen who had purchased alcohol online. Some 12% said they did. Of course, it’s ridiculous to extrapolate from a stat like that — one buyer could be known by many people, and it’s impossible to measure overlap. Consider a high school of 1,000 students, with 20 who have bought booze online and 100 who know about the purchases. If 100 of the school’s students are surveyed at random, you’d expect to find two who have bought and 10 who know someone who has — but that still represents only two buyers, not 10. (Not to mention the fact that thinking you know someone who has ordered beer online is quite different from ordering a six pack yourself.)
Karen Gravois Elliott, a spokeswoman for the wholesalers’ group, told me, "The numbers are real," but referred questions about methodology to Teenage Research. When I asked her about the potential problems of conducting the survey online, she said the medium was a strength of the survey: "We specifically wanted to look at the teenage online population."
Nahme Chokeir, vice president of client service for San Diego-based Luth Research Inc., told me that some of his online panel comes from word of mouth, which wouldn’t necessarily skew toward heavy Internet users. He added that some clients design surveys to screen respondents by online usage, though Teenage Research didn’t.
I asked Michael Wood, a vice president at Teenage Research who worked on the survey, whether one could say, as the liquor trade group did, that millions of teenagers had bought alcohol online. "You can’t," he replied, adding, "This is their press release."
Another recent Teenage Research Unlimited survey, also conducted through Luth, received widespread press coverage. The study of women’s purchasing inclinations, sponsored by the Oxygen Network and called "Girls Gone Wired," found that "the majority of women are hungry — even voracious — for technology," according to the press release. It said that "77% would even prefer a new plasma TV to a diamond solitaire necklace!"
This survey also was fielded online, so a disproportionate number of respondents may have been technophiles.
But the survey was slanted in favor of technology in a more fundamental way: It included only females aged 15-49. Women 50 and older — who numbered 48.4 million in July, according to a Census Bureau estimate — might feel they have more of a claim to womanhood than 15-, 16- and 17-year-olds, yet they were excluded.
"We certainly did not mean to be misleading," Oxygen spokeswoman Kassie Canter told me. She said that in discussions with media outlets, she emphasized the limited age range. "We’re very open with what our methodology was." USA Today and Reuters identified the age range studied, but only after leads that described the preferences of "women" without clarification; PC Magazine omitted the age range.
In a column last year, I criticized a public-service ad campaign that said "one in five children has been sexually solicited online." That stat came from a 1999 survey, funded by the advocacy group National Center for Missing and Exploited Children. I felt the ads oversimplified the issue in an unnecessarily frightening way, and relied on a definition of sexual solicitation that was too broad.
In an updated survey released last week (and again conducted on behalf of the advocacy group by the University of New Hampshire), researchers found that one in seven children reported being sexually solicited online, a marked decrease, though other threatening activity — such as online harassment — increased.
The National Center focused on the bad news in its headline, which stated, "Youth Online Exposed to More Sexual Material and Harassment." The center’s president, Ernie Allen, told me, "I’m encouraged that the overall numbers of kids solicited online has declined." He said his group likely would use the one-in-seven stats in new public-service ads. "There’s no real magic in the number," Mr. Allen said. "We understand going in that this is based on survey research."
Yet the news is even better than it seems on the surface, because children are spending a lot more time online now than they did in 1999. Just 13% of respondents in 1999 said they used the Internet for two hours or more each day they were online; that proportion was 23% in the new survey. Yet a smaller proportion of online kids are being sexually solicited.
David Finkelhor, director of the Crimes against Children Research Center at UNH and one of the authors of the studies, said that several peer-reviewed papers currently being submitted would examine more closely the results when controlled for time spent online. He had a reasoned analysis of the results: "We need to be concerned about kids getting into difficulty online because it’s a changing, fluid environment where there are risks, but I don’t think we should jump to the conclusion that the migration of kids onto the Web has made kids more at risk than they were before. We don’t know that. In fact, the evidence suggests that’s not the case."
