Copywriters are Not Always Scientists

This week’s Participatory News assignment is to fact-check a dubious claim found out in the wild.

I enjoy skimming Men’s Health magazine each month. It combines useful nutritional information, workout plans, and the occasional life advice about topics like careers and personal finance. More than anything, the ever-present photos of salmon and blueberries remind me to buy and eat such foods.

The publication is guilty, however, of a classic media tactic: They publish an incredibly short summary of a recent study, and then tack on their own advice on how to work the findings into your life. To be fair, I’ve seen many newspapers and magazines use this formula. It’s the natural result of cramming science journalism into 2 sentence blurbs. But there are a few big problems with this format.

First, a single study is rarely, if ever, enough evidence to warrant a change in behavior. Anyone who’s ever read surprising results in a study and then outright laughed upon getting to the Methodology section understands why. Many studies are conducted on twenty graduate students, or actual lab rats, or the design of the study is clearly biased towards the eventual results. I’m not the kind of guy who reads studies for fun, but these problems are fairly obvious when you read the original paper. It’s one reason scientists wait until there is clear agreement across a variety of research before advising action. It’s also why they always seem to conclude a study with the line “Further studies are needed” (besides the fact that “further funding is needed”). As one example, the very study we’ll be looking at substantiates its claims regarding what men and women seek in a mate only after finding consensus among “studies that have spanned 20 years and often include international datasets with sample sizes in the tens of thousands.” When a magazine summarizes a single study in a single sentence, we aren’t provided nearly enough context to think critically about how the results were achieved. We’re just told to believe them.

Even lazy science journalism serves a purpose. The publishers take a dense academic study and make it accessible for a mass audience. And perhaps by interpreting the results into actionable advice, they believe they’re helping the lay reader. What they’re often doing, however, is making unsubstantiated claims on the back of a scientific study (itself of varying quality and methodology). By including their own advice in the same small paragraph as the peer-reviewed study, the publication encourages its reader to extend the scientific study’s credibility to the copywriter’s tidbit.

This is all fine and good when we’re talking about new strategies for bigger biceps. The advice regarding how to interact with women, however, borders between wildly misogynist and downright hilarious. We could probably solve overpopulation if every man did what Men’s Health says to do, and every woman behaved as Cosmo advises. I’ve known how silly these prescriptions were since, oh, puberty, but I thought I would take the opportunity of this week’s fact-checking assignment to look into some of the advice and see what the studies referenced actually found.

The magazine has a regular feature dedicated to relations with women, titled SEX BULLETIN (to be fair, there are also more thoughtful, longer pieces about women). Five findings are presented on one page:

Men's Health Sex Bulletin

  1. “The Last Longer Jab”…”An injection of hyaluronic acid gel in the head of your penis” may delay premature ejaculation “by about 6 minutes.”
  2. “Women fake orgasm to keep men from straying”…”That phony “O” may be her attempt to show she’s committed and reduce the chance that you’ll cheat”…”Other ways she reels you in: flirting with guys in front of you and calling at unexpected times.”
  3. Three percent of women say using lube makes them feel inadequate.
  4. “Women are less likely to regret random hookups if the sex was satisfying.” (This advice may be provided to obscure the well-documented research establishing that in general, women are more sociosexually restricted, or less desirous of casual sex than men (Schmitt, David P., 2005)).
  5. And lastly, the one fact I might reasonably be able to check:

Wanted: Self-Made Millionaire

Ninety percent of women prefer a long-term partner who earned his money rather than inherited it. “Women associate self-earned wealth with reliability, self-sufficiency, intelligence,” says study author Peter Jonason, Ph.D. If you have family money, play up your generosity– say, donate to charity. That shows you don’t take wealth for granted. (emphasis mine)

The study referenced provides a quote from its author, and no other identifying information. We’re lucky. In other examples, the factoid in question is followed by vague statements like “according to Japanese research.”

Taking a closer look at the study referenced, the results come from a sample size of 145 women. More importantly, the abstract concludes that “In sum, financial security appears to have minimal effects and associations on mating psychology despite the paramount role that sociocultural psychologists argue it has” (Jonason, ix). So, the study’s author is actually arguing the exact opposite of Men’s Health selective quotations: Financial security was found to have less of a role than has traditionally been argued.

I wasn’t able to find any scholarly work regarding Men’s Health‘s advice that men should make up for inherited wealth (and a lack of reliability, self-sufficiency, and intelligence) by donating to charity and then telling women about it. Perhaps we can design such a study. Just don’t ask to see the methodology.

Posted in Uncategorized

Fact Checking Food Advocacy

This week’s assignment invited students to analyse the rhetoric of a public claim and check the facts. The issue I picked was a very popular petition on Change.org (and related media) to encourage the United States Department of Agriculture to stop using a certain kind of beef product in school lunches.

Exploring the issue involved:

  • Learning about different kinds of meat processing
  • Studying recent history of food safety concerns
  • Reading 10-15 articles, and watching several videos
  • Reading USDA food certification records
  • Learning about the US National School Lunch programme
  • Tracing claims through the churn cycle of news media to find to find the original source
  • Trying, unsuccessfully, to reach various people in the press as well as the source for a claim
  • Figuring out the agenda of various sources, whether trade magazines or activists
  • Deciding how deep to go into an issue when fact checking. At a surface level, a lot of claims appear very flimsy. Verifying a claim in the news is especially difficult when something has little evidence to support it. One feels drawn to prove or disprove something that might not be proveable.

Technology Ideas

Checking facts is really hard work. Not a lot of it can be split up or optimised. Here are some thoughts:

  • Tools like Churnalism.com can help fact-checkers identify sources. One can imagine a similar version which could track a quote through media to find the earliest case.
  • It would be awesome to be able to know the origin or funders of a website when you’re looking at it.
  • Part of fact-checking involves doing back-of-the-envelope thought experiments to evaluate the feasibility of a claim. These could be handed off to someone else.
  • At Texperts, we employed a large team of expert web researchers with a variety of complementary research competencies. A similar distributed team of workers could be very effective at fact-checking.
  • If they are able to attract enough interest, Hypothes.is offers a fascinating model for social annotation of things that people mistrust or want to check

My Article: Is Pink Slime Going to Kill Us All?

Has “pink slime” invaded the American food supply, where it is spreading bacteria and contributing to malnutrition? Does 70% of all American supermarket beef contain this allegedly harmful substance? Or is this just a food scare created by activists who have been irresponsible with the facts?

A recent Change.org petition by children’s food blogger Bettina Elias Siegel has received over two hundred thousand signatures in a week’s time. Bettina is urging the United States Department of Agriculture, who purchase food for the National School Lunch Program, to stop buying meat pcontaining “boneless lean beef trimmings,” primarily produced by the company Beef Products.

In this post, I’m doing three things. Firstly, I’m going to analyse the rhetoric used by “pink slime” campaigners to get mechanically recovered beef off the menu. Secondly, I’m going to look into the context of the issue. Finally, I’m going to investigate Bettina’s claim that 70% of all supermarket beef includes so-called pink slime.

Understanding the Campaign Rhetoric

Looking at the Change.org petition and Bettina’s blog, here are some of the prominent persuasive tactics deployed by the campaign:

  • Control the language: “pink slime” is much easier and more interesting to write than “ammonia-treated lean beef trimmings.” It’s also a terrifying notion, one which is very likely to collect a high number of clicks.
  • A disgusting photo on the change.org petition features regular ground beef rather than “lean beef trimmings.” It’s the basic ground beef photo on Wikipedia, and it is often reused by “pink slime” activists. The other frequently-used image appears to be of chicken, not beef. Actual photos of “pink slime” look more appealing.
  • Obvious bad guy: The “Pink Slime” campaign has a single target: Beef Products Inc.
  • Simple change: In addition to convincing supermarkets and fast food companies to give up lean beef trimmings, this petition calls on the USDA to stop purchasing it.
  • Appeal to instinctive disgust: The petition goes further than using the language of “slime.” It argues, “it is simply wrong to feed our children…scraps that were, in the past, destined for use in pet food.”
  • Please think of the children: It’s easier to convince the USDA to stop purchasing lean beef trimmings than it is to convince them to rate the product as unsafe. Since the USDA does the school lunch program purchasing, it’s perfectly appropriate to ask us to think of the children. It also happens to be an incredibly powerful persuasive tactic.
  • The fact they’re hiding from you: Betinna and others often repeat the claim that “70% of the nation’s ground beef” contains pink slime. By citing a number of that size, activists convince us that we, too, should care.

Context

The backstory of this issue can be found a December 2009 New York Times article by investigations reporter Michael Moss which exposed broad lack of testing by the USDA of ammonia-treated meat, a lack of testing which led to cases of e. coli and salmonella in the food supply. Moss’s article, The Burger That Shattered Her Life went on to win a Pulitzer for Explanatory Reporting.

Looking over the USDA’s Advanced Meat Recovery documents, the story of lean beef trimmings has been an ongoing cycle of testing the wrong thing and trying to correct oversights. In the 1990s, Advanced Meat Recovery techniques were developed to reduce the amount of bone included in meat extracted from scraps down to levels safe for human consumption. AMR did this successfully. Then there were concerns around spinal tissue in the scraps, which led to a new set of tests. More recent are concerns around the use of ammonia to increase the alkalinity of the meat, killing bacteria. If the ammonia gets too low or the system stops working, bacteria can get into the food supply, especially where lean beef trimmings are mixed with other meat.

70%: how a speculation becomes fact

Bettina and others point to a report by ABC news that 70% of ground beef in supermarkets contains “pink slime.” It’s a number I have seen repeated frequently by campaigners. Where does this “fact” come from? ABC cites former USDA researcher Gerald Zirnstein, who coined the term “pink slime” in 2002 and opposed USDA approval.

Zirnstein offers no evidence to support his claim, and he’s been gone from the USDA since at least 2004. Nevertheless, Zirnstein’s figure has been quoted in The Huffington Post, Fox News, Mashable, the New York Times, The Daily Beast, Yahoo, Jamie Oliver’s Food Revolution, and CNN.

Zirnstein could not be reached for comment. ABC did not respond to emails or tweets.

The Beef industry has started to push back. Beef Daily, a beef industry blog by Penton Media, has called pink slime a myth. Beef Products Inc has created the website Pink Slime is a Myth, featuring informational blog posts and video interviews which explain the process of making Lean Beef Trimmings. The American Meat Institute, a meat and poultry trade organisation, has created the website Meat MythCrushers, which also features informational videos, resource guides, and information from food industry experts. The American Meat Institute has also posted a Questions and Answers PDF. Meat industry supporters have also tried to rally opposition through the #pinkslimeisamyth hashtag on Twitter, to limited effect.

Unsuccessful back of the envelope calculation: Is pink slime as widespread as critics claim? Whole Foods and Costco do not stock it at all. Organic labeled beef does not include it. Let’s do some math. USDA regulations require that no more than 15% of ground beef contain “pink slime.” The total US beef consumption in 2010 was 26.4 billion pounds. BPI produces 7 million pounds of product a week, for an annual total of 364 million pounds a year. According to the USDA, the US exports 2.3 billion pounds of beef a year. According to an Oklahoma State University brochure….. (at this point, I had to stop because it appears to be impossible to know how much ground beef is sold annually in the US).

Should we be worried?

The real dangers here seem to be e. coli and salmonella, both of which can be killed by properly cooking your food. Although food including lean beef trimmings will be much less tasty and nutritious, there doesn’t seem to be any immediate health risk for adults or children.

More broadly, the inclusion of lean beef trimmings into other meat offers a worrisome potential for contamination, especially if companies continue to enjoy the level of immunity from testing and labeling which the New York Times revealed in 2009.

Posted in Uncategorized

MAS S61: assignment #4

On March 12, 2012, Republican presidential candidate Mitt Romney responded to President Barack Obama’s progress report on the Blueprint for Secure Energy Future. Rather than take the White House’s infographics on gas prices as gospel, I checked the data from the U.S. Energy Information Administration’s historical gas prices to verify Mitt Romney ‘s claim “with gas prices setting new records.” and posted the findings below:

Posted in Uncategorized

The Journalism Innovation Spiral: A Method for Journalism Innovation Design

How can designers imagine innovative technologies for news and journalism? I think I know the answer. In this post, I propose a model and demonstrate it by picking apart the “Profile article” for innovative ideas. The end result is a browser plugin which can attach blogging tools to any text form on the web.

http://civic.mit.edu/blog/natematias/the-journalism-innovation-spiral-a-method-for-journalism-innovation-design

Posted in Uncategorized