It’s that time of the year again. It’s not only a good time to indulge one’s favorite treats; it is also the perfect time to look back at 2017’s best life hacks. Such little “nudges” could perhaps help to avoid the weight gain that often comes along with the feast (Helander et al., 2016). The nudging idea is so powerful that it earned Richard H. Thaler a Nobel prize this year, so this post will be quintessential 2017 science.
REVEALED: Why eating cereal from a SQUARE bowl will make you feel fuller
I admit, that’s not my headline so credit to express.co.uk for having the courage to always reveal nothing but the truth. But they were not the only media outlet to jump on the bandwagon. Here is a collection of my three favorite headlines surrounding the study:
Eating From a Square Cereal Bowl While Standing Is a Weight-Loss Secret, Study Shows
Eat standing up and NEVER have cereal in a round bowl: The formula for the perfect breakfast revealed (and it could make you feel fuller and more satisfied)
CEREAL THRILLER You’ve been eating your breakfast wrong… and making this one easy change will make it taste better
Seems pretty amazing to me. Let’s have a look at the details of the study because we believe in now-forbidden phrases such as “evidence based”. Often, such press releases lack the necessary information to thoroughly evaluate a study, so we’ll have a look at the paper instead.
Ah, okay, there is no paper on this fabulous study. What a bummer. But I am sure there is a report or a preprint or something alike backing up the results that are telling us we’re having breakfast all wrong?
Nope, nothing online. It could be work in progress. I guess I should wait for half a year and check again while desperately monitoring the top nutrition/obesity journals to finally see the study published.
Hmm, still no proper report in November? Maybe there is a delay or the authors are still polishing the draft. The principal investigator will certainly be happy to share the draft with fellow researchers by now.
Or maybe not. Sorry, I have reached out to the PI, but sometimes silence is golden I guess. We are left with the information scattered throughout the press releases because there is nothing else.
CONCEALED: Leading a very nutritious fishing expedition leads you nowhere
Now that is my headline. I’ll put the pieces together for you.
Design: “a two-day experiment with 78 women” dailymail.co.uk; “aged 18-65”; “All participants were given 30g of cereal, a 125ml of cold semi skimmed milk, and a stainless steel spoon.”; express.co.uk
Rationale: “to find out which way of eating breakfast left participants fuller and more energized” dailymail.co.uk
Conditions: “They were then asked to eat the cereal in one of 10 different ‘eating contexts’. Some were given round bowls, some square.“; express.co.uk
Main outcomes: “When they had finished, they were given detailed questionnaires about how nourished they felt, how the cereal tasted and how it had affected their mood. Their bowls were weighed to see how much had actually been consumed in each case.” express.co.uk
Here comes a sentence that I’ve never seen myself typing: hats off to the Express Newspapers for providing the most detailed record on this seminal research to date.
If this does not illustrate the mess of a study, then let’s break it down further. We have N=78, a between-subject design (I guess? Was it two-days in total or two days per participant, by why then only one condition?) and 10 different conditions. One of them was eating from a square bowl, one was eating from the regular “fattening” square bowl, one was eating while standing up, one was eating after 10 min of mindfulness training, one was eating in a calm atmosphere, one was eating with noise and distraction, one was with coffee, and we are still not quite there. This study was not suffering from a lack of creativity.
REVEALED: factorial designs only RESTRICT an author’s creativity to spot a signal
Okay, I’ll stop now. What becomes clear from this list: this was not a factorial design, where specific factors were systematically manipulated to study their effects and potential interactions. A factorial design is something you would want to do if you had an idea what you are actually looking for. In this study, we are left with 7-8 participants per cell (assuming random and balanced assignment to conditions). If we compare only two of the 10 conditions that had (luckily) 8 participants assigned to them, the power of this design would be appropriate to investigate effects with a minimum Cohen’s d > 1.5. Without any correction for multiple comparisons.
James Heathers kindly collected some strong effect sizes as benchmarks for comparison. We’re already in the range of differences in height between men and women within a population, or in anxiety between patients suffering from generalized anxiety disorder vs. healthy controls. I’d call that optimistic to begin with, but I have been rightly accused of my unsubstantiated skepticism towards square-bowl interventions against obesity before.
But there are more problems that make me want to scream. For example, there is no obvious control condition. One could calculate all 10 conditions × 9 other conditions simple pairwise comparisons = 90 tests. Again, this is per outcome. We don’t know exactly how many outcomes there were in total, but we read about leftover cereal, perceived taste and fullness as part of a questionnaire, so we can safely assume that the number of potential tests exceeds 270. Without any sound hypothesis or appropriate statistical design, what we’re left with in this study is essentially a wansink full of nutrition nonsense.
Another bad story about nutrition, so what?
I understand the indifference that we feel when we read stories reciting studies that obviously lack any plausible theory or mechanism. We got used to the way how bad science is covered widely in the media such that the public knowledge becomes largely disconnected from the scientific debate. To get a story published, you just need a minimum of scientific appeal: some sort of data, a Dr. you can interview, a prestigious university (not any university in this case, it is Oxford!) and a surprising result to make the headline. Perhaps I shouldn’t bother.
But it bothers me. It bothers me when a chef is using dopamine to science up his unoriginal new diet simply to sell more books while we are at the same time struggling to secure the financial support to run proper studies on these topics. This research is not a trivial endeavor. Should one even blame funders for being misled to believe that either many things are obviously known or, alternatively, this is such a huge mess that we can’t even come up with a single well-grounded hypothesis based on this hodgepodge? All the good science is quickly swamped in the same-old, same-old. Even I can’t change any of that, I am also not willing to silently agree to that bad practice.
Based on what premise are such nudges supposed to work against obesity?
Back to the idea of simply nudging your way to a slimmer body. Let’s suppose for a minute that you’d truly eat less cereal if it comes in a square bowl and you were asked to eat it while standing up. Maybe you save 50 kcal using this technique. We can do even better than that. Instead of a spoon, you will eat the cereal with a knife (don’t try this at home without having a graduated experimenter around!). In this case, you’d save a stunning 100 kcal and take a mindful lot more time to finish eating. If we do the math now, you could shed a pound if you keep doing that for a month (the often cited ~3500 kcal rule is an oversimplification, but a useful guestimate for our purpose; (Hall, 2008)) all things being equal.
Is that a realistic assumption? Of course not, because you’ll likely compensate for the deficit in energy intake in one way or the other and your metabolism will also be affected by changes in food intake (Hall et al., 2011). Bottom line: Any two-day study will be insufficient to tell us if a nudge can support weight loss to a meaningful extent. Eating less food or feeling a little more sated in one instance will not suffice in making a difference, unless there is a known neurobiological mechanism that we could expect to maintain it on a long-term basis.
I know I am reiterating the limits of a nudge because the message is powerful, yet deceiving at the same time. I often heard people say they had “tried EVERYTHING and nothing has worked so far” referring primarily to very obscure fad diets or nudges. Obviously, companies are trying to make a quick dime from the hope for fast and effortless solutions. But again, we shouldn’t silently accept this as a given. Nutrition is a important topic and we need to be open and clear about what works and what does not.
Hall KD (2008) What is the required energy deficit per unit weight loss? Int J Obes (Lond) 32:573-576.
Hall KD, Sacks G, Chandramohan D, Chow CC, Wang YC, Gortmaker SL, Swinburn BA (2011) Quantification of the effect of energy imbalance on bodyweight. The Lancet 378:826-837.
Helander EE, Wansink B, Chieh A (2016) Weight gain over the Holidays in three countries. N Engl J Med 375:1200-1202.