The Atkins Diet and the scientific method

In the comments, matt asks:

We can write off any temperature change as as just a shift in global temperature patterns if we want to. The problem with such a method is that since we don't have old enough to data to even grasp what a cycle might look like, broadly (according to some) we couldn't hope to understand what a human-made change might look like. This is an argument I use on theists all the time: what would it take to convince you that your beliefs are false- i.e. that global warming is real?

That is a very important question, and one that very few people are asking, much less answering. I will give my answer in the near future, but in the meantime, I want to ask everyone a similar and hopefully easier question.

We have all heard of the Atkins Diet, and I am sure that many of us have been intrigued by it. I think most of us accept the proposition that taking in less calories than your body expends will result in weight loss. The further claim that Atkins supporters make is that the low carbohydrate Atkins Diet is more effective at weight loss than conventional dieting.

The great thing about empirical science is that you can give yourself your own standard for believing something before carrying out your experiment. Here is my question. You, a human being trapped on this crusty rock, want to gain knowledge about the universe. Suppose that specifically, you want to know if the claims made by the Atkins supporters are true. How, in very specific terms, would you carry out an experiment to determine if the Atkins Diet 'works'? And what kind of results from this experiment would it take to convince you that it 'works'?

Share this

The question that Matt poses

The question that Matt poses is somewhat of a red herring.

I already believe that global warming is real. The problem is that I don't at all agree with the conclusions drawn by the IPCC and the enviro-illuminati that says "ergo we must have global socialism NOW!"

Even by the IPCC's estimates, Kyoto only buys us 6 years before the temperature hits what it would hit without action. Trillions of dollars and I expect more than 6 years. I expect amelioration, which is the (dishonest) claim of Kyoto proponents, that if one follows the treaty, we will tackle our problem.

The real question is- how is one going to deal with global warming? If the sea level is going to rise, what does one do about that? Replying that "one must reduce emissions" is a frightening non-sequitur. I wonder if they'll sign us up for fiddle lessons, too, when they foresee Rome burning.

The answer to global climate change is to deal with it- seawalls (in the short term), carbon sequestration, possibly putting a solar shade over the poles to artificially chill them, etc etc. Science fiction in some respects, but its at least addressing the problem. Right now, conservatives and libertarians rightfully see Kyoto as a stalking horse for more radical socialist plans in the future, since it cannot work (and will not, even by IPCC's admission).

That's great, but save your

That's great, but save your thunder for next week and quit trying to hijack my thread. :P

This is about Atkins, not global warming.

Jonathan, This is a

Jonathan,

This is a straightforward study: Professor X gets grant money, puts up ads seeking participants in weight-loss study. Participants get randomly assigned to 2 groups, Atkins and non-Atkins. They aren't told which group they're in, and the RAs collecting the data aren't told either ("double blind"). Diets are then implemented. Wait for some time period to pass. Then measure weightloss results, and test to see if they're statistically different.

Of course, even if we find that "Atkins works", that doesn't change the fact that eating meat three meals a day is *disgusting*, on all kinds of levels.

Actually, this is a good

Actually, this is a good (and unexpected) segue from climate change. The argument is in some ways similar: Atkins skeptics (the mainstream) argue that carbohydrates behave in quite a simple linearish manner, being metabolized more quickly than fat, no? Atkins proponents contend that there are all sorts of higher order effects that we don't understand, but that it is likely that carbohydrates are worse for us than dietary fat. (I am working off the top of my head here, so please correct me if I get it wrong.)

The bottom line is still, with Atkins as with climate change: we're lacking a theory here. Until that's resolved, the argument is going to continue.

Even if I was considering the Atkins diet (which I'm not), the prospect of ketone breath is far too horrible to contemplate...

Brian, my question was

Brian, my question was intended for "human-influenced global warming" skeptics. If you accept that argument (which, incidentally, I do if I haven't already said that) then my question doens't apply to you in this way. Though it's a good question generally, I think.

Honestly, I've only heard of the Atkins diet in passing before, and I know almost nothing about nutrition. So I'm having a hard time even starting to answer this question.

As for Ketone breath, though- it's really wierd that people would call that "bad." My experience with Ketone breath (diabetic ex-GF)has been oddly pleasant- it's faintly sweet-smelling.

Andrew, A couple of question

Andrew,

A couple of question -
What does the 'non-Atkins' group mean?
What is an RA?

I think the point you are

I think the point you are trying to make Jonathan is that there is no way to completely control for all variables when using the scientific method (i.e, controlled experimentation). Thus, if a study was run where you controlled one group's diet to follow Atkins recommendations and then allowed the other group to eat a diet you had designated to be somewhat normal, how do you know that some other factors that you did not control for did not influence the results?

However, my point would be that although there is no true method to completely control for all factors, there is a better probability that you have come closer to the truth than before doing the experiment.

I thought the main

I thought the main controversy surrounding Atkins was not whether it works, but at what cost - i.e. what are the side effects. Most people agree that it is good for weight loss, but if it makes your hair fall out and caused other nasty and harmful things, it might not be such a good thing afterall.

Reason had an article a few months ago severely criticising Atkins. I'm still withholding judgement.

mgl, The bottom line is

mgl,

The bottom line is still, with Atkins as with climate change: we're lacking a theory here. Until that's resolved, the argument is going to continue.

I think that is less important than most people realize. As I said in a prior post, there is a different between the scientific method, which is used to make observations, and the philosophy of science, which is a way to make theories out of those observations.

For example, we know that having people breathe isoflurane will cause them to pass out, and anesthetics such as this are used thousands of times every day to perform surgery. Yet, nobody *really* knows how anesthesia works. The scientific method shows that isoflurane 'works', although the philosophy of science has not yet developed a convincing theory of what the exact mechanism is.

In the same fashion, the theory behind the Atkins diet is irrelevant to whether or not it 'works', and a lack of a theory should not preclude the determination of whether or not it 'works'. And that is one of the beautiful aspects of the scientific method.

matt, Honestly, I've only

matt,

Honestly, I've only heard of the Atkins diet in passing before, and I know almost nothing about nutrition. So I'm having a hard time even starting to answer this question.

I don't think you need to know much about the Atkins diet to design a study to see whether it works or not.

Barney, I think the point

Barney,

I think the point you are trying to make Jonathan is that there is no way to completely control for all variables when using the scientific method (i.e, controlled experimentation). Thus, if a study was run where you controlled one group's diet to follow Atkins recommendations and then allowed the other group to eat a diet you had designated to be somewhat normal, how do you know that some other factors that you did not control for did not influence the results?

I think that you can come darn close to having a true control in studying diets.

However, my point would be that although there is no true method to completely control for all factors, there is a better probability that you have come closer to the truth than before doing the experiment.

Probably so.

With the exception of

With the exception of Andrew, nobody has put forth a description of an experiment by which to study the Atkins Diet. (And Andrew gave a great design).

I am really not concerned with the controversy behind it or the side effects, but rather the method by which one might hypothetically determine whether or not it 'works'.

An addendum to Andrew's

An addendum to Andrew's experiment: don't mention that it is the Atkins diet you are testing, else it may skew the results. (For example, people who dislike Atkins from the outset may "cheat", while people who want Atkins to succeed may stick to the diet more stringently than normal, eat less, and excercise more.)

When assigning participants to the two (or more) groups, first put them through a sort of "jury selection," asking whether they have ever heard of Atkins (and other diets, just so they don't know that is the one you are testing), whether they have ever used Atkins, and so on.

I actually think

I actually think understanding Atkins is essential to designing such an expiriment. Just from catching a few tidbits that people mentioned about atkins, I can point out problems. For instance "not telling them which group they are in" (Andrew) is silly when the expiriment itself involves eating normally vs. eating meat three meals a day! Who's not gonna notice that? You have a placebo problem.

Bad Breath caused by the diet could heavily affect other variables- sex being an obvious one, but also the consumption of sugared gum, as mentioned in the FAQ page as potentially problematic to the effects of the diet.

Generally, I agree with Andrew's expiriment, but I'd like to make it more specific (as you asked) with details like these.

I actually think

I actually think understanding Atkins is essential to designing such an expiriment. Just from catching a few tidbits that people mentioned about atkins, I can point out problems. For instance "not telling them which group they are in" (Andrew) is silly when the expiriment itself involves eating normally vs. eating meat three meals a day! Who's not gonna notice that? You have a placebo problem.

What placebo? As far as I can tell, Andrew's experiment did not call for a placebo.

Bad Breath caused by the diet could heavily affect other variables- sex being an obvious one, but also the consumption of sugared gum, as mentioned in the FAQ page as potentially problematic to the effects of the diet.

This may all be true, but things like knowing the mechanisms behind metabolism, and physiology is absolutely not essential in determining whether or not Atkins works.

Johnathan, Your null

Johnathan,

Your null hypothesis is "Atkins works no differently than non-Atkins diets." That's what "non-Atkins" group means. They're the ones who get some other diet, presumably one that includes carbos so you can test the Atkins claim that low-carbo diets are better.

"RA" is short for research assistant. They're the ones in grad programs who do the grunt work collecting data in these kinds of studies.

By the way, the experiment I described is the standard double-blind study method used to test everything from drug effectiveness to the benefits of physical therapy. There are lots of details to fill in, of course. But in general, that type of experiment is the standard scientific method for resolving claims like this, and the journals are full of them. Any intro stats book will describe experiments like this.

In fact, I'd be surprised if nobody has done this study yet, given what's economically at stake for those pitching Atkins, or opposing it.

Andrew, Thanks for

Andrew,

Thanks for clarifying. I was just trying to figure out what you were using as your control group.

If by 'works', one means that people lose weight on Atkins diet, then the control group would be people who do not diet.

If by 'works', one means the low carb Atkins Diet is superior to other conventional diets, then the control group would be people who go on conventional diets.

If we are using the latter as the control group, than it is importnat that the total calories eaten by people each group should be the same, even though the makeup of those calories is different.

Another question for you - how would you double blind the study? The biggest problem would be that people know when they are eating pasta (high carb) vs when they are eating steak (low carb).

Another question for you -

Another question for you - how would you double blind the study? The biggest problem would be that people know when they are eating pasta (high carb) vs when they are eating steak (low carb).

This is why I mentioned the "jury selection" bit. If you want a truly double blind study, your test subjects will need to be people who are not familiar with Atkins, either by name or by method.

Micha, This is why I

Micha,

This is why I mentioned the "jury selection" bit. If you want a truly double blind study, your test subjects will need to be people who are not familiar with Atkins, either by name or by method.

True, that would help. However, just by looking and seeing the food, there would be a degree a "sight" (lack of blindness to the study) for the experimentees. In drug trials, a true placebo is indistinguishable from the pill that serves as the independent variable by look, taste, feel, and smell.

I eat carbs daily - bread, pasta, veggies, etc. If I was unfamiliar with Atkins and volunteered for the study, I would still think something funny was going on when I would not be allowed to eat bread, pasta, veggies, etc. That might introduce some sort of bias on the part of the experimentees.

Then maybe the test subjects

Then maybe the test subjects should be fed intravenously? ;)

Could we perhaps use animal

Could we perhaps use animal experimentation to determine the effects of an Atkins-style diet? Surely there are omnivorous animals with similar digestive systems to ours? It would be much easier to control for environmental factors such as exercise levels. It seems to me that the ethical problems are pretty minor in this case, since you're just feeding them.

OK, so this is how we'd do

OK, so this is how we'd do it with, say, pigs (or whatever):

-Feed each animal a tightly controlled high-carb, low-fat diet for some months, proportionate to body weight, with the aim of making each pig slightly overweight.

-Switch half the animals to the Atkins-style high-fat, low-carb diet, keeping the others on the original diet (or a reduced version, if the aim is to compare diets). Observe.

I mean, we get constant monitoring, stool and blood samples whenever we want them, health data, you name it. Shouldn't this work?

What placebo? As far as I

What placebo? As far as I can tell, Andrew's experiment did not call for a placebo.
I was asking the same thing that you asked with regard to "double-blindness." I assumed that the placebo would be the "normal diet group" (maybe I just should have said control group.)

This may all be true, but things like knowing the mechanisms behind metabolism, and physiology is absolutely not essential in determining whether or not Atkins works.
you asked for specific methods, and I'm saying that my lack of familiarity makes me unable to participate in such a way. Animal testing is interesting as well, but I'm sure there are problems with animal testing fundamentally. I just don't know them. This is my problem.

OK, so this is how we'd do

OK, so this is how we'd do it with, say, pigs (or whatever):

-Feed each animal a tightly controlled high-carb, low-fat diet for some months, proportionate to body weight, with the aim of making each pig slightly overweight.

-Switch half the animals to the Atkins-style high-fat, low-carb diet, keeping the others on the original diet (or a reduced version, if the aim is to compare diets). Observe.

I mean, we get constant monitoring, stool and blood samples whenever we want them, health data, you name it. Shouldn't this work?

I think it would probably. However a couple of issues:

1)I like your idea of making the pigs overweight by feeding them the same diet prior to splitting them into the study group and the control group. This controls for prior diet, which may be important for human subjects.

2)The Atkins pig food would have to look exactly like the non-Atkins pig food. Otherwise, the look of the food would not be controlled for. It may not matter for pigs, but a true control would be identical in every way except the independent variable (which in this case would be the high-fat composition of the diet).

3)When the splitting of the groups occurs, it's imporant that the control group be fed a negative caloric balance diet to match the Atkins group, i.e, less calories than when the pigs were being 'fatted'. Otherwise, you would have two different (not so) independent variables - caloric intake and caloric composition.

4)Even if this experiment resulted in a statistically different outcome between the two groups, being the skeptic that I am, I would not believe that the Atkins diet works in people. Many studies that work in animals do not show similar results in people.

5)Being the collectivist that I am, I don't see why people necessarily could not be part of a study like this, as individuals are sacrificable for the greater good. ;)

Micha, Then maybe the test

Micha,
Then maybe the test subjects should be fed intravenously? ;)

Actually, that is not a bad idea. A couple of issues:

1)It is possible to feed lipids through IVs, but it has lots of harmful side effects and is used only as a last resort in even the sickest patients.

2)Even if it was possible, it would not truly answer the question of whether or not the Atkins *diet* 'works'. Lots of metabolic changes occurs to food from the time it enters the mouth to the time it gets into the bloodstream.

3)A similar solution might be possible. I don't know if anyone remembers a show from the mid-80's called "Otherworld" but it was about a family who gets transported to a parallel universe while visiting the Egyptian pyramids. Anyway, the grocery stores in that universe were stocked with canned food simply labeled 'good food' and 'juice'. Inside the cans was processed mush which the spoiled protagonists found disguisting.

My idea is that perhaps the food, whether it be fat or carbs, could be procesed beyond recognition to 'sameness'.

Another idea is that 'they' can make soy look like just about anything these days - beef, fish, tofu, milk, etc. I'm sure that there is a way to make the food used in the two groups look alike.

Jonathan, 5)Being the

Jonathan,

5)Being the collectivist that I am, I don't see why people necessarily could not be part of a study like this, as individuals are sacrificable for the greater good.

Well, we could try it on prisoners, I guess. Or orphans. Or--hey!--the comatose!

Alternatively, instead of

Alternatively, instead of "eat the rich" we could force-feed the rich.

Jonathan, In order to truly

Jonathan,

In order to truly control this experiment (with use of your sameness foods), you need to identify why this needs to be identical. How does the subject recognizing being different affect the results? Using a double-blind design must be justified. But before you determine the design of the study, you have to set an objective for the study.

You mention that you want to see if the diet works? What does works mean? Weight loss or improved cardiovascular condition or increased quality of life? Once you determine the variables you will be measuring, then you can go about designing the experiment appropriately.

Barney

Barney, In order to truly

Barney,

In order to truly control this experiment (with use of your sameness foods), you need to identify why this needs to be identical. How does the subject recognizing being different affect the results? Using a double-blind design must be justified. But before you determine the design of the study, you have to set an objective for the study.

That is a great point. I'm not sure the study does need to be double-blinded. In a drug study, if the subjects know that they are not receiving a potentially life-saving drug, (i.e., if they know they are receiving the placebo) they might not keep up with the strict regiment the study protocol calls for, or might find a way to obtain and take the drug on the black market. This would yield misleading results.

The only thing I can think of for blinding the subjects in an Atkins study is the remote, highly unlikely chance that the appearance of the food or knowing what you're eating somehow affects weight loss.

You mention that you want to see if the diet works? What does works mean? Weight loss or improved cardiovascular condition or increased quality of life? Once you determine the variables you will be measuring, then you can go about designing the experiment appropriately.

Yes, and that is why I have repeatedly used quotes around the word "works". I meant to leave that part of the question open-ended for purposes of discussion. To me, the real question behind the Atkins diet is whether or not the composition of the calories makes a difference in weight loss when dieting. Thus, 'works' means a greater weight loss under an Atkins Diet compared to the weight loss under conventional diet with intake of equal number of calories.

If we are then measuring

If we are then measuring weight loss as our primary endpoint, then I think only a single blind study would be feasible. The problem is that you would never get patients recruited or willing to stay in a study for any lengthy period of time if they were just eating food from a can. Previous weight loss studies involving drugs have shown efficacy over a short period of time but in the long run (usually within 1 year) patients end up regaining the weight. So in order for any type of diet to really work, you have to change not only the diet of the patient but also the mindset. So, we would have to design a study where you not only change the intake but also provide counseling to all patients. The unfortunate part of this is that then the study does not mirror a real-world situation.

In this experiment, not only do we have to control the caloric intake but we would also have to somehow closely control the amount of exercise each individual gets every day.