Follow Up on Butter

Welcome readers. I wanted to post some follow up details on my experiments so far and say a bit more about the themes I will be writing more about. You'll notice that my headlines so far have been phrased as questions. This is not an homage to a popular quiz show -- they are questions because I don't know the answers. So with that in mind I wanted to revisit the results of my first experiment (Is Butter as Powerful as a Statin?).

Science is hard, and if you attempt it, you need to stay vigilant about not fooling yourself, because, as Richard Feynman has said, you're the easiest person to fool. So I'll be reviewing my initial results to see whether I've fooled myself. Of course when it comes to personal science, we should always be aware of this principle in reviewing work done by others. If someone has managed to fool himself (even for a moment), perhaps they might fool you too. I hope my readers will keep this in mind.

Commenter EricT highlighted the biggest weakness in my butter experiment.  I originally added the butter to my diet to see what sorts of subjective effects it might have (they were positive, but I did not track anything quantitative that I can present). I tracked my lipids at the time only in order to monitor whether they might move in an adverse direction. It looked like the Kerrygold butter caused beneficial changes while other brands did not. I therefore changed my hypothesis mid-stream, which is a substantial source of bias, effectively rendering my statistical analysis invalid (this particular weakness is not applicable to the second experiment on safe starches vs. HDL, though of course other weaknesses do apply). Meanwhile I am continuing the butter experiment and will present all additional data points in a couple of weeks so we can see whether the effect holds. In the mean time, by then I will have data from another VAP cholesterol test so I can have more confidence about what my CardioChek meter is actually measuring and how consistent it is. Stay tuned for that.  Meanwhile I have declared a "grain of salt" advisory on the butter post.

If the additional data points do not show a continuation of the effect seen previously, it would weigh strongly against the hypothesis that butter causes beneficial changes in lipids (though this is somewhat independent to the question of whether butter is good or bad from a cardiovascular perspective). Many alternative hypotheses could explain a negative outcome -- the effect could have been the result of the bias, a short-lived disruption of homeostasis, a systematic user error or environmental sensitivity with the CardioChek device, or an unknown and temporary cause coinciding in time with the applicable data points. Since I did not decide beforehand how long my experiment would last, the effect could also have been a statistical anomaly which only seemed "significant" because of the particular day on which I decided to compile my results (that is, you can't just wait until you get the result you want and choose that as the date on which to terminate your study).

In the mean time, I've been measuring another health marker that is related to gut health and the microbiome.  I am going to write about about that and then present some interesting and hopefully useful data over the next few weeks.