Applying Experimental Design to Dieting

Disclaimer

First off I think it is important to note that I am not a health professional. It is likely that some of my assumptions are oversimplifications if not simply incorrect. But I think this is an interesting story that may actually have practical implications. Don't take any of this as health advice.

Background

Every summer I aim to lose a couple of pounds, get lighter for bike season, and bring out somr of the muscle definition. A win win. My prefered way to do this is to just get my diet super locked in for a couple of months. I love food so I would rather do a no compromises lock in for a couple of months then get back to regular schedule programming then do a longer slower approach. I usually sit around 210, and during summer I like to be around 195. Nothing huge, but 7% of total weight which is something.

I will get in depth later, but basically I have been under the impression that I am actually pretty bad at calorie tracking, and undertook this project hoping I could find a consistent factor of error in my calorie tracking that I could use to kind of re-calibrate my estimates.

Setting Up The Experiment

I track calories on myfitnesspal, and I get activity tracking from garmin. All my cycling has power meters and heart rate, so colorie estimates should be fairly accurate there. One thing I have noticed is that myfitnesspal will sometimes tell me "If you keep eating like this you will weigh X pounds in a month" and X is always a super small number, so I have suspected for a while that theres a systemic error in my tracking process.

I was thinking about this and realized there is an opportunity to frame this as an experiment. Specifically the economics side of my mind lights up a bit when I have an equation with just one missing vairable, I want to solve. This felt like that to me.

weight loss in pounds = ((calories consumed - calories burned)+error)/3500

3500 is a rule of thumb, people say 3500 is a pound of fat. I am aware that I probably lose more than just fat when I do a quicker cut as I like to do. So this is an assumption. But regardless, when looking back, I know how much weight I lost, I know many calories I said I consumed, I know how many calories I said I burned, so I should be able to solve for my error.

My thinking can be summed up like this: Losing weight is a game, and the body keeps score. I can tell you I ate in a 3500 calorie defecit for 4 weeks, but unless I lost 4 pounds, in my interpretation, I didnt. actually eat in a 3500 calorie defecit for 4 weeks.

Data Cleaning

All the data in this project came from Garmin. There isn't a Garmin API for hobbyists, however you are able to do static data exports so I pulled one of those. There are definetly some other fun projects I could take on using this data set and I recommend pulling yours if you're a likeminded individual.

During the period where I was dieting, I tracked very consistently at 86.7% of days. I imputed this using a rolling average segmented by weekdays and weekends. So I fill missing weekends with a rolling 4 weekend-day average and the weekdays with a rolling 4 weekday average. This is done with the understanding that I am more likely to go over calories on the weekend. If I were to get further into ML on this project, this would be the type of imputation where I would want to include a bool to represent if it was imputed or not. I wouldnt be surprised if on average a missing value is indicative of a day where I ate above my planned calories. So this adds some bias in the downward direction to my calroie tracking. But this project wasnt meant to be too deep, just a bit of an analysis.

My weigh ins are also super inconsistent, I imputed this as a smooth line from weigh in to weigh in. But next year if I do this again I will weigh in more consistently. Especially because there is so much noise in a weight number. Water retention etc.

Personal Conclusions

During this window I overestimated my deficit by about 1000 calories a day. This is huge. By overestimate my defecit I mean the combination of underestimating my consumption and overestimating my expenditure added up to about 1000 caories a day. I lost 11.2 pounds, and according to my calorie tracking I should've lost 41.9 pounds. This is much more than I actually wanted to lose, so I'm not saying aw shucks I wish I lost 40 pounds, I am more just highlighting that I was way off.

High consumption days seem to drive my error. There is a positive relationship between my estimated burn and my error. Frequently I bike commute to work, and Garmin tells my I burned 900 calories each way. This always seemed too good to be true - perhaps it is. This is a directional conclusion. Perhaps in a future project I can try to further diagnose the source of my error. Right now consumption and exertion are lumped together in defecit, but I could do more work to seperate them and analyze closer how they seem to effect error.

It is tough to think of how I correct for this. Do I just bump 1000 calories off my goal, and proceed tracking as I do? Or do I try to get better at tracking? I think theres some inate problem here. For me personally, when I look in myfitnesspal, and see that ive got 100 calories left, and then im inputing everything I ate for dinner I am more likely to round down then if I say I had 1000 calories left. Kind of conforming my estimates to my goal - even if subconsciously. Also, I was kind of hoping my error would be super consistent, but it really wasnt. I'll include an image. While there is a steady mean, its not like its consitent to the point where I feel I can confidently backwards calculate day by day my actual defecit using some adjustments.

Broad Conclusions

Here is what I learned beyond I am really bad at calorie tracking.

In summary I do think there are some broad applications of this concept. Not just how can I adjust, but how can these findings be applied to help people meet their goals. I will lay out some ideas.

Move away from calorie tracking:

I know a lot of people have a tough relationship with food. For these people, tracking calories can carry a stigma and be problematic. I really did try pretty hard to track calories during this window. Especially the first half. I had the scale out and everything. But I was way off. Regardless, I met my goals. So what does this mean? It wasn't really the calorie counting that got me there. How could it have been? I did it so poorly. More likely it was an increased focus on meal prepping to avoid reaching for temptations combined with getting some big bike mileage. Perhaps for some people it isnt even worth trying to track.

Setting the goal post:

I've seen a lot of buzz about these AI image processing calorie trackers. I have been pretty pessimistic on them. Sense of scale, hidden oils, how can the model be that good? Well maybe I should rephrase this from "How can the model be that good?" to "Can the model be less bad then me?". Honestly I could see this being the case, maybe I will have to try one of those out.

Closing the Loop:

I think at the very least, myfitnesspal should be flagging when my results are not following where they should. I don't have premium, maybe they do something like this on premium. But it seems like a missed opportunity to me that myfitnesspal will tell me over and over that i'm going to lose 2 pounds a week, and then when I continue to lose 1 pound a week, it gives no feedback. A even an under the hood adjustment could be nice. This would look like Myfitnesspall identifies my systemic error and adjusts for it in its estimates.

Show and Tell

I am pretty open about my health stats and have no problem sharing them. So I have made some of my analysis public. I like the idea of continuing to build out a bit of a health hub for. myself where I can view some metrics. I definetly want to do some more projects using this health data set, I find it super interesting.

Heres the analysis over the period of time where I was aiming to lose some weight