Seth Roberts and Self-Experimenting: Thoughts on his excerpt in 'The 4-Hour Body'

I've been following Seth Roberts for some time. He's been a self-experimenter for many years, and applies his psychology experience research to testing all kinds of things out on himself, primarily health-related. He came to light recently in Tim Ferriss's post The Value of Self-Experimentation where he shares Seth's excerpt from Tim's understated The 4-Hour Body: An Uncommon Guide to Rapid Fat-Loss, Incredible Sex, and Becoming Superhuman. If you look at Roberts' work, especially his The Shangri-La Diet ("The No Hunger Eat Anything Weight-Loss Plan") you'll see how they might be kindered spirits.

Roberts' excerpt is an excellent introduction to self experimentation. Overall it gives a history of his fascinating experiments, argues that self-experimentation plays an important role that larger studies can't play, and explains the factors (financial and political) that cause ineffective treatments to be promoted, or vice versa. Below I list a few notes and comments about his piece, but I encourage you to read it yourself. If you want more of him you can watch Seth's video Stop worrying and start experimenting! over at the Quantified Self. I've listed more Seth-related links below [1].

Enjoy!

From my acne research I learned that self-experimentation can be used by non-experts to (a) see if the experts are right and (b) learn something they don't know.

I love this attitude, though I think it can take time to cultivate the idea of having power to try things out on your own, in spite of expert advice, such as your doctor's advice.

After several years, I ran out of things to try. All my ideas about what might help had proved wrong.

This shows that experiments often require perseverance. As I tried to explain in Variables, human bodies are complex, and it can take time both for results to manifest, and for us to figure out what's going on.

For unrelated reasons, I changed my breakfast from oatmeal to fruit. A few days later, I started waking up too early every morning instead of half the time.

Roberts' creativity for coming up with surprising things to try is uncanny. Standing on one leg; fruit for breakfast; eating butter and testing math ability. It argues for trying lots of things. A question I have is can we learn or get inspiration for what to try?

To make sure the correlation reflected causality, I went back and forth between fruit and oatmeal. The results showed it was cause and effect. Fruit for breakfast caused more early awakening than oatmeal for breakfast.

In email conversations with Seth I've learned that the usual name in psychology for this type of back and forth process is "ABA designs" or "ABA-type designs" (see the Reversal or ABA designs section of the Wikipedia article, Single-subject research). As I understand it, this is the fundamental method for single-subject experiments. The only other design type I've heard of is a kind of "parallel" one (rather than serial ABA types) that takes advantage of our bilateral symmetry by trying different treatments on left and right sides at the same time. I'm told this is called a "blocking experiment." For example, a friend used this to test different treatments for a bout of poison ivy. I recently used this to test different cold weather clothing for mountain biking (different footwear on left and right sides).

Roberts describes three uses for self-experimentation: To test ideas, generate new ideas, and to develop ideas. I like that he highlights the role of surprise in coming up with new ideas, such as when he accidentally tried to put shoes on standing up and found his balance was improved. (You might like my post One way to enjoy the ride - celebrate surprise!)

the fact that we monitor ourselves in a hundred ways, makes it easy for self-experimentation to reveal unexpected side effects

Implicit in Roberts' work is that he is measuring many variables at once. For me this takes a lot of effort and diligence, which limits how many experiments I feel able to run concurrently. This makes me think about the role of Miller's Law of Requisite Parsimony which "asserts that human beings can only deal simultaneously with between five and nine observations at one time" (from the PDF Engaging in a Dialogue Game by Alexander Christakis).

Accidental discoveries cannot be placebo effects.

This insight comes from Roberts' response to critiques about self experiments not being "blind." I simply love it.

He lists three advantages of self-experimentation over conventional research done by experts: More power (faster, cheaper, and more wisdom via mistakes), stone age-like treatments (simple environmental changes) are easy to test, and better motivation.

Always do the minimum - the simplest, easiest experiment that will make progress.

The ideas of doing something fast and cheap, and designing it to fail quickly are hallmarks of making good prototypes that are essential to efficiently testing out new ideas with a minimum of effort and investment. (More on what constitutes "good failures" at How Google Grows...and Grows...and Grows.)

Fear of loss of job, grant, or status also makes it hard for professional scientists to propose radical new ideas. Self-experimenters, trying to solve their own problem on their own time, are not trapped like this.

This makes me wonder how we can apply self-experimentation to innovation. What would the rules be? Involve non-experts? Try things that have already been "proven" to not work? How these relate to skunkworks projects?

Because they had total freedom and plenty of time ... [non-professionals] were able to use the accumulated knowledge of their time better

This brings the question about whether self-experimentation is accessible only to the privileged, such as us. This comment on my post Is There a Self-Experimentation Gender Gap? describes how the commenter has to remind herself what she takes for granted, including that she has "the privilege of time and space to think about these things." I think the tools and technology may be out of reach to most people, but the experimental mindset - that should be reachable for anyone. This is why I'm focusing on the philosophy first, along with the tools.

[1] Some Seth Roberts resources:

• His blog
• Scientific American articles Self-Experimenters Step Up for Science and Self-Experimenter Freed Himself from Insomnia, Acne and Love Handles
• The Boing Boing article Seth Roberts' fascinating self-experiments
• The MetaFilter entry Self Experimentation is Credible Science??
• A little on the controversy around self-experimenting: Learning from self-experimentation - Statistical Modeling, Causal Inference, and Social Science

Comments

[Justin Wehr]

Nice! I, too, am very interested in Seth's work and ideas and have learned a lot from him.

Just one point: You said "Implicit in Roberts' work is that he is measuring many variables at once." But in fact Seth is outspoken against complexity in research design. You'll notice that all of his experiments test just one thing, like arithmetic speed. And it seems that most of the tests he run start as hunches either because he noticed a change in something (mood, sleep, energy, etc.) or because he read about someone else's hypothesis.

[Matthew Cornell]

Hi Justin. Your point about keeping it simple is exactly why I brought this up. There is a conflict between good science - making one change and measuring one (or a few) thing, vs. accounting for the complexity of our bodies and the world. The latter argues that we need to keep track of a number of things when we are first trying to figure out things might be at play. For example, in tracking sleep I might want to know about diet, exercise, mood, sex, etc. I'm not saying this as clearly as I'd like, but Seth's point about "monitoring ourselves in a hundred ways" brought this up. Does that make any sense?