"One of my goals is to catalyze an army of good self-experimenters"

One of my goals is to catalyze an army of good self-experimenters; part of my job is therefore to train readers to do their own homework. Richard Feynman famously remarked, "It doesn’t matter how beautiful your theory is, it doesn’t matter how smart you are. If it doesn’t agree with experiment, it’s wrong."

Tim Ferriss in his Wired interview Tim Ferriss Wants to Hack Your Body.

Of course I love this quote, which expresses something Ferriss and I have this in common. His approach is through doing extreme experiments on himself, then sharing them compellingly and controversially with an large, receptive audience that's he's fostered. He's done a ton of good in promiting an experiment-driven life.

My approach is through developing and sharing the wider philosophy (I call it Think, Try, Learn) and creating tools that make it easy for anyone to create, track, and learn from their experiments, and help others with theirs. With the recent Edison features of quantified data and group experiments, the tool now has enough in place for me to figure out how well it does this.

Is The Great American Experiment really one, and how do we measure it?

In Radical Optimism, Kevin Kelly asks the question,

What set of 10 measurements taken over 10 years would convince you that there was progress, for the average human, either in one location or globally? Let's pick a 10/10 metric and measure it.

The timing of my coming across this article is perfect. In my They did WHAT? series I learn about how the word "experiment" is being used in the news, and I've found that the phrase "The Great American Experiment" comes up a lot (about 5 million hits on Google).

This made me curious, because if it is an experiment, what exactly are we trying, what measurements should we be making, what are the current results, and what does this tell us about our national priorities? I wonder if what impact, if any, would creating a national "American Experiment" scoreboard have on our thinking?

With our current polarization and political impotence I can't imagine even coming up with "a modest base of shared values" that people agree on (see below), but wouldn't an honest and frank national discussion that frames it as an experiment be exciting?

I'd like to get a conversation going around this. I was going to list my own childish ones, but the post morphed into references and starting points. I'd love to read your thoughts on the topic. Sections:

• How "Great American Experiment" is used
• Existing measures
• Responses to Kelly's post
• Ideas from the Constitution

How "Great American Experiment" is used

Here are some posts to stir the pot.

• Empire at the End of Decadence by Charles Blow ("Financing for education and social services isn't simply about handouts to the hardscrabble, it is about building an infrastructure that can produce healthy, engaged and well-educated citizens who can compete in an increasingly cutthroat global economy.") His graphic, American Shame, is sobering. It shows us in comparison to many other countries with respect to:
  • Income Inequality
  • Most recent unemployment rate
  • Level of democracy
  • Gallup Poll "Level of Well-Being Index"
  • Life Expectancy at Birth
  • Prison Population
  • Student Performance: Math
  • Student Performance: Science
• Can data predict the next revolution?, in response to The Kindling of Change, also by Charles Blow ("data won't provide the complete answer ... What demographic, political, and economic data is really, truly most indicative of the potential for revolution?")
• Renewing the American Experiment | Living Economies Forum ("many of the things that economists count as contributions to economic growth actually devalue the quality of our lives and ultimately result in high hidden costs, which ironically are then counted as though they were positive contributions")
• PBS - John Gardner - Education and Excellence ("The long-term task is to move toward some modest base of shared values.")
• The Great American Experiment ("What their ancestors really were was scientists. Experimenters. Radicals who always considered the impossible possible.") He provides clues to metrics:
  • Could we create a strong and stable centralized government?
  • Could we grow without destabilizing?
  • Could we solve the problem of slavery?
  • Could we truly create a melting pot in which to forge Americans out of peoples of all nations?
  • Could we give women the vote?
  • Could we accept Jewish people as true Americans?
  • Could we desegregate?
  • Could we assure civil rights regardless of sexuality?
• Who coined the term "The Great American Experiment"? ("The term derives from Alexis de Tocqueville's book Democracy in America, written in the 1830s.")
• Up With America - Saving Our Union ("A democracy is always temporary in nature; it simply cannot exist as a permanent form of government. A democracy will continue to exist up until the time that voters discover that they can vote themselves generous gifts from the public treasury. From that moment on, the majority always votes for the candidates who promise the most benefits from the public treasury, with the result that every democracy will finally collapse due to loose fiscal policy, which is always followed by a dictatorship.") His progression is fascinating:
  1. Bondage
  2. Bondage to Spiritual Faith
  3. Spiritual Faith to Great Courage
  4. Great Courage to Liberty
  5. Liberty to Abundance
  6. Abundance to Complacency
  7. Apathy to Dependence
  8. Dependence Back to Bondage

Existing measures

I often wonder, "How much is enough?" We seem founded on consumption and change, but I think our rich lifestyle has moved our thinking away from the essentials. The NYTimes.com post Alternatives to the G.D.P. lists some arguments against using GDP, and alternatives (listed next). Related is debatewise's question, Could a range of statistics be used to measure a country's wellbeing rather than GDP?, Helium's Debate: Should society measure progress not just by increases in GDP, but rather through a set of more precise quality of life indicators?, and the Quality of life Wikipedia entry. Finally, I like that someone thought to encourage our being proactive (the purpose of the Experiment-Driven Life): How to Measure the Quality of Life.

• Toward National Well-Being Accounts, By Daniel Kahneman, et. al. ("Here we propose measuring national well-being by weighting the time allocated to various activities by the subjective experiences associated with those activities.")
• The Happy Planet Index ("The Index doesn't reveal the "happiest" country in the world. It shows the relative efficiency with which nations convert the planet's natural resources into long and happy lives for their citizens.") You can calculate your score here. The US scores "Poor" on their map.
• Index of Sustainable Economic Welfare ("Rather than simply adding together all expenditures like the gross domestic product, consumer expenditure is balanced by such factors as income distribution and cost associated with pollution and other unsustainable costs.")

  ISEW = personal consumption
  + public non-defensive expenditures
  - private defensive expenditures
  + capital formation
  + services from domestic labour
  - costs of environmental degradation
  - depreciation of natural capital

• Genuine progress indicator ("GPI is an attempt to measure whether a country's growth, increased production of goods, and expanding services have actually resulted in the improvement of the welfare (or well-being) of the people in the country.")
• Gross national happiness, detailed in Time's What About Gross National Happiness? ("Wangchuck still maintains that economic growth does not necessarily lead to contentment, and instead focuses on the four pillars of GNH: economic self-reliance, a pristine environment, the preservation and promotion of Bhutan's culture, and good governance in the form of a democracy.")
• Human Development Index ("composed from data on life expectancy, education and per-capita GNI (as an indicator of standard of living or income) collected at the national level"). From What does the Human Development Index measure?: "The new Report surprisingly shows that there is just little correlation between income growth and changes in health and education over time."
• The 2010 Legatum Prosperity Index("Rather than replicating other measurements that rank countries by their actual levels of wealth, life satisfaction, or development, the Prosperity Index produces rankings based on the foundations of prosperity.") Their findings fell into three categories:
  • i. Prosperity is found in entrepreneurial democracies that have strong social fabrics.
  • ii. Prosperity is a blend of wealth and happiness, but not as one might think.
  • iii. Global prosperity is changing in unexpected ways.

Responses to Kelly's post

Here are the two list-of-10 answers to Kelly's post:

JonHumber:

1. Connectivity (ease of finding opportunity, community, information - may even encompass mobility - ease of travel)
2. Critical Thinking (not just 'education' but education that actually results in the ability to innovate, solve problems, and think critically.)
3. Access to clean drinking water (you might also measure this in other ways, like prevalence of water-born illnesses)
4. Self-reliance (as opposed to increased reliance on the government and social programs, entitlement, victim mentality, and the expectation that everything should be fair or equal because that doesn't seem like an economically sustainable philosophy)
5. Decrease in chronic and non-age related disease (arthritis, asthma, colitis, eczema, etc. I realize this trend is impacted by the broadening definition of these conditions and increased diagnosis, but I'd argue that it isn't progress if we are not solving them faster than they are growing)
6. Freedom (freedom from oppression, tyranny, dictatorship - honoring basic human rights)
7. Violent Crimes/Killing of any kind (including warfare)
8. Economic growth and stability (not that everyone is getting rich, per se, but that the world economy becomes more stable)
9. Reduced debt (both personally and in governments)
10. Cost of living (heat, shelter, food)

iaing:

1. Happiness Index (the British government's new measurement system version is fine).
2. Hours involuntarily spent away from loved ones versus hours voluntarily spent with loved ones (relative hours per week per person).
3. Longevity (average life span of world population).
4. Number of disaster events (number per year of events above the mean of normalised killed distribution and/or damage cost (eg Ilan Noy))
5. Atmospheric CO2-e (ppm). It should go without saying that "progress" = "decrease in ppm". Let's respect science.
6. Ocean stocking rate of principle fish species (first preference would be species extinction rate generally - but these stats are too easily manipulated - also fish stock rate is *the* classic TOC governance problem that is yet to resolve adequately - so is fine)
7. Either; transport fuel production (mbd), or, travel miles (miles per person per year)
8. Asset equality per person (USD is fine).
9. Average level of "body burden" compounds in the global population.
10. Solution found to the fermi paradox.

Ideas from the Constitution

Finally, going back to the basics I can imagine lots of simple measures based on the constitution, though there's plenty of gray areas open to interpretation. I know nothing about it (constitutional law is not my specialty), but what the hell, I'll make a childish stab at it.

It starts,

We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defense, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity, do ordain and establish this Constitution for the United States of America.

So we'd need to measure:

• justice
• domestic tranquility
• defense
• general welfare
• liberty for ourselves
• liberty for future generations

But these are too vague. For example, "establish justice" is implemented by the court system, but the measure "Has court system" (yes/no) isn't useful enough.

Similarly, we could analyze the Bill of Rights and ask what measurements can we infer from the Amendments. For example, Amendment I says:

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.

There's a summary here. Again, lots of interpretation, such as how many guns, and of which types, are legal, and if that's even appropriate today.

New Edison Feature: Ongoing Group Experiments

I'm pleased to announce a new Edison feature: Ongoing Group Experiments. This is simply a new state where people can sign up at any time. This is perfect for an experiment with no set endpoint that you want to invite people to at any time. For example, my Official Group Experiment Example (a sandbox where you can try joining a group experiment) is ongoing.

The vision here is that Edison will have many hundreds of experiments that people can jump in and try right away, without having to go through the steps of a more formally-run experiment. I'm working now on such a thing: The 21 procrastination techniques that Brian Tracy shares in Eat That Frog!: 21 Great Ways to Stop Procrastinating and Get More Done in Less Time (see my post Some thoughts on "Eat that frog!" by Brian Tracy for my take on it).

Would anyone like to create an ongoing experiment. How about one for people to try out a Slow Carb diet (see How to Lose 20 lbs. of Fat in 30 Days… Without Doing Any Exercise)? Or how about taking a small step to reduce your carbon footprint?

Happy experimenting!

How I wasted two years on Twitter, all because I wasn't tracking

[Cross-posted to Quantified Self]

Between 2007 and 2009 I spent a ton of time in Twitter before it finally hit me that 1) the net improvement to my life was zilch, and 2) had I thought of it going in as an experiment, I would have quit a long time ago and freed up energy for more effective efforts. Of course social media tools can provide plenty of value, but, as Alex said, Social media is an addictive time suck.

How do we go about measuring the value of Twitter? Business calls it ROI, but I think of it as simply what you hope to get out of it. The key is deciding why you're using it. In my case I was dabbling, which is a fine motivation, as long as it's done experimentally. After all, how many discoveries came from just getting curious and trying out something new? But here I should have set a time limit, and I'd still want to have something quantified, even it it's as soft as "perceived value."

But for more specific uses, coming up with measures is important. Are you trying to get more customers? Do you want to hear from people who can give you ideas for your product or book? Or maybe it's more of a social pulse use - keeping in touch. Some metrics are straightforward, such as # inquiries about your business, or number of tweets from others that made you smile. However, I think a major challenge is latency - the time delay between action on your part and resulting effects seen in your life. For example, it might be months before you hear from someone who's been silently reading your tweets. Maybe in those cases we could make the measure more direct by asking them explicitly what the impact is. I'm not sure.

While I didn't treat using Twitter as an experiment per se, I managed a few times to use Twitter itself as a platform for experimentation. For example, I had a business trip to North Carolina coming up, so I tested using the tool to make business contacts for the trip. I subscribed to a bunch of local folks, followed their tweets, and replied when appropriate. The result (measured in # business meetings set up) was zero, but the "at least now I know" feeling was satisfying. One thing that's vexing is irreversibility. Any experiments we might do with our followers has permanent effects because individuals have memory ("Oh yea, isn't he the guy who tried giving away the iPads?"), and the account has memory (I might loose or gain followers s a result). One workaround is to make separate accounts for testing, but unfortunately you can't clone followers. Then again, making us start from scratch is probably a good way to keep the experiment clean.

Others have used Twitter this way, such as a much-publicized psychic experiment. (I found the comments on Richard Wiseman's Blog announcing the experiment to be fascinating from the experimental design perspective.) A little searching turned up other posts like Twitter for Research, The start-up chronicles: Experiments with Twitter, Using Twitter for Market Research, and a Twitter Tip Sheet for Experiments. I don't think I'd count The Anybody/Everybody Twitter Experiment as one, though; what was measured?

What do you think? Have you used Twitter in one of these ways, i.e., as itself an experiment, or as a platform for running them? And if you regularly tweet, I'm curious to hear what you get out of it, and what you've measured to determine that.

"I can live with doubt, and uncertainty, and not knowing..."

I can live with doubt, and uncertainty, and not knowing. I think it's much more interesting to live not knowing than to have answers which might be wrong. I have approximate answers, and possible beliefs, and different degrees of certainty about different things, but I’m not absolutely sure of anything, and in many things I don’t know anything about, such as whether it means anything to ask why we’re here, and what the question might mean. I might think about a little, but if I can’t figure it out, then I go to something else. But I don’t have to know an answer. I don’t feel frightened by not knowing things, by being lost in a mysterious universe without having any purpose, which is the way it really is, as far as I can tell, possibly. It doesn’t frighten me.

Richard Feynman from The Pleasure of Finding Things Out: The Best Short Works of Richard P. Feynman