Tuesday
Aug052008
Why Blogruptcy is a great idea but doesn't work, and why SPAM is easy to fix and information overload isn't
Tuesday, August 5, 2008 at 2:04PM
I won't quote you a zillion statistics on the problem information overload causes. I'm sure it's one of the items on the Galactic Scorecard that the Federation of Sentient Species uses to evaluate progress. You know, rate on a scale of 1-5 the following:
This is especially noticeable for RSS feeds, where there's always more blogs, posts, and ideas. While talking about this with my friend and collaborator Tara Robinson (site, blog, book) she mentioned being ready to commit "blogruptcy," a phrase with surprisingly few hits.
The idea is simple, with steps as inspired by the original Lessig story Declare Email Bankruptcy:
This is a fine practice, and I'm using it right now. It's very often the case that most of the feeds aren't crucial to our work/lives, and that we always have more than we need. This is in the general category of going on a media diet [1].
The only problem? It doesn't work. The tactic is OK with email because (depending on your power/repute/fame) there's a built-in recovery mechanism, a pull that will bring important things back to your attention. RSS, however, has no such pull. No one is watching your reading (the equivalent to responding to email messages) to see what you missed and make sure you're getting valuable information - there's no helpful nagging.
Of course few people can use this "Bill Clinton" approach (you need me, I'm hard to get in touch with, I have an assistant, so just keep trying), so we're stuck doing it ourselves. That's the bad news. The good news is a solution is possible, and - like many things - it depends on collaborative filtering, in this case by trusted people in your personal network who know you and can predict with high accuracy what information you find valuable. I've no doubt a trainable program will emerge, but that approach is hard [2].
An intermediate "manual" solution might be to track it yourself. As I wrote in Information Provenance - The Missing Link Between Attention, RSS Feeds, And Value-based Filtering, what's missing is a way to feed back value from information sources. I don't know of tools that do this yet.
A related approach you'd think would generalize to information overload is something like Gmail's SPAM filtering. It uses Gmail as what Nicholas Carr in The Big Switch calls a General Purpose Technology - a platform on which many different tools or applications can be constructed. Like their infrastructure [3], Gmail uses millions of eyeballs. To the question "Is this SPAM?" Gmail answers "I don't know. Ask someone else." In this case, 50 million someone elses. Every time you click the "SPAM" button, you're helping everyone else just a little. It's very cool.
However, this breaks down when applied to information overload. This is because SPAM messages are pretty much universally recognizable. If you think it's SPAM, I probably do too. But, like the old saw "One man's treasure is another man's trash," value is not implicit to the content when it comes to information - it has to be contextualized through our history, interests, and ultimate purpose. And these things can't yet be codified in a model that can be run separate from the brain [4]. So we need trained eyeballs.
I see two ways to get them: Money or Love.
Money: Use Amazon's ingenious Mechanical Turk to outsource people who will find important things for us. Problem: We either need the same group of people each time (they have to learn, right?) or we have to express specifically what we're interested in. Oh wait, that's the attention models I mentioned below. So scratch that. Cool idea, though.
Love: Ask people who know us to Send Good Stuff. I tried this as an experiment when I asked you to Feed The IdeaMatt! (and I got a few nice responses - thanks!) More generally, this brings us to why social networking sites like LinkedIn and (increasingly, given the number of connection invitations I've received) Plaxo, integrated with next-generation RSS feed readers, might give us a simple way to solve information overload. We'd need a small network of peers that we ask/allow to send us recommendations. They'd need to know us and keep an eye out for relevant information (see How To Help People for the gist), and send it along using some kind of easy-to-tag system.
Could something this obvious and simple work? Hell, maybe it's already here - delicious is the classic "bookmark and tag" system. Maybe we just need a little layer to make sharing explicit. But like picking up the phone instead of sending an email, it has that simple, directly-connected personal feel of helping. Simple tools to help us connect and share meaningful stuff, and filter out the garbage? Sounds like a great use of technology.
Thoughts?
- Plays well with others (didn't annihilate selves with nuclear bombs)
- Picks up after self (didn't dirty the nest with pollution or global warming)
- Is careful with lunch money (didn't squander finite oil supply)
- Learns well (managed to create an effective planetary data network)
- ...
This is especially noticeable for RSS feeds, where there's always more blogs, posts, and ideas. While talking about this with my friend and collaborator Tara Robinson (site, blog, book) she mentioned being ready to commit "blogruptcy," a phrase with surprisingly few hits.
The idea is simple, with steps as inspired by the original Lessig story Declare Email Bankruptcy:
- Unsubscribe from all your feeds.
- Try going feed-free for one [week | month].
- Evaluate: What did you really miss?
- Add those back in.
This is a fine practice, and I'm using it right now. It's very often the case that most of the feeds aren't crucial to our work/lives, and that we always have more than we need. This is in the general category of going on a media diet [1].
The only problem? It doesn't work. The tactic is OK with email because (depending on your power/repute/fame) there's a built-in recovery mechanism, a pull that will bring important things back to your attention. RSS, however, has no such pull. No one is watching your reading (the equivalent to responding to email messages) to see what you missed and make sure you're getting valuable information - there's no helpful nagging.
Of course few people can use this "Bill Clinton" approach (you need me, I'm hard to get in touch with, I have an assistant, so just keep trying), so we're stuck doing it ourselves. That's the bad news. The good news is a solution is possible, and - like many things - it depends on collaborative filtering, in this case by trusted people in your personal network who know you and can predict with high accuracy what information you find valuable. I've no doubt a trainable program will emerge, but that approach is hard [2].
An intermediate "manual" solution might be to track it yourself. As I wrote in Information Provenance - The Missing Link Between Attention, RSS Feeds, And Value-based Filtering, what's missing is a way to feed back value from information sources. I don't know of tools that do this yet.
A related approach you'd think would generalize to information overload is something like Gmail's SPAM filtering. It uses Gmail as what Nicholas Carr in The Big Switch calls a General Purpose Technology - a platform on which many different tools or applications can be constructed. Like their infrastructure [3], Gmail uses millions of eyeballs. To the question "Is this SPAM?" Gmail answers "I don't know. Ask someone else." In this case, 50 million someone elses. Every time you click the "SPAM" button, you're helping everyone else just a little. It's very cool.
However, this breaks down when applied to information overload. This is because SPAM messages are pretty much universally recognizable. If you think it's SPAM, I probably do too. But, like the old saw "One man's treasure is another man's trash," value is not implicit to the content when it comes to information - it has to be contextualized through our history, interests, and ultimate purpose. And these things can't yet be codified in a model that can be run separate from the brain [4]. So we need trained eyeballs.
I see two ways to get them: Money or Love.
Money: Use Amazon's ingenious Mechanical Turk to outsource people who will find important things for us. Problem: We either need the same group of people each time (they have to learn, right?) or we have to express specifically what we're interested in. Oh wait, that's the attention models I mentioned below. So scratch that. Cool idea, though.
Love: Ask people who know us to Send Good Stuff. I tried this as an experiment when I asked you to Feed The IdeaMatt! (and I got a few nice responses - thanks!) More generally, this brings us to why social networking sites like LinkedIn and (increasingly, given the number of connection invitations I've received) Plaxo, integrated with next-generation RSS feed readers, might give us a simple way to solve information overload. We'd need a small network of peers that we ask/allow to send us recommendations. They'd need to know us and keep an eye out for relevant information (see How To Help People for the gist), and send it along using some kind of easy-to-tag system.
Could something this obvious and simple work? Hell, maybe it's already here - delicious is the classic "bookmark and tag" system. Maybe we just need a little layer to make sharing explicit. But like picking up the phone instead of sending an email, it has that simple, directly-connected personal feel of helping. Simple tools to help us connect and share meaningful stuff, and filter out the garbage? Sounds like a great use of technology.
Thoughts?
References
- [1] The best two resources for this are Bit Literacy (interview A Conversation With Mark Hurst, Web Usability Expert And Author Of "Bit Literacy") and The 4-Hour Workweek. Mark's model: Create a media portfolio with two main components: Lineup and Tryouts. Lineup: Those that've earned their place as your most valuable sources. (The three types: Stars, Scans, and Targets.) Candidate sources get into the lineup by going through a tryout phase. Guidelines: Be discerning, be intentional, and be biased toward rejecting.
- [2] The research group I worked in before changing careers is a good example of these machine learning efforts.
- [3] See Google platform or Jeff Dean on Google infrastructure. I esp. like the summary at Narayanan Shivakumar on Google's Software Infrastructure, including bullets on Google File System (GFS), MapReduce, and BigTable (gotta' love those names :-)
- [4] While at KDL I speculated about "attention models," digital artifacts that I could train then set out into the world (AKA "agents"). A fun variation would be to send mine to yours, where you would give it permission to see some subset of your bitstream. Of course I'd want yours too :-)
Reader Comments (18)
I love Google Reader for a number of reasons, but one feature from Bloglines that I really miss is the "Unsubscribe from this feed" link that I recall being right there when you're reading. It's been years, so I don't remember the details, and I hope it's still there.
This also has something to do with the default "River of News" view in Google reader, which encourages simply glossing over posts you're not interested in, and dissociates the posts a little from their parent feeds. Bloglines, on the other hand (this could have changed), defaults to making you read all of the posts from a given feed in a group, making it easier to evaluate the value you're really getting from said feed.
I also used Gregarius for a while, with mixed results. Which reader(s) do you use?
One thing that is somewhere between love and money is your own blogosphere (the collection of blogs you follow, and potentially those that follow you). If you slurp in feeds from people who have similar interests, you are bound to find people referencing articles and topics over time.
What you need is tool that brings all your information sources together and gives you some kind of "interesting-to-you" measure, based on all those sources. AideRSS is one of those attempts, but I was never patient enough to get it flowing for me. I really liked SharpReader (available, but no longer developed) for its ability to show threads of articles that reference one another.
Of course, this is somewhat complicated by the fact that people do this in their blogs, Twitter (and other microblogs), Facebook, .... You'd need to point to all the sources you want to process.
NewsGator has some spiffy mechanisms in their "heavy" client app that allows you to identify and view the blogs you tend to read most often (your own "most popular" list). The online newsreader has a spiffy "Unsubscribe" button (along with "Mark all read," "Mark all page read," etc.). (Nondisclaimer: I don't work with, for, near, or get paid by any NewsGator organization, money, or people.)
Your point about the lack of pull for RSS info sources is right on! I was, in fact, mentally making that very same protest halfway through the first few sentences of your post. Good follow-through :).
However, I have to say that we need a or some tool(s) that allow us to distinguish between "people who are friends but may be a bit wonky as info sources go" (like one of my real-life very good friends, whom I value greatly, who keeps sending me invitations to add bizarre applications on Facebook) and "people who may or may not be friends but are very valuable information resources" (e.g., the people I add to my network on del.icio.us because they have a high signal-to-noise ratio in bookmarking).
Overall, a good thought-provoking post. Thanks for writing it.
Hi David,
> NewsGator has some spiffy mechanisms in their "heavy" client...
Good pointers, David. Here's the Mac desktop app: [ NetNewsWire | http://www.newsgator.com/Individuals/NetNewsWire/Default.aspx ] and their [ NewsGator Online | http://www.newsgator.com/Individuals/NewsGatorOnline/Default.aspx ]. I'd be curious how well the "Recommended Content" feature works...
> "people who are friends but may be a bit wonky as info sources go"...
I love it. I was thinking binary - Good Source (or not). Identifying source quality is one of those hard problems that the intelligence agencies has to deal with, interesting. So maybe our information overload problems will be solved by software developed for the CIA. Interesting irony...
> Overall, a good thought-provoking post. Thanks for writing it.
Much appreciated! Thanks for reading, and for your comment.
Hi Matt
Hope you're well. I'm hoping I have the answer to your problem - coincidentally I was coming at some of your challenges from a different perspective in a couple of posts last week that I'd titled the Social and Information Web Drug http://justseventhings.com/?s=Web+Drug
Specifically, a colleague managed to get one of the last beta downloads for partcils, an 'attention management engine'
There's an article here which explains: http://www.readwriteweb.com/archives/particls_launches_advanced_alerts_platform.php
It looked brilliant - using implicit (if tracks what you look at/ click through) and explicit (ability to rank and weight content) attention weightings.......
..... but, a bit like Christmas being conditional on being good ;-), I'm not sure on when the formal launch is....
Hopefully one tick in the box for sentient beings?
Si
Hey Si,
Thanks for the pointer. On your post you wrote:
I really like the phrase "attention management engine," and what they call their "Policy of Diminishing Attention Consumption." Unfortunately I couldn't find anything describing how they'd manage this. Again, my thought is that generalized filtering ala Gmail won't give good results. It needs to be collaborative via our social networks, as you imply.Could you find any details on Particls' algorithm?
Thanks again for your comment!
Hi Matt,
Thanks for the great post and the props. :) A note about my own blogruptcy process. My steps are:
1. Go to Google Reader, mark everything as read.
2. Scan my feed subscriptions to see if there are any I am no longer interested in; dump those.
3. Vow to check in and read new posts on the remaining feeds more regularly.
When I've done this, I reconcile myself to the fact I'm missing content. If the not-knowing creates itchiness, then declaring blogruptcy doesn't work. However, my blogosphere (as one of your readers suggested) does supply hints when a feed might need to return to my subscriptions list.
Thanks again for the honorable mention. :)
Best wishes,
Tara
You're welcome, Tara. And thanks for the processes summary.
...or I'm just lazy. One or the other.
I don't have this problem because, well, I guess I'm just not that interested in tracking everything that people around the world write. Just as when I walk into a bookstore or library and accept the fact that I can't/won't read everything, I accept the fact that I can't/won't read every interesting post in the blogosphere.
Am I missing some goodies? Undoubtedly. But what I gain is more time to do the stuff that's really important (play with my cat, spend time with my wife, cook dinner, and, um, actually *work*).
I also know that the few blogs I do follow -- yours, for example -- will alert me to anything really great that pops up. Which might create a tragedy of the commons situation, but I'm confident that there are enough people who do nothing with their lives *but* read blogs that this won't be a significant problem. So keep reading for me ; )
Good comment, Dan. Your discipline is admirable, and a good example of restraint combined with purposeful choices. You've made me wonder what the facts of the situation are. How's this:
Therefore:
Thanks also for the reminder about [ Tragedy of the commons | http://en.wikipedia.org/wiki/Tragedy_of_the_commons ]. It's valuable to me to have these big ideas come at me - another reason I love blogging. Thanks again.
Great topic.
I believe the metaphor we are all trapped in is one of being out there on the shiny edge of an ever expanding bubble of posts. As content expands we fight to be on the periphery, to be on the immediacy of the expansion, and as that bubble grows the amount of territory we have to cover increases as well, just as the surface area of a sphere grows as it expands.
If that is the metaphor, there is no other solution than to check out as we all are suggesting, unless you begin to think differently about the issue of immediacy. I would argue that the most recently written material on a subject is not necessarily the best and that if we think of all the content we are tracking not as on the film of a bubble but as in a pond (lake, sea...) filling over time, the issue becomes where do we fish, not how do we stand before the floodgates.
I also think that as intellectual citizens of the blogsphere we serve a greater purpose by knitting together the best material in our metaphorical pond, rather than just re-reporting on the the latest half completed idea. Once you get off the edge of that bubble everything changes.
Alright, enough of this. I have to get back and see what I missed in my Reader during the last five minutes. I'm sure it was crucial to something, somehow...
Doug
www.dougist.com
(btw: Very well thought out post, and nice site)
Hey Doug, thanks for your comment. Plus, I like how you talk purty.
> on the shiny edge of an ever expanding bubble of posts
I really like the shift you suggest - from bubble to spreading sea. If we're not on the margins (and esp. not constantly ruminating on them) then we're free to fish. Fishing is good a good one. I've never done it, but conceptually it:
> the most recently written material on a subject is not necessarily the best...by knitting together the best material in our metaphorical pond, rather than just re-reporting on the the latest half completed idea
Woo! Thanks for saying this. I'm exempt, of course, but I'm glad it's out there now. You'll be seeing this come back soon in an up-coming controversial (I hope) post.
> (btw: Very well thought out post, and nice site)
Much obliged. P.S., I got some good chuckles from your [ Background | http://dougist.com/index.php?page_id=6 ] page.
Matt, you have attributed Tim Ferris's book to Mark Hurst. Considering this article (yes, I agree it's journalistically dubious)
http://whakate.com.dd19920.kasserver.com/lead-articles/what-is-wrong-with-gtd/
at Whatkate, both Tim and Mark might object to sharing an authorship room.
I read too fast and missed the parentheses. Funny how often I'm wrong when I get righteous about proofreading. Keep up the good work.
Hi Rex. Thanks for looking out for these kinds of errors - I appreciate it. In this case, I think the sentence is fine. To see that, take out the parenthetical portion, which makes it read,
I.e., I'm listing two books, with detail on the first - including the author - in parentheses. Does this make sense? I might be parsing it incorrectly...Now I just saw *your* reply. Thanks again, Rex.
Matt you are on fire :)
Just a note, do you know about the 'links for you' function in del.icio.us? When you tag something that you want me to see, just tag it for:urlwolf, and it'll show up in my 'links for you' page. See:
del.icio.us/help/for
It implements your 'love' solution well, and it saves you one email with a link to your friends.
Very kind compliment. Much appreciated! I did not know about the "for:" feature. Looks cool. The help on it has moved. From the [ FAQ on Delicious | http://delicious.com/help/faq#network ]:
What is my Inbox (formerly known as "links for you"), and how do I use it?
This is where you can receive bookmarks from other Delicious users. Your Inbox is not visible to other users. The Inbox link, which is at the top of all pages on Delicious, shows the number of new bookmarks that you have received.
To send a bookmark to another user who's part of your Network, you can click the "for:auser" tag provided for you when saving a bookmark, or you can type "for:auser" directly for anyone (in your Network or not). Sending a link within Delicious can be easier and quicker than email or instant messaging, and your friend or coworker will have a permanent copy of the link that's easy to add to their own bookmarks.
When you tag a bookmark with "for:auser" , only you will be able to see that tag, even though the whole bookmark may be public.
P.S. [ Happy Birthday AP.com! | http://www.academicproductivity.com/2008/happy-birthday-apcom/ ]