Thursday
Apr072005
Camera Phones and Ten Cent Augmented Reality
Thursday, April 7, 2005 at 2:15PM
I just finished reading Why Not, which, at the beginning, suggests asking yourself: What really bugs you? to generate problems that need solving. The first thing that came to me was this: Often I'm in our neighborhood video store browsing videos (of all things), and I'll come across a title that seems good but whose quality I'm nervous about. (I don't watch much TV, so they're all pretty new to me.) I have a few rules that help me decide ("Don't rent anything that Larry King likes" comes to mind), but what I really want are review summaries from trusted sources. Naturally I want them quickly and easily, as there are many videos to look at.
So here's the problem: How can we quickly and easily deliver customized reviews to a patron at the video store?
One solution is to simply look up the movie in one of the review books placed around the store. However: a) I'm lazy and I don't want to do the work, and b) the books don't show the latest movies. So (as all of us technical folks would think) why not use those ubiquitous cell phones everyone has to look it up on the internet? Ignore for a minute how we'd input the title, you can imagine a service that returns something like Yahoo's Critical Consensus portion of reviews (here, for example).
Now the question becomes: how do we specify the movie to the server? Remember that it has to be quick (e.g., round-trip time for reviews should be less than ~5 seconds) and easy. The first is a technical issue that I assume can be solved, but the second is where the fun thinking comes in. Here's what I came up with:
Solution 1) involves speaker-independent continuous speech recognition which is still relatively unsolved. It's possible for limited vocabularies, and movie titles are very constrained - we'll assume your server knows all the titles, so this one may be possible. Plus, talking to a computer and getting a data response has a HAL 9000 feel to it! A possibility.
Solution 2) This one is technically easy (all the recognition is done by the person, but typing in a dozen digits is way too tedious, slow, and error-prone. Not a possibility.
Solution 3) This one is the coolest, because a) it gives all those people with camera phones something useful to do with them (other than emailing them, of course), and b) it gets us into the main point of this post - using camera phones for a cheap form of augmented reality. Whether you take a picture of the box itself or the barcode doesn't matter much - it's another technical issue, I think. (More on barcodes below, though.) It turns out there are efforts afoot to put barcode recognition software into cellphones (read more here), so the rest should be straightforward. A possibility.
So I have a natural question: Why haven't we seen this yet in stores? It's really cool, and at this point a no-brainer! Anyone want to work on it?
However, there's a bigger idea here: Until we have the cool digital specs, tiny fast computers, and very high speed image recognition, this 'take a picture of a barcode' idea might be a useful transition technology. Think of the things that have bar codes on them - books, wines, etc. The same technology could augment these by providing reviews, safety information, comparison products, pricing, etc. Kinda neat. However, I'd like to take it beyond things that have barcodes. For example, I'd like to get dynamic information on things like restaurants ("try the wild mushrooms on toast"), parks ("free boat rides on Thursdays"), and businesses ("there's a better realtor down the street"). All you (or a business) would have to do is print your own barcodes (or Semacodes), and then register them somewhere. There are possible problems of course (e.g., spam), but we should be able to translate solutions from other 'lots of users commenting on things' applications.
Thoughts?
So here's the problem: How can we quickly and easily deliver customized reviews to a patron at the video store?
One solution is to simply look up the movie in one of the review books placed around the store. However: a) I'm lazy and I don't want to do the work, and b) the books don't show the latest movies. So (as all of us technical folks would think) why not use those ubiquitous cell phones everyone has to look it up on the internet? Ignore for a minute how we'd input the title, you can imagine a service that returns something like Yahoo's Critical Consensus portion of reviews (here, for example).
Now the question becomes: how do we specify the movie to the server? Remember that it has to be quick (e.g., round-trip time for reviews should be less than ~5 seconds) and easy. The first is a technical issue that I assume can be solved, but the second is where the fun thinking comes in. Here's what I came up with:
- Speak the title into the phone.
- Type in the barcode numbers with the keypad.
- Use the camera to take a picture of the box or of the barcode.
Solution 1) involves speaker-independent continuous speech recognition which is still relatively unsolved. It's possible for limited vocabularies, and movie titles are very constrained - we'll assume your server knows all the titles, so this one may be possible. Plus, talking to a computer and getting a data response has a HAL 9000 feel to it! A possibility.
Solution 2) This one is technically easy (all the recognition is done by the person, but typing in a dozen digits is way too tedious, slow, and error-prone. Not a possibility.
Solution 3) This one is the coolest, because a) it gives all those people with camera phones something useful to do with them (other than emailing them, of course), and b) it gets us into the main point of this post - using camera phones for a cheap form of augmented reality. Whether you take a picture of the box itself or the barcode doesn't matter much - it's another technical issue, I think. (More on barcodes below, though.) It turns out there are efforts afoot to put barcode recognition software into cellphones (read more here), so the rest should be straightforward. A possibility.
So I have a natural question: Why haven't we seen this yet in stores? It's really cool, and at this point a no-brainer! Anyone want to work on it?
However, there's a bigger idea here: Until we have the cool digital specs, tiny fast computers, and very high speed image recognition, this 'take a picture of a barcode' idea might be a useful transition technology. Think of the things that have bar codes on them - books, wines, etc. The same technology could augment these by providing reviews, safety information, comparison products, pricing, etc. Kinda neat. However, I'd like to take it beyond things that have barcodes. For example, I'd like to get dynamic information on things like restaurants ("try the wild mushrooms on toast"), parks ("free boat rides on Thursdays"), and businesses ("there's a better realtor down the street"). All you (or a business) would have to do is print your own barcodes (or Semacodes), and then register them somewhere. There are possible problems of course (e.g., spam), but we should be able to translate solutions from other 'lots of users commenting on things' applications.
Thoughts?
Reader Comments (3)
Go to a video store where the clerks actually watch movies, and ask a clerk.
Do your video shopping through your local public library's online catalog, which lets you browse the shelves from home and do your imdb lookups (or whatever your favorite movie-site is) from the comfort of your own computer.
Shop at a video store that not only has those big outdated books at it, but also has a live internet terminal hooked up to IMDB for you to do review-searches from within the store.
No camera-phones needed!
(the store I have in mind is Liberty Video in Ann Arbor MI, which also ha a 6-6-6 special - 6 videos, 6 days, $6.66.)
Great suggestions, Edward. Asking the clerks (which I've done some, and should do more - they can be helpful) is a good way for me (a geek) to get more social contact as well. And I like the in-store computer idea. I wonder if I can get them to do it; they're somewhat low-tech there, which is part of the appeal. Thanks!
Here's an interesting related project, sent to me from a reader: [ Hear&There
An Augmented Reality System of Linked Audio | http://smg.media.mit.edu/projects/HearAndThere/ ].