wearing augmented reality glasses (note: characters are fictional)
Do you think your natural senses are inadequate? Have you ever wished for more information about your surroundings? Augmented reality (AR) glasses could solve your problem. AR refers to the idea of overlapping a computer display on top of your field of view. Last week Google demonstrated its work on this technology, Project Glass, although the focus was on the glasses' camera and not the display. Project Glass is still a few years away from being a consumer product. Right now it seems only good for streaming videos and pictures, and we are told that it has a small computer display. The concept video below shows Google’s idea of how we might one day use such technology, but such versatility could be many years in the making.
Many people use their smartphones to compare prices for products seen in “brick and mortar” stores. This requires a shopper to take the smartphone out of its pocket and either snap a photo or manually search for a product. With AR glasses, one could simply stare at a product for a few seconds, and online prices would be automatically displayed in the shopper's vision. The glasses should make information more natural and personal to access—to the point of forgetting that you are using technology to look up prices. In the eyes of a user, the physical and digital worlds are seamlessly blended to create an information-rich environment.
One major obstacle for this technology is control. How do you communicate with a pair of glasses? The three most likely solutions are voice command, gesture command, and using a smartphone as a remote. It seems natural to build AR glasses like the next generation of Bluetooth headsets—an extension of a smartphone rather than a standalone device. This way many verbal commands could be processed by software like Siri, and a lot of time handling the smartphone would be eliminated. Another method of control is gesture recognition. Much progress has already been made in this field through Microsoft's Kinect and MIT's SixthSense. To take pictures, a user could frame a shot with his/her hands. The AR glasses would recognize the gesture and act accordingly. Pointing at something might initiate search queries. Handshakes could activate some social media function. Sign language could be translated in real time.
Google’s main selling point seems to be that parents will want an unobtrusive way of filming their children. When you photograph your month-old baby with AR glasses, he/she is looking into your eyes, not the camera. Wearing the glasses all day, parents can start recording at a moment's notice. The baby's first words, first steps, and countless other moments would be preserved that might have gone missed. But this accessibility extends beyond parents. Think of YouTube videos. How many more embarrassing moments, personal injuries, and hilarious cat videos will be uploaded when people wear cameras on their heads most of the time. Tutorial videos might also get a boost in quantity with instructors providing a first-person view of what they are doing. AR glasses could also have telepresence applications. Anyone wearing a pair could have a live video stream (as was demonstrated by Google with the skydivers and bicyclists). Concerts, conferences, street performances, movies, and many other events could have countless online viewers seeing through the eyes of one who is physically present.
Google’s Project Glass is loaded with various sensors. It can sense the wearer's global position, direction of view, and angle of view. With so much data ready to be streamed online, Google could create a highly detailed and constantly updated digital model of the world. Nature trails, tour routes, store departments, museums, caves. . . all could be easily mapped and uploaded to Google Earth. Photos could be automatically sent to Street View. Anyone wearing Google glasses could serve as a virtual cartographer for Google Earth. With so much geographical data, people could have GPS guidance for finding their favorite book section, the nearest food stand, or even the nearest restroom.
It would be amazing to record your thoughts with a computer, would it not? The continuing efforts of researchers at UC Berkeley are making that possibility more feasible every year. Their latest experiment involves reconstructing videos based on fMRI readings. First a subject is shown various movie trailers while their brain is scanned. A computer correlates patterns in the fMRI data with visual patterns in the trailers. For part two of the experiment, the subject watches movie trailers while being scanned again. This time, however, the computer analyzes millions of seconds of YouTube videos to find clips that closely match the brain scan data. The matching clips are overlapped into a single video that very roughly resembles what the subject was viewing.
So far the clarity of the reconstructed videos is extremely limited. But the scientists' work is still quite astonishing. They can create videos of what people see just by scanning their brains! In the future this should result in the ability to create videos just by imagining them and clicking record on a computer. That would create quite a stir on YouTube. Recording dreams would fulfill a longtime wish for many. Perhaps within the next few decades there will be a slim headset that one puts on before going to bed. In the morning, a nearby computer has wirelessly received the brain activity information and generated an easily watched video file. After watching your own dream with the benefit of wakefulness, you might upload it to YouTube, email it to your psychiatrist, or just file it away for later viewing.
There are still a few obstacles to get through before we can do all that. First of all, MRI machines take up most of a room, and being scanned by one is like laying in a coffin. Researchers at the University of Toronto are working on using ultrasound headsets to scan brain activity, but sonic waves are not yet as capable as magnetic ones. Another limitation is that fMRI and ultrasound collect data on blood flow in the brain, not the electric and chemical signals between neurons. This is an indirect way of scanning thoughts, and it will always mean a delay between when a thought occurs and when the equipment actually detects it. An ideal setup would be a headset that can scan for the signals between neurons instead of measuring blood flow.
UPDATE (09-29-11): Swiss and Japanese researchers are working on cars that can read a driver's thoughts, such as knowing that a right turn is coming up.
Finally, a major step towards flexible display screens has been made. UCLA has developed a stretchy sheet of plastic that glows blue when given an electric current. It is a combination of carbon nanotubes and polymer layers. The future possibilities for this kind of technology are endless. The first thing that comes to my mind is a super-portable, resilient computer screen. Imagine a newspaper that plays videos, and you get the idea. As smartphones become more and more capable, it might be convenient to keep an external monitor in your pocket. As you sip your morning coffee, you could pull out a folded-up sheet of plastic. Unfold it, plug it into your iPhone, and have the iPhone download the latest news stories to be displayed like an old-fashioned newspaper. Better yet, go to YouTube, and you have videos playing on a high-definition display. If touch-sensitivity was included, then you could fold the sheet in half for a makeshift laptop. The flexible screen would serve as keyboard and monitor, and the iPhone would do all the computing.
There are other exciting applications, too. Think about safety—construction workers and other people exposed to traffic dangers. Instead of vests that reflect light, they could wear ones that glow in the dark like neon signs. Traffic cones that glow, too. Clothes with computer displays would come in handy. Again, smartphones might be crucial. A jacket covered with the flexible screen could enable a user to access a smartphone without taking it out of its pocket. Connected to a smartphone, a [smart-jacket?] could show the time of day on its sleeve, like a watch. If the user wants to dial a number, then the sleeve shows a keypad. For self-expression or advertising purposes, the front and back of the jacket become personal billboards. The jacket might even be used for camouflage. Imagine wallpaper and curtains that, once wirelessly connected to your computer, can illuminate a room from all angles. What strange lighting effects that might produce! Selecting a picture or video instantly turns an entire wall into a panoramic vista. Interior design would be radically changed by walls that can change color and pattern at the press of a button. Do you have any ideas?
