As soon as Sony threw its weight
behind ubiquitous computing, connected digital devices such as cameras,
video cameras, MP3 Audio, PCs etc started to roll off its production
line. First they used USB and removable media such as the Memory Stick,
then Firewire, now they are using 802.11 and Bluetooth (or indeed all of
the above). Take a trip to any electronics store and you'll see
that the company understands gizmos.
So trying sang a sneak
preview of the future I took a trip to Sony Computer Science Laboratory.
Unlike the many corporate research laboratories, which are placed high
on a hill somewhere away from the grubby hands of product development
people, CSL is based near the old Sony Walkman factory just a
stone's throw away for the 30,000-worker campus in Shinagawa in
central Tokyo. You can see why, Sony like many Japanese companies leaves
the big thinking to the US and European research institutions and
focuses on the more practical applications and getting products out the
door. There are no ivory towers here.
Simple
Simon
Certainly, designing products that excite the
public is easier said than done. Now that Bluetooth and 802.11 and other
wireless technologies are being implemented across the company's
product line the next phase is to design new user interface technology
that makes these products easier to use. In short the mission here is to
bring computer devices to the next level where they interact both with
each other and the real world in a more integrated
fashion.
"You really can not ask users to type in DNS and
Internet Protocol or even email address when all they are trying to do
is move a photo from one computer to another," says Dr. Jun
Rekimoto, Director of Sony's Interaction Laboratory. We are
standing the shared area in the Lab. There are dozens of small Sony sub
notebooks, large screen displays, projectors, web cameras and various
projects dotted around the lobby. Around the sides are offices with
white Perspex walls. So virtually every bit of all space can and is used
as a either a whiteboard or projection screen. "In order to design
the future computer environment, you've got to live in it," he
explains.
"What we are interested in is designing new
human computer interaction interfaces for highly portable computers,
that are aware of their surroundings and that can assist the user
automatically rather than having to wait from a specific command,"
he says. "Before the end of the decade, we expect that such
computing devices will be as commonplace as today's Walkmans,
electronic hearing aids, eyeglasses, and wristwatches."
Dr.
Rekimoto then points a chopstick or pen like device at one hand held
computer, grabs an object (in this case its an MPEG Movie) and points to
screen on the wall and the movie appears. What he has actually done is
transfer the movie from one computer to another. The technology is
called pick and drop because it goes a stage further than drag and drop
technology.
"You see, no need to mess around with DNS
addresses and the rest. What we are trying to do is remove the physical
boundaries between various types of computing devices. In the future our
current graphical user interface will just not work because it is too
limited in is application," he says. "I am Asian so I like
using the chopsticks metaphor, but you could use anything including your
finger."
Treating Information like a Physical
Object
He then drops the movie on the screen and
proceeds to write some text with the pen (or chopstick). Then its time
for another object from another computer - this time it's a
photograph, which is dropped onto the screen. Now a graphic and so on.
In fact he can even drop and animation onto the screen and grab another
object such as a Java routine than influences the behavior of that
object. By the time he is finished the screen is filled with animations,
movies and text.
Collaborative project work, teaching or just
sharing a movie, MP3 songs or photographs are the applications that Dr.
Rekimoto sites for his creation. "Programs such as PowerPoint are
very good if you know exactly what you are going to say but they are no
good in an interactive environment where the presentation is constantly
changing."
Furthermore, Dr. Rekimoto points out that is
would be a great way for a school teacher to create an animated
presentation for students by simply picking and dropping software
objects from a laptop or PDA to a presentation screen.
He then
moves on to his Tile computer and he explains that we often define
computers in much too limited a fashion. Using a computer currently
excludes people who are not used to the graphical user interface. The
tile computer is a flat surface, which takes 12 4 inch X 4 inch tiles.
Each tile has a different but specific function. For example one tile
provides weather information, another address book information and so
on. To operate this computer the user (its actually designed for
children or old people) places the tile on the surface and the tile
light up with the relevant information.
Both of these
technologies contribute to Sony CLS's "Augmented
Surfaces" concept. This allows users to transfer digital
information between disparate computer environments such as Laptops or
PDA but also uses video cameras to enable the computer system to
recognize physical objects such as videotapes. Using projectors, white
boards and active displays it is possible to attach digital notes to
physical objects.
For example, a video tape lying on a surface
might have a digital note that says that has had one rough edit and that
the final cut must be ready the following day. So not only does Dr.
Rekimoto want to remove the digital boundaries between different
computers but also remove the boundaries between physical objects and
digital objects.
A Trend
Emerges
Indeed, Sony is not the only company
working on this concept. In fact, far from it, many large technology
companies, universities and research institutions such as Microsoft,
IBM, Philips, Intel, HP, Stanford Research Institute, the EU, Sun,
Stanford, Berkley, MIT and University of Washington, to mention but a
few, have similar ubiquitous computing and user interface projects on
the go. (Please see The Incredible Shrinking Computer
Pt. 1 and 2)
The next stop on the tour is wearable key or
personal id tag. This combines technology such as active badge (which
carries the users personal information in a radio based Id tag with a
concept called BodyNet that uses the static electricity in human skin to
create a network. That way different devices such as a wristwatch and a
PDA could communicate via the human body and be relayed, if necessary,
to the out side network using Bluetooth. "Currently, we have
BodyNet running at 9.6kbps but we can get the speed up to about one
megabit per second with a little work," says Rekimoto.
We
then retire to his office for a chat and some more demonstrations. His
disk is littered with Sony notebooks and PDAs. I counted six but there
were probably others lurking under the stacks of clutter. He booted up
his personal computer and showed explained that current graphical user
interface technology is pretty inadequate even for today's
computing needs.
"So one way to make current computers
more powerful would be to make them three dimensional. This means adding
the third dimension of a time line to the current 2D
interface."
His desktop, sports a WindowsXP interface with
a difference. It has a timeline. Move the timeline to last week your
last week's desktop appears. The user interface simply stores an
image of the file and on the desktop and keeps track of where the file
is moved. "Flat file folders are often a bad way of storing
information because a file may belong to several different
categories," says Rekimoto.
Certainly, concepts such as
Augmented Reality or Pick and Drop have been around for a while but its
only now that we have wireless technologies that will support high speed
data transfer that they can become a reality. The computing device is no
longer the ivory colored box tied to the one spot by its size, weight
and the spaghetti like mess of wires. In those days the GUI served us
well but now we are no longer constrained by the technology so the
sooner we move to new user interfaces like the ones that Dr. Rekimoto
and his team are developing the sooner we'll be able to remove
boundaries created by computers.
And because Sony is working on
these concepts I would be very surprised if one or more of these
technologies didn't find their way into PDA, sub notebook PCs and
other devices in the next 12 - 24 months. But that was one question Dr.
Rekimoto could not answer. Time will
tell.
It's Mobile Asia Week on
TheFeature! Check back daily for reports, analysis and in-depth
articles.
Niall McKay is a
freelance journalist based in Tokyo Japan. He can be reached at www.niall.org.