Geek in progress

At OOPSLA I listened with interest to a keynote talk given by Pattie Maes. Originally from Belgium, she became well known for her work on agents and artificial intelligence. Nowadays she works at the renowed MIT, where she focuses on ambient environments and technologies that should provide users with better experiences when moving in such an invironment.

The premise of her talk was the fact that there is a lot of information sitting idly around us. This information is not easily accesible. Moreover, relevant information that might help us understand our environment, or help us make decisions is rarely looked up when it is actually required. A typical example of this is shopping. There is much more we could learn, if the information about products was made available in a timely and intuitive fashion. Making this happen in a seamless manner is the goal of Pattie’s current work.

An initial step to making this happen is by equipping users with a wristband containing an RFID-reader. The RFID tag of various products can then be sent to a device capable of interacting with the information world, such as a PDA that can access the web.

This basic piece of equipment can be augmented by a gesture detection system. Such a system allows the users to rapidly browse the information, without requiring him to do awkward stuff of the PDA. A flick of the wrist might mean we want more information. The idea got a little scary when this is applied to people. Touching a person, e.g., by shaking his hand, could give you access to the public (ahem) information on that person. This would be handy when interviewing job candidates. It seems to me that there are much more scary possibilities, that invade our rights or privacy. I can imagine the police force finding this bit of technology quite handy when they have to hunt down people in a crowd. Legitimate use would be chasing criminals, but as we have seen in recent developments, people can be placed on a list of criminals (i.e., the no-fly list) for no other reasons than eating certain types of food.

Anyhow, the obvious step that follows the previous system is providing physical objects with an interface, i.e., creating meta-objects that can be attributed to real-world stuff. These object then provide the information people are interested in for each individual object.

Taking this further, makes us arrive at gaze-based interaction. Here, infra-red detectors determine the point at which somebody looks. A system like this can be used in musea, to broker information about specific details of a piece of art to visitors. Or it can be used to educate users about products.

Minimising the wristband yields a ring that can be used to point at objects. The target can then respond, e.g., by turning a LED green if the product matches your profile, or red if it does not. Said profilecould be a shopping list. Or a list with substances the user reponds allergic to. In this way he can select the goods that do not contain any of these substances without having to scan the list of ingredients over and over. Personally, I think this might be quite handy, although after several times shopping, you do get to know what you can buy and what not.

All of the above imposes technical challenges for both software and hardware:

• what is the user focusing on
• find relevant (personalised) information
• offer this information to the user in an unobtrusive way

I am not sure if this counts as unobtrusive but an example Pattie gave was the following. Objects could gain awareness, and try to attract your attention by lighting up when one passes by. An example is browsing a bookstore, where you can easily see which books match your interest.

A pretty cool example was that of the smart stickies. Current stickes cannot interact with people looking at it. Quickies are post-it notes that can be searched and located and send the writer reminders. For example, a to-do, a tag in a book, meeting data, etc. They can look up people in an address book an notify them automagically. They can also be used to keep track of personal items.

New research focuses on items that have I/O capabilities: I/O objects. An example is a pillow that can act as an interface to send messages to its peer, residing in your home. Another example was given by shutters that will automagically close to keep you out of the sunlight, while allowing light to pass to the rest of the room, so you can read in peace, without all that annoying sunlight getting into your eyes.

The most nifty example of an I/O object were the siftables. These are tiny machines, with their own screen, CPU, etc. that can detect neighbours. They can interact with each other when touching or residing near to each other. For example, if the screens show a face, the image is automatically adjusted such that the face looks at the neighbouring siftable. A possible use would be to teach people the concepts of OO-programming (which seemed to be the only relation the talk had with the main OOPSLA topics), as they have physical objects that can be manipulated and that respond to interaction with other object.

Pattie closed with mentioning the overall goal is to rethink how people can interact with the physical world by delivering just-in-time information in a non-disruptive way. This information would be accessed by different means than what we currently use: a keyboard and a mouse (or substitues for that, such as
a touchscreen).