NASA, Xerox to Demonstrate 'Virtual Crew Assistant'

Intelligent conversation with robots -- long the bread and butter of science fiction authors -- soon may take another step closer to reality for astronauts on the International Space Station.

Scientists from NASA Ames Research Center in California's Silicon Valley and Xerox Corporation  will demonstrate a sophisticated, voice-operated computer system on June 26 at the Association for Computational Linguists' 25th annual meeting at the University of Michigan, Ann Arbor. Called Clarissa, the system was developed in an effort to ease astronaut workload.

"Clarissa is a fully voice-operated 'virtual crew assistant,' enabling astronauts to be more efficient with their hands and eyes and to give full attention to the task while they navigate through the procedure using spoken commands," said Beth Ann Hockey, project lead on the team that developed Clarissa at NASA Ames.

Plans call for astronaut John Phillips to complete the Clarissa training procedure onboard NASA's International Space Station as early as June 27 in preparation for later use of Clarissa.

"This will be the first use of the system in space," Hockey said.

Clarissa is "hands-free" and responds to astronauts' voice commands, reading procedure steps out loud as they work, helping keep track of which steps have been completed, and supporting flexible voice-activated alarms and timers.

Astronauts now perform about 12,000 complex procedures to maintain life-support systems, inspect space suits, conduct science experiments, perform medical exams and other routine tasks.

"Just try to analyze a water sample while scrolling through pages of a procedure manual displayed on a computer monitor while you and the computer float in microgravity," challenges astronaut Michael Fincke, who recently completed a six-month stay on the ISS. "To be able to speak to the system and hear the step-by-step instructions while my hands are free to complete the procedure will be like having another crew member aboard."

Because the system is required to always be ready to accept a voice command, the original version tried to process all spoken words, including conversations between crew members. As a result, Clarissa had difficulty discerning between conversations and commands given to the system.

In 2004, Clarissa lead implementer Manny Rayner of NASA Ames contacted Xerox researcher Jean-Michel Renders of Xerox Research Centre Europe in Grenoble, France, about a possible collaboration. They hoped that Xerox's experience in machine learning, linguistics and text categorization would increase the system's accuracy on the "open microphone" task.

"NASA wanted the system to be ready to assist at any time and without requiring artificial activation commands," said Renders. "Therefore, a simpler 'Star Trek' solution - like having crew members address the computer by stating a specific word such as 'computer' before posing a question or speaking a command to the system - wasn't a viable solution. We needed to improve the performance of the system in discriminating between commands and conversation."

The technology developed by Renders to address the NASA speech-recognition problem is also being used at Xerox to improve categorization results for printed or digital documents. Xerox researchers at Grenoble have developed a number of leading-edge software capabilities that make it easier for Xerox customers to manage document content.

The Xerox methodology allows Clarissa to more accurately analyze each utterance. It can recognize words, sentences and word context and can act on a variety of commands phrased in different ways. The system now looks at all the individual words within the sentence, takes into account the system's confidence that it has correctly recognized each individual word, and uses a sophisticated machine-learning algorithm to weigh the various pieces of positive and negative information.

This significantly increases the system's ability to determine the difference between commands directed to the system and side conversations. According to Renders, the improvements have cut the error rate of the system by more than half.

Clarissa currently supports about 75 individual commands, which can be accessed using a vocabulary of some 260 words. The team plans to increase the commands and add to the vocabulary in the future.

"Some commands are rather simple, but others are quite complex," Hockey said. "A lot of the time, you're just saying 'next' or 'go to step eight'. But you also might need to say something like 'cancel the alarm at 10:25' or 'set challenge verify mode on steps three through fourteen.'"

FMI: http://tc.arc.nasa.gov/projects/clarissa