ADELPHI, Md. -- Dialogue is one of the most basic ways humans use language, and is a desirable capability for autonomous systems. Army researchers developed a novel dialogue capability to transform Soldier-robot interaction and perform joint tasks at operational speeds.
The fluid communication achieved by dialogue will reduce training overhead in controlling autonomous systems and improve Soldier-agent teaming.
Researchers from the U.S. Army Combat Capabilities Development Command’s Army Research Laboratory, in collaboration with the University of Southern California’s Institute for Creative Technologies, developed the Joint Understanding and Dialogue Interface, or JUDI, capability, which enables bi-directional conversational interactions between Soldiers and autonomous systems.
The Institute for Creative Technologies, or ICT, is a Department of Defense-sponsored University Affiliated Research Center, or UARC, working in collaboration with DOD services and organizations. UARCs are aligned with prestigious institutions conducting research at the forefront of science and innovation. ICT brings film and game industry artists together with computer and social scientists to study and develop immersive media for military training, health therapies, education and more.
This effort supports the Next Generation Combat Vehicle Army Modernization Priority and the Army Priority Research Area for Autonomy through reduction of Soldier burden when teaming with autonomous systems and by allowing verbal command and control of systems.
“Dialogue will be a critical capability for autonomous systems operating across multiple echelons of Multi-Domain Operationsso that Soldiers across land, air, sea and information spaces can maintain situational awareness on the battlefield,” said Dr. Matthew Marge, a research scientist at the laboratory. “This technology enables a Soldier to interact with autonomous systems through bidirectional speech and dialogue in tactical operations where verbal task instructions can be used for command and control of a mobile robot. In turn, the technology gives the robot the ability to ask for clarification or provide status updates as tasks are completed. Instead of relying on pre-specified, and possibly outdated, information about a mission, dialogue enables these systems to supplement their understanding of the world by conversing with human teammates.”
In this innovative approach, he said, dialogue processing is based on a statistical classification method that interprets a Soldier’s intent from their spoken language. The classifier was trained on a small dataset of human-robot dialogue where human experimenters stood in for the robot's autonomy during initial phases of the research.
The software developed as part of the collaboration with USC ICT leverages technologies developed in the institute’s Virtual Human Toolkit.
Originally published by
U.S. Army CCDC Army Research Laboratory Public Affairs | July 27, 2020