SUSIE: Supervising Swarm Intelligence

Unmanned Vehicle Systems (UVSs) will considerably evolve within the next two decades. In the current generation of UV Systems, several ground operators operate a single vehicle with limited autonomous capabilities, whereas, in the next generation of UV Systems, a ground operator will have to supervise a system of several cooperating vehicles performing a joint mission, i.e. a Multi-Agent System (MAS)  In order to enable mission control, the autonomy of the vehicle and of the system will increase and will require new and richer forms of Human-system interaction.

The SMAART project demonstrated that interaction between a Human operator and a swarm intelligence is far from being trivial to implement. As this technology shows great promises for empowering large distributed robotic systems, the SUSIE project (SUpervision de Système d'Intelligence en Essaim, i.e. supervising swarm intelligence systems) was initiated in order to propose new modes of interaction beyond the simple "mouse clicks - waypoints" paradigm: in a large system the individual robot is probably not the right level of action for an operator, and it may even be superfluous to display individual robots' positions.

SUSIE is an Exploratory Research and Innovation contract (Recherche Exploratoire et Innovation: REI) of the French Defense Procurement Agency (Délégation Générale pour l'Armement: DGA, Mission pour la Recherche et l'Innovation Scientifique: MRIS) conducted by Télécom Bretagne (LUSSI department) and INRIA LORIA (MAIA team).

On-line Interaction

To properly supervise any system, an operator should be able to answer the following questions at any moment:

  • what is happening?
  • why is this happening?
  • what is going to happen next?

Among other things, this implies an understanding of the functioning of the system on the part of the operator and the access to intelligible information about the system. Understanding  the system comes through the construction of a mental model of the automation, in the same way that we have a theory of mind to interact with other Human beings. SMAART has shown us that basic interactive tools are not sufficient to allow an operator to build an efficient mental model of a swarm intelligence: this leads to frustrated efforts and poor performance.

In order to leverage the respective capacities of the Human operator and of the swarm intelligence system, it is necessary to develop new dedicated forms of interaction: new objects, concepts, modalities, etc. that stand at the border between the two representation spaces. In this perspective, using multi-touch interaction on a tabletop system seems promising: the continuous tabletop surface is very familiar to Humans and maps naturally to a grid of pheromones.

We plan to investigate other modalities, like the use of an oculometer in order to use the gaze direction to adapt the level of detail of the interface, or using sound to represent the state of the system as different audio textures.

Concerning the objects and concepts to be developed, it might prove interesting to transpose the control by policy paradigm to swarms (notably via spatial constrains) or the Playbook approach by specifying swarm-level activity patterns rather than individual commands.

Off-line Interaction

Currently, the appropriate behavior of a swarm intelligence depends strongly on the good configuration of the system by an expert (discretization of mission space, number of agents, evaporation and diffusion values, etc.)

We intend to develop solutions in order to help future operators of these system in the mission preparation stages, as they will not be IA experts but rather technicians. These decision-aid solutions could be build upon artificial learning techniques fed by simulation, models and manual configurations previously performed by experts.

Evaluation

The objective of SUSIE is not as much as to develop new modalities but to try them out and find what is more effective for the different tasks that a swarm is able to perform under Human supervision. Therefore, the different ideas will be evaluated through experimental campaigns in controlled environments on generic operational scenarios.

Funding

Partners

  • Télécom Bretagne
  • INRIA LORIA, MAIA Team

 

 

Contact: francois.legras@telecom-bretagne.eu

IMT Atlantique