Notice

Challenge 3: additional information and innovation challenge elements

Updated 10 August 2017

1. Challenge 3: additional information and innovation challenge elements

1.1 Make effective use of operator cognitive capacity, particularly by human-machine teaming

In complex, high workload environments where information is often incomplete, incorrect, high volume and broad in scope, cognitive limits are being reached. An answer is to exploit the particular strengths of our people and technology. To achieve this requires considerable improvements in the means of interaction between people and systems.

We want to progress beyond collaborative human-machine sensemaking, to develop approaches that might also enable collaborative decision making and intelligence analysis to support the activities of planning, plan refinement and mission execution. Ultimately we want humans and technology to be effective parts of the same team, be that with technology providing individual personal assistance, or where multiple humans and machines are providing the substance of team membership.

1.2 Example of possible innovation within a military scenario:

In the future, command centres will constantly receive fragmented information. They will use a ‘cognitive aiding system’ to record each key piece of information of interest. Relationships with prior information have already been identified and operators will be able to easily lodge this, potentially via voice command, with their machine team members. In the background, the system is automatically detecting new relationships, making operators aware of these and marking those likely to be of interest (identification being learned from on-going interactions). Operators will lodge with their machine team-members their hypotheses, reasoning and assumptions, which their machine colleagues are comparing against changes in the underlying data, providing timely and contextualised warnings when key elements will be supported or refuted by new evidence.

Where the machine team-members provide advice/warning of change, they can explain their reasoning in an intuitive human-understandable, trusted manner. There will be a constant dialogue between the system and the operator in which each is making the other aware of its status e.g. complete tasks, changing priorities, identifying critical activities, high workload / stress, and the degree of certainty regarding the information. In this way each team member (man or machine) shapes the nature of the on-going dialogue and work to the needs of the other.

2. What we are interested in specific to challenge 3

• innovative solutions that address equipment, technology and human capability, and prepare the human members of the human-machine team
• solutions that enable the advantages of a machine operating as part of a human team, for example in improving ways-of-working
• technical solutions which obtain and take account of the overall team context that is essential for it to be part of a human machine team
• solutions that simplify and reduce the extent of training required by human operators and analysts, and increase their productivity
• simple intuitive solutions that can be readily adopted by non-experts operating in conditions of high workload and stress
• proposals that start small and simple, can have initial rapid application, but have the potential to scale up
• approaches such as operator-driven, dynamic, exploratory visualisation.

3. What we are not interested in specific to challenge 3

• solutions which attempt to replace the human component or relegate it to a role which does not exploit human strategic level thinking
• solutions which fail to take account of the previously identified pitfalls with human-machine teaming solutions such as lack of trust, inability of technology to understand its limits of capability, uncertainty, unpredictability and bias without transparency to human team members
• solutions which are overly complex, require substantial training, and imply the forcing of people into unnatural ways of operating and behaving
• stand-alone solutions focussed on supporting more natural human machine interaction that do not include integration with other proposed solutions delivering information and processing capability
• solutions for human machine interaction that use static information visualisation solutions

4. Challenge 3 technical areas of investigation

Challenge 3 seeks original and innovative proposals to ensure that limited human cognitive capacity is applied to those parts of a problem where human consideration is an advantage. Presently too much capacity is spent on mundane activities such as recording and retrieving partial information and reasoning, monitoring multiple data streams, sorting relevant from irrelevant information, and identifying relationships, correlations and deficits.

5. Specific areas of interest for challenge 3

5.1 Memory

There are significant limitations on human ability to rapidly record and recall important information, either on a minute-to-minute or longer-term basis. This usually results in constant and ineffective attempts to record, exploit and integrate information that is available, be this in hand-written, audio and application represented forms.

We are interested in solutions to enable operators to rapidly and easily store a range of relevant information including interesting discoveries, facts, contacts, reminders, progress/status information, and documents/multi-media objects, and provide means to permit any or all information to be quickly and readily retrieved. Operators should also be able to log relationships between these, with machine assistance inferring others, supporting recall of all previously stored things and anything associated with them. Using the example scenario provided this might include the storage of observations of interest, hypotheses about correlations, with each being linked to related multi-media objects.

We are interested in novel, intuitive solutions which enable military operational staff to readily add to their own individual and group memory, enabling them to quickly and easily capture and log a wide range of information types, including evidence and observations gleaned from processed sensor outputs, potentially linking these back to the original sources and evidence.

5.2 Reasoning

In addition to the ability to assist memory by capturing key information, operators should also be able to store and represent outstanding questions, hypotheses and assumptions.

We’re interested in solutions where machine assistance can support human operators by continuously checking human reasoning against the constant stream of incoming information, and also applying further reasoning to generate new findings and identify related observations and evidence. This might also be used to assist the human operator in making new discoveries by operating on incoming data, and relating this to already stored material. We’re particularly interested in systems which can ‘explain’ their reasoning in a human intuitive manner, allowing operators to understand ‘why’ the machine assistance has proposed particular answers.

5.3 Teaming – relevant roles

In assigning roles between people and technology there is a need to recognise the strengths and weaknesses of each. For example, people have abilities such as: the ability for abstraction and pattern matching across substantial and diverse life-experiences; self-assessment and reflection; a degree of idiosyncrasy; creativity and inventiveness. Such abilities are essential in dealing with complex tasks involving high levels of uncertainty.

There has been a tendency to ignore many of these human strengths, and attempts to over-automate activities, as a result the human is relegated to a minder/overseer of a technological solution and/or assigned to perform a range of diverse functions which are too difficult to automate (the ‘leftover’ approach). This leads to disengagement, loss of job satisfaction and an inability to spot errors and correct for them. We’re interested in novel approaches that demonstrate automated solutions that assign more relevant roles to humans and machines, and that take better account of the needs, strengths and weaknesses of the human and technological components.

5.4 Teaming – individual and team interaction

It’s not enough to just assign relevant roles to human and machine components, as the end result of their collective but individual activities can be far from optimal.

We’re interested in solutions that provide more effective inter-working between human and machine components. This requires more than technology itself, and might include improvements to the way that people are trained and educated. It may also entail the machine component having a greater means to understand, not just precise questions presented, but other contextual factors too. This might better shape what it does, and also when and how it responds. We’re also interested in improving the human-machine relationship, which could include the machine presenting itself in a more human like manner to support more natural interactions. Finally, the machine component could also potentially learn from experience what the capabilities are of both itself and the human component. An example could be discovering where its limitations are and what the human operator is good at; hence linking the C3 (relevant roles) and C4 (individual and team interaction) topics.

We’re also interested in approaches that provide a more effective means for people to convey both specific detail and contextual information, which in a natural human-only setting, other team members would intuitively capture and account for. So a key question to be addressed is how will team work be undertaken in teams which include both human and non-human members? Also, how will the dynamic aspects of human-human team work, be recreated and potentially re-shaped in hybrid human-machine teams?