Meaningful Human Control

Meaningful human control is an important goal as digitalization and AI are implemented. This presentation aims to define MHC by addressing three key areas: design, operations, and learning. Key design issues for MHC include adopting a system approach, using human-centered design best practices, conducting task analysis to manage cognitive workload, designing alarms to support situational awareness (SA). Key operational issues include managing change (MoC), addressing error traps and maintaining physical and mental conditions to enable MHC. We observe that it can take 10 minutes to observe, understand and act correctly in crises. Main issues in learning from accidents must be to identify root causes including poor design and trying to understand reasons for human SA and actions. Accident investigations  should prioritize learning over blame.

MHC and human oversight are challenging because they depend on human strengths and weaknesses, system design, knowledge and training, and organizational factors like responsibilities, staffing, and work processes. MHC is more useful than human oversight because it ensures that systems, technology, and organizational structures are designed to keep humans in control of safety-critical operations, thereby preventing disasters. However, to be useful, MHC needs to be defined and specified.

My talk aims to define MHC by addressing three key areas: design, operations, and learning. Key design issues for MHC include adopting a system approach, using human-centred design best practices, conducting task analysis to manage cognitive workload, creating consistent interfaces for quick situational understanding, designing alarms to support situational awareness (SA), and establishing work processes that promote shared SA across teams.

Key operational issues include ensuring safety, managing change (MoC), addressing error traps and training, and maintaining physical and mental conditions to enable MHC in all situations. In a critical situation, we observe that it can take 10 minutes to observe, understand and act correctly in crises.

Main issues in learning from accidents must be to identify root causes including poor concepts/design and trying to understand reasons for human SA and actions. We have used “Human Error” as a starting point for analysis, not as a cause. Learning and understanding should drive change and improvement in governing values, prioritizing learning over blame.