Calhoun

iniQiuiic^iul Ar{hiv« of tilt Mil vdl Poii^roduiit School

Calhoun: The NPS Institutional Archive □Space Repository

Theses and Dissertations

1. Thesis and Dissertation Collection, all items

2013-03

Three-Dimensional Space to Assess Cloud Interoperability

Beierl, Carl P.; Tschirley, Devon R.

Monterey, California. Naval Postgraduate School

http://hdl.handle.net/10945/32818

Downloaded from NPS Archive: Calhoun

DUDLEY

KNOX

LIBRARY

htt p://w ww. n ps. e du/l ib ra ry

Caflwuo is the Naval Postgraduate School's public access digital repository for research mate rials and institutiional putjlicatkios created by the NPS community. Calhoun is named for Professor of Mathematics Guy K. Caftiouo, NPS's first appointed — and putJlished — schoteily author.

Dudley Knox Library / Naval Postgraduate School 411 Dyer Road / 1 Univefsity Circle Monterey, California USA 93943

NAVAL

POSTGRADUATE

SCHOOL

MONTEREY, CALIFORNIA

THESIS

UNMANNED TACTICAL AUTONOMOUS CONTROL AND COLLABORATION SITUATION AWARENESS
by
Carl P. Beierl
Devon R. Tschirley
June 2017
Thesis Advisor:	Dan C. Boger
Co-Advisor:	Scot A. Miller

Approved for public release. Distribution is unlimited.

THIS PAGE INTENTIONALLY LEET BLANK

REPORT DOCUMENTATION PAGE

Form Approved 0MB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188) Washington, DC 20503.

1. AGENCY USE ONLY 2. REPORT DATE I 3. REPORT TYPE AND DATES COVERED

(Leave blank) June 2017 I Master’s thesis

4. TITLE AND SUBTITLE 5. EUNDING NUMBERS

UNMANNED TACTICAL AUTONOMOUS CONTROL AND COLLABORATION SITUATION AWARENESS

6. AUTHOR(S) Carl P. Beierl, Devon R. Tschirley

7. PEREORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

Naval Postgraduate School

Monterey, CA 93943-5000 _

9. SPONSORING /MONITORING AGENCY NAME(S) AND ADDRESS(ES)

N/A

8. PEREORMING ORGANIZATION REPORT NUMBER

10. SPONSORING / MONITORING AGENCY REPORT NUMBER

II. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government. IRB number _ N/A _ .

I2a. DISTRIBUTION / AVAILABILITY STATEMENT

Approved for public release. Distribution is unlimited.

13. ABSTRACT (maximum 200 words)

I2b. DISTRIBUTION CODE

The Unmanned Tactical Autonomous Control and Collaboration (UTACC) program is a Marine Corps Warfighting Laboratory (MCWL) initiative to build a Marine-robotic collaborative infantry fire team. The impact of robotic teammates on the situation awareness (SA) of the fire team is a central concern for this program. The proliferation of SA enhancing technology to the lowest echelons of Marine infantry forces often involves a tradeoff between focused and distributed SA due to limited attention resources. UTACC seeks a means to measure SA tradeoffs due to the incorporation of robots into infantry fire teams.

This thesis reviews present models of individual and team SA that are applicable to the military infantry environment and proposes individual and team models of SA that address the unique requirements of UTACC. The authors then applied SA principles to Coactive Design in order to inform robotic design. The result is a methodology framework using interdependence analysis (lA) tables for informing design requirements based on SA requirements. Future research should seek to develop additional lA tables for the entirety of the Marine Corps infantry fire team mission set.

14. SUBJECT TERMS

Unmanned Tactical Autonomous Control and Collaboration (UTACC), situation awareness, situational awareness, SA, team situational awareness, shared situational awareness, coactive design, interdependency, interdependence analysis, common ground, shared mental model, human robot interaction

18. SECURITY 19. SECURITY

CLASSIEICATION OE THIS CLASSIEICATION OE PAGE ABSTRACT

Unclassified Unclassified

17. SECURITY CLASSIEICATION OE REPORT

Unclassified

15. NUMBER OE PAGES

129

16. PRICE CODE

20. LIMITATION OE ABSTRACT

NSN 7540-01-280-5500

Standard Form 298 (Rev. 2-89) Prescribed by ANSI Std. 239-18

1

THIS PAGE INTENTIONALLY LEET BLANK

11

Approved for public release. Distribution is unlimited.

UNMANNED TACTICAL AUTONOMOUS CONTROL AND COLLABORATION SITUATION AWARENESS

Carl P. Beierl

Captain, United States Marine Corps B.S., United States Naval Academy, 2008

Devon R. Tschirley Captain, United States Marine Corps B.S., University of Washington, 2007

Submitted in partial fulfillment of the requirements for the degrees of

MASTER OF SCIENCE IN SYSTEMS TECHNOLOGY (COMMAND, CONTROL, AND COMMUNICATIONS)

and

MASTER OF SCIENCE IN

INFORMATION WARFARE SYSTEMS ENGINEERING

from the

NAVAL POSTGRADUATE SCHOOL June 2017

Approved by: Dr. Dan Boger

Thesis Advisor

Scot Miller Co-Advisor

Dan Boger, Ph.D.

Chair, Department of Information Sciences

THIS PAGE INTENTIONALLY LEET BLANK

IV

ABSTRACT

The Unmanned Tactical Autonomous Control and Collaboration (UTACC) program is a Marine Corps Warfighting Laboratory (MCWL) initiative to build a Marine- robotic collaborative infantry fire team. The impact of robotic teammates on the situation awareness (SA) of the fire team is a central concern for this program. The proliferation of SA-enhancing technology to the lowest echelons of Marine infantry forces often involves a tradeoff between focused and distributed SA due to limited attention resources. UTACC seeks a means to measure SA tradeoffs for the incorporation of robots into infantry fire teams.

This thesis reviews present models of individual and team SA that are applicable to the military infantry environment and proposes individual and team models of SA that address the unique requirements of UTACC. The authors then apply SA principles to Coactive Design in order to inform robotic design. The result is a methodology framework using interdependence analysis (lA) tables for informing design requirements based on SA requirements. Future research should seek to develop additional lA tables for the entirety of the Marine Corps infantry fire team mission set.

V

THIS PAGE INTENTIONALLY LEET BLANK

VI

TABLE OF CONTENTS

I. INTRODUCTION . I

A. UTACC VISION, PROGRESS, AND RELATED WORK . I

B. NECESSITY OF UTACC SA . 2

C. THESIS ORGANIZATION . 3

II. LITERATURE REVIEW . 5

A. UTACC CONCEPT OF OPERATIONS . 5

B. SITUATION AWARENESS . 5

C. SA MODELS . 7

1. Fracker’s Situation Assessment Model . 7

2. Endsley’s Model of SA . 9

3. Smith and Hancock’s Perceptual Model . II

D. COACTIVE DESIGN . 14

E. USMC INFANTRY MISSIONS . 18

F. SITUATION AWARENESS IN THE INFANTRY

OPERATIONAL ENVIRONMENT . 19

G. TEAM SA . 22

1. Endsley’s Team and Shared SA model . 24

2. Salas, Prince, Baker, and Shrestha’s Framework for

Team SA . 25

3. Sulistyawati, Chui, and Wickens’ Team SA Elements . 26

H. SA EVALUATION . 27

1. Freeze Methods . 28

2. Non-intrusive Methods . 29

3. Post-Mission Reviews . 30

4. Self-Rating Methods . 31

I. TEAM SA EVALUATION . 31

1. Endsley and Jones’ Shared SA Evaluation Methodology . 32

2. Saner, Bolstad, Gonzalez, and Cuevas’ Individual SA

Measurement . 33

3. Sulistyawati, Chui, and Wickens’ Team SA Evaluation . 34

J. CHAPTER CONCLUSION AND SUMMARY . 34

III. RESEARCH METHODOLOGY . 35

A. DEFINITION OF THE PROBLEM . 36

B. SA MODEL . 39

C. TASK BREAKDOWN AND lA TABLE . 39

vii

D. CHAPTER CONCLUSION AND SUMMARY

,42

IV. UTACC SA MODELS AND COACTIVE DESIGN RESULTS . 43

A. INFANTRY/INDIVIDUAL SA MODEL EXPANSION FOR

UTACC . 43

B. COMMON GROUND . 45

C. TEAM SA MODEL . 47

D. INTRATEAM SA AND OPD . 50

E. CONDUCT FIRE AND MOVEMENT lA TABLE . 50

1. Mission SA Requirements . 51

2. Enemy . 53

3. Terrain and Weather . 55

4. Troops and Fire Support . 56

5. Time Available . 58

6. Space . 59

7. Logistics . 60

F. CHAPTER CONCLUSION AND SUMMARY . 62

V. SUMMARIZING RESULTS AND RECOMMENDATIONS FOR

FURTHER RESEARCH . 63

A. SUMMARIZING RESULTS . 63

1. General Comments . 64

2. Benefits of Individual SA Assessment to UTACC UxS

Design . 64

3. Benefits of Team SA Assessment to UTACC UxS Design . 65

4. All-Marine Fire Team versus Marine-UxS Fire Team . 66

5. Differences between Marines and Machines . 66

6. Evaluation versus Comparison of Components and

Designs . 67

B. RECOMMENDATIONS FOR FURTHER RESEARCH . 67

1. SA Requirements Analysis for All Fire Team T&R

Events . 67

2. lA Tables . 68

3. Assessments for when UTACC Is Mature Enough . 68

C. CHAPTER CONCLUSION AND SUMMARY . 69

APPENDIX A. INFANTRY BATTALION THROUGH INDIVIDUAL

MARINE T&R EVENTS . 71

APPENDIX B. INFANTRY MOUT SA ELEMENTS . 79

viii

APPENDIX C. CONDUCT FIRE AND MOVEMENT T&R EVENT INF-

MAN-3001 . 85

APPENDIX D. CONDUCT GROUND ATTACK T&R EVENT INF-MAN-

4001 . 87

APPENDIX E. FIRE AND MOVEMENT lA TABLE . 89

APPENDIX F. REPRESENTATIVE LIST OF SA QUESTIONS . 103

LIST OF REFERENCES . 105

INITIAL DISTRIBUTION LIST . 109

THIS PAGE INTENTIONALLY LEET BLANK

X

LIST OF FIGURES

Figure 1. Visual Depiction of Fracker’s 1988 Situation Assessment Model . 8

Figure 2. Model of Situation Awareness in Dynamic Decision Making.

Source: Endsley (1995) . 10

Figure 3. Perceptual Model of SA. Source: Smith & Hancock (1995) . 12

Figure 4. Intersection between Environment and Agent within the

Consciousness. Adapted from Smith and Hancock (1995) . 13

Eigure 5. Support for Interdependence as an Orthogonal Dimension to

Autonomy and Some Opportunities this Dimension Offers. Source: Johnson (2014) . 16

Eigure 6. Coactive System Model Based on OPD. Source: Johnson (2014) . 17

Eigure 7. Infantry Eocused Model of Individual SA. Source: Endsley et ah,

(2000) . 22

Eigure 8. Team and Shared SA. Adapted from Endsley (1995) . 25

Eigure 9. Conceptualization of Team Situation Awareness. Source: Salas et

al. (1995) . 25

Eigure 10. Aspects of Team SA. Source: Sulistyawati, Wickens, and Chui

(2009) . 26

Eigure 11. MOUT SA Requirements: Primary Goal Structure. Source:

Matthews, Strater, and Endsley (2004) . 27

Eigure 12. Possible Shared SA States. Source: Endsley and Jones (1997) . 32

Eigure 13. Proposed Model of Individual SA. Adapted from Endsley (1995) . 45

Eigure 14. Model of Team SA from a Team Member’s Perspective. Adapted

from Sulistyawati et al. (2009) . 47

XI

Figure 15. Model of Team SA from the Fire Team Leader’s Perspective.

Adapted from Sulistyawati et al. (2009) . 49

Figure 16. Levels of SA Applied to OPD . 50

Figure 17. Conduct Fire and Movement T&R Event INF-MAN-3001 . Source:

DON HQ USMC (2013) . 85

Figure 18. Conduct Ground Attack T&R Event INE-MAN-400L Source: DON

HQ USMC (2013) . 87

xii

LIST OF TABLES

Table 1. Summary of Representative Situation Elements for Infantry SA.

Source: Endsley et al., (2000) . 20

Table 2. Team SA Theory Comparison Table. Source: Salmon et al. (2008) . 23

Table 3. Team SA Evaluation Comparison Table. Source: Salmon et al.

(2008) . 24

Table 4. SA Assessment Methods Summary Table. Source: Stanton et al.

(2013) . 28

Tables. UTACC SA lA Table Eormat. Adapted from Zach (2016) . 41

Table 6. UTACC SA lA Color Scheme. Source: Zach (2016) . 41

Table 7. lA Table: Mission SA Requirements . 52

Table 8. lA Table: Enemy SA Requirements . 54

Table 9. lA Table: Terrain and Weather SA Requirements . 55

Table 10. lA Table: Troops and Eire Support SA Requirements . 57

Table 11. lA Table: Time Available SA Requirements . 59

Table 12. lA Table: Space SA Requirements . 60

Table 13. lA Table: Eogistics SA Requirement . 61

Table 14. Infantry Battalion through Individual Marine T&R Events. Source:

DON HQ USMC (2013) . 71

Table 15. Infantry MOUT SA Elements. Source: Matthews et al. (2004.) . 79

Table 16. Eire and Movement lA Table . 89

xiii

THIS PAGE INTENTIONALLY LEET BLANK

XIV

LIST OF ACRONYMS AND ABBREVIATIONS

AAR	assistant automatic rifleman
AOA	analysis of alternatives
ADDRAC	alert, direction, description, range, (target) assignment, and (fire) control
ATC	air traffic control
AR	automatic rifleman
BAMCIS	begin planning, arrange reconnaissance, make reconnaissance, complete the plan, issue the order, and supervise activities
CAS	close air support
COA	course of action
COE	campaign of experimentation
CONOPS	concept of operations
DON	Department of the Navy
DRAW-D	defend, reinforce, attack, withdraw, and delay
EM	electromagnetic energy
EMECOA	enemy most likely course of action
EMDCOA	enemy most dangerous course of action
ESCM	fire support coordination measures
EOV	field of view
ETE	fire team leader
HMI	human-machine interface
HRI	human-robot interaction
HQ	headquarters
lA	interdependence analysis
lERs	information exchange requirements
lEREP	in-flight report
MCPP	Marine Corps Planning Process
MCTE	Marine Corps Task Eist
MCTP	Marine Corps Tactical Publication
MCWE	Marine Corps Warfighting Eaboratory

XV

MEDEVAC	medical evacuation
MET	mission essential task
METT-TC	mission, enemy, troops, terrain, time, and civil
METT-TSE	mission, enemy, troops and fire support, terrain and weather, time, space, and logistics
MOEs	measures of effectiveness
MOPs	measures of performance
MOS	military occupational specialty
MOUT	military operations in urban terrain
NPS	Naval Postgraduate School
OCOKA-W	observation, cover and concealment, obstacles, key terrain, avenues of approach, and weather
OPD	observability, predictability, and directability
OSMEAC	orientation, situation, mission, execution, administration/ logistics, and command/signal
RIP	rifleman
SA	situation awareness or situational awareness
SAGAT	situation awareness global assessment technique
SAEUTE	size, activity, location, unit, time, and equipment
SME	subject matter expert
SOS	system of systems
T&R	training and readiness
TA	time available
TPS	troops and fire support available
TW	terrain and weather
T&R	training and readiness
UAV	unmanned aerial vehicle
UGV	unmanned ground vehicle
USMC	United States Marine Corps
UTACC	Unmanned Tactical Autonomous Command and Collaboration
UxS	unmanned system

XVI

ACKNOWLEDGMENTS

First and foremost, we would like to thank our advisors, Dr. Dan Roger and Scot Miller, for your guidance and oversight throughout this entire process. You advocated on our behalf when it was necessary, pointed us in the right direction when we desperately needed it, helped us scope the problem when we lost focus, and let us run with things when we saw opportunity. Thank you both for a rewarding experience.

We would also like to offer a special thank you to numerous individuals who have been essential to our work. Dr. Matt Johnson, your instruction on SA, interdependency, and Coactive Design has been invaluable. Thank you for your time and assistance in helping us identify how all of these pieces fit together and aiding us in identifying where our efforts should be focused. Thank you to Mike Malandra and Tony Padgett with the Marine Corps Warfighting Laboratory for the opportunity to participate in and contribute to the UTACC project. Bob Daniel, we hope our discussions on the detailed inner- workings of Marine Corps fire team dynamics have been as helpful for you as they have been for us. Finally, and most importantly, we would like to thank our families: Amanda, Emelia, Shaunna, Clara, and Timothy. We would be remiss not to recognize the sacrifices you all have made over the course of our preparing this thesis, along with the support, patience, and encouragement you have so steadfastly offered.

xvii

THIS PAGE INTENTIONALLY LEET BLANK

I. INTRODUCTION

A. UTACC VISION, PROGRESS, AND RELATED WORK

This thesis is the seventh in a series supporting the Marine Corps Warfighting Laboratory (MCWL) development of the Unmanned Tactical Autonomous Command and Collaboration (UTACC) unmanned system (UxS). The UTACC UxS is a system of systems (SOS) consisting of robotic team members that will collaboratively operate with a team of Marines at a higher capacity as a team that far exceeds the operation of a single ground or aerial vehicle. A basic premise is that UTACC looks less like an operator controlling some type of Unmanned Aerial Vehicle (UAV), Unmanned Ground Vehicle (UGV), or combination thereof, and more like a UxS that is an integral part, a true “team member” of the larger United States Marine Corps (USMC) fire team.

The UTACC program development is using an incremental design process and similarities and overlapping material will undoubtedly exist between this thesis, preceding Naval Postgraduate School (NPS) theses, and concurrent theses. The first thesis developed concept of operations (CONOPS) for UTACC and highlighted the necessity of collaborative autonomy in the form of authentic collaboration between Marines and machines on a complementary playing field as teammates (Rice, Keim, & Chhabra, 2015). The second thesis offers a “red cell” critique of the CONOPS that analyzed the threats and vulnerabilities of the UTACC SOS, particularly those threats that were of a technological and information assurance nature (Batson & Wimmer, 2015). The third thesis utilized Coactive Design as a development method for human-robotic systems to provide design requirements that supported resiliency of the system through the flexibility of the fire team’s interdependent relationships (Zach, 2016). The fourth thesis identified measures of performance (MOPs) and measures of effectiveness (MOEs) to support the UTACC program (Kirkpatrick & Rushing, 2016). The fifth thesis conducted an analysis of alternatives (AO A) of prospective UAVs that would be capable of employment within the UTACC UxS (Roth & Buckler, 2016). The sixth thesis used those MOPs and MOEs previously identified by Kirkpatrick and Rushing to describe a

1

campaign of experimentation (COE) for UTACC that will assist in the realization of UxS as a functional system (Larreur, 2016).

Two other projects are in progress concurrently with this thesis. The eighth thesis is narrowing the scope of the MOPs and MOEs developed by Kirkpatrick and Rushing to further identify those MOPs and MOEs specific to the human-machine interface (HMI) in order to determine the appropriate sensor suite necessary for UTACC’ s information exchange requirements (lERs) (Kulisz & Sharp, 2017). The ninth thesis is identifying the lERs for a limited set of immediate action drills commonly performed by a USMC fire team (Chenoweth & Wilcox, 2017). Due to a paucity of known evaluation methods focused on human-machine teaming, the purpose of the current thesis is to define situation awareness (interchangeably referred to as situational awareness or SA) models, requirements, and methods of evaluation for the UTACC human-machine fire team.

B. NECESSITY OF UTACC SA

As will be reviewed in Chapter II, SA has been and will continue to be critical to decision making in infantry operations (Ends ley et ah, 2000). Eurthermore, the inputs on SA have increased rapidly alongside the evolution of technological advances. Rapid technological developments have created environments where a seemingly endless stream of data is available. Simultaneously, the processing speed of computing machines has maintained a similarly dizzying pace. The challenge is in leveraging the processing of the correct type of data to produce the desired type of information devoid of the unnecessary details. Unlike remotely operated vehicles in which the operator’s cognitive focus is on the vehicle or at best the individual task of the vehicle, a specific goal of UTACC is to reduce the cognitive load on the operator by leveraging the collaborative autonomy of the entire team.

Though the components of the UTACC team are separate physical entities, namely individual Marines and a UxS that combine to form a human-robot fire team, the focus of this thesis’ analysis of SA is on their collective mission as opposed to merely their individual SA requirements. To illustrate this point, a robot, like a human, has an array of sensors that can provide the necessary information to build the SA of that

2

specific entity. In some cases, a robot’s sensors are more limited in their field of view (FOV). In other cases, however, the UxS may be capable of sensing its environment in a way that a human is incapable of (e.g., infrared electromagnetic [EM] energy or other non- visible portions of the EM spectrum). Whereas the human brain automatically “fuses” various sensory inputs (for example, auditory, visual, and tactile), a UxS must be designed and programmed to fuse its various sensor inputs. Though each entity may share certain environmental data while other environmental data is unique to one entity, the collective SA of the fire team as a whole is ultimately the requirement for appropriate decision-making. In other words, individuals have individual data needed to perform their individual taskwork and shape their individual SA. In a team, however, individual taskwork is inevitably interdependent with other teammates’ individual taskwork, and the same is true for individual SA.

C. THESIS ORGANIZATION

This thesis is organized into four additional chapters. Chapter II is a literature review that explores the concept of SA and various SA models. Coactive Design methodology, the adaptation of SA into the infantry environment, SA evaluation methods, team SA, and SA evaluation techniques. Chapter III details the research methodology in evaluating various SA models and their use in the infantry environment. Chapter IV presents a UTACC team SA model, an illustrative SA requirements analysis for a common Marine Corps infantry fire team task, and various SA evaluation methods. Chapter V summarizes the results of the thesis and provides recommendations for future research.

3

THIS PAGE INTENTIONALLY LEET BLANK

4

II. LITERATURE REVIEW

A. UTACC CONCEPT OF OPERATIONS

The development of a UTACC CONOPS was the initial step within the research initiative put forth by MCWL (Rice, Keim, & Chhabra, 2015). Their thesis laid the groundwork and formulated a roadmap for follow-on research. Key findings and recommendations of Rice et al., including a threat and vulnerability analysis, the importance of realizing the risk in attempting to achieve some type of fully automated solution, and the necessity of explicit information requirements to support a complementary interface between robots and humans, all formed the basis of subsequent theses. This thesis makes use of their extensive task-oriented analysis for a reconnaissance mission derived from the Marine Corps Planning Process (MCPP) in order to form a basis for modelling UTACC SA.

B. SITUATION AWARENESS

A concrete and quantifiable definition of SA is necessary to build an effective method of evaluation. Multiple researchers have defined SA as either the “process of gaining awareness, the product of gaining awareness, or a combination of the two” (Salmon et al., 2008, p. 299). The initial significant and most widely accepted definition of SA is as a product, or a “state of knowledge,” that results from a process of “situation assessment” (Endsley, 1995, p. 36). Endsley used the following definition of SA for her work on measuring SA in military aviators: “Situation awareness is the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (1995, p. 36). Although this definition of SA is limited to being merely a product, when combined with her definition of situation assessment, Endsley produced a whole concept of SA that accounts for the interdependence between the process and product involved in SA (1995, p. 36).

A contemporary of Endsley defined SA as “the knowledge that results when attention is allocated to a zone of interest at a level of abstraction” (Eracker, 1988,

5

pp. 102-103). From this definition, the “focal region” is “the intersection of zones of interest with levels of abstraction” (Fracker, 1988, pp. 102-103). Fracker’s definition assumed that attention was a limited resource, and that SA was better with a narrowly scoped focal region compared to a broader focal region. Fracker defined the zones of interest in a similar manner as Endsley, but noted that they were not necessarily nested or encapsulated within each other. He defined levels of abstraction as the context of the assessment (Fracker, 1988, p. 103). Understanding of mission context, for example, is different from specific threat context. Different levels of abstraction, unlike zones of interest, were hierarchical. In this way, a pilot who understands mission intent can better understand the impact of a specific threat at a specific time and spatial location (Fracker, 1988, p. 103).

Smith and Hancock defined SA as not only a product or a process, but instead as an interconnected whole concept that could not necessarily be defined by the sum of its parts (Stanton et ah, 2013, p. 243). They defined it as an “adaptive, externally directed consciousness.”

[Smith and Hancock] take consciousness to be that part of an agent’s knowledge-generating behavior that is within the scope of intentional manipulation... [Smith and Hancock] view SA as generating purposeful behavior (behavior directed toward achieving a goal) in a specific task environment. The products of SA are knowledge about and directed action within that environment. [Smith and Hancock] argue that SA is more than performance. More fundamentally, it is the capacity to direct consciousness to generate competent performance given a particular situation as it unfolds. (Smith & Hancock, 1995, p. 138)

Smith and Hancock viewed knowledge about and decisive action in the confines of the environment as the results of SA, a distinctly different view from Endsley and Eracker (Smith & Hancock, 1995, p.l38). They argue that SA is not possible without prior experience that developed a certain “level of adaptive capability,” a notion similar to Tracker’s view of schemata (Smith & Hancock, 1995, p. 139). Thus, SA is cognition that drives the behavior that searches the environment for the cues that will enable effective action within the constraints of the task and environment (Smith & Hancock, 1995, p. 141).

6

Common to all of the preceding definitions of SA are the concepts of a process that generates knowledge from the environment and product, or state of knowledge, which represents a threshold for decision making in order to achieve an explicit goal. The research of both Tracker and Endsley was conducted in a military environment, which is mission-goal oriented in all its tasks. Smith and Hancock’s work went a step further and clearly distinguished between SA as a product of external or “environmental” goals versus introspection as a product of internal or “agent” goals (Smith & Hancock, 1995, p. 138). The agent (i.e., a Marine or a robot) must be performing an externally oriented task from which to derive the need for information about the environment that will inform the SA and decision making of the agent. The requirement that SA is task oriented is what limits the necessary information to only that which is pertinent to the task.

Within the context of UTACC, it is important to note how SA applies to the “machine” component of the Marine-machine team. As both agents perform these externally oriented tasks, the robot component will require both environmental information and a goal/mission-based context to analyze, compare, and make decisions just like a Marine.

C. SA MODELS

Multitudes of SA models currently exist and are in extensive use within military and aviation contexts, among others. Most models include some type of process in which environmental data is received, processed, and compared against pre-formulated schemata. Models differ in how they emphasize the importance of SA, as either a process or a product, both of which are tightly coupled to decision making as a whole. This section explores various SA models applicable to UTACC.

1. Fracker’s Situation Assessment Model

Fracker viewed the measure of a situation assessment model as one that indicated

methods that would improve SA and methods that would not (Fracker, 1988, p. 103).

Fracker modeled situation assessment as the intake of environmental data, the

comparison of that environmental data with long term memory “schemata,” and the

application of those schemata to the situation until the agent achieves a level of SA. Here,

7

“schemata” is the term used for knowledge that is stored in long-term memory. He saw the usefulness and application of those schemata as inversely proportional to the level of effort that working memory needed to expend. A brief example is useful for illustrating this key concept.

A veteran pilot with significant stored knowledge is able to rely on minimal environmental data in order to choose the correct schemata to apply to the environment and rapidly build SA with minimal working memory effort. A novice pilot, on the other hand, does not have the experience to conduct pattern matching and so must seek out a greater amount of environmental data in order to build SA using multiple rudimentary schemata (Fracker, 1988, pp. 103-104). The novice must expend more effort and needs more time to define the situation than the veteran, who relies heavily on rapid recognition and pattern matching to achieve the same quality of SA. Figure 1 is a visual depiction of Fracker’ s model of situation assessment, as interpreted by the authors.

Understanding of Focal Region

Perception of Focal Region Data Needs

Pattern Recognition

- j -

Working Memory

A Environmental Data k

Figure 1. Visual Depiction of Fracker’s 1988 Situation Assessment Model

8

Fracker’s inverse relationship between working memory load, depth, and quality of schemata is worth highlighting in particular because he viewed attention as a finite resource (Fracker, 1988, p. 102). A novice pilot must expend more attention than a veteran pilot does on non-situational assessment tasks like basic aircraft operation and therefore has less attention available to expend on situation assessment. The novice needs more attention resources than the veteran does in order to conduct situation assessment. A veteran pilot, on the other hand, has more attention resources but needs less in a similar situation. Knowledge and experience are the critical factors that enable rapid situation assessment that can deliver quality SA.

That point will have particular impact on UTACC given its context. Marines typically deal with situations that are at least slightly different from their schemata in some manner, regardless of training and experience. More knowledgeable and experienced Marines typically have more developed schemata available to them and they have experience matching environmental data to their schemata. The training and readiness criteria for Marine Military Occupational Specialties (MOSs) increase in complexity and scope over time and incorporate previous, narrower schemata into those more developed schemata (Department of the Navy: Headquarters United States Marine Corps [DON HQ USMC], 2013, p. 1-2). One of the key tasks of assessing the SA impact within UTACC will be the measurement of the robotic team member’s impact on the attention resources of the fire team. This will facilitate measurement and assessment of interface mediums and methods between the robot and other fire team members.

2. Endsley’s Model of SA

Endsley’s model of SA is depicted in Figure 2. She defined three levels that make up SA: perception, comprehension, and projection (Endsley, 1995, p. 35). Although the levels are hierarchically numbered, Endsley nested the levels within each other in her model because the three levels cannot exist in isolation (Endsley, 1995, p. 35).

9

Figure 2. Model of Situation Awareness in Dynamic Decision Making.

Source: Endsley (1995).

Perception requires the agent to gain awareness of pertinent data about relevant objects within the environment in order to comprehend their impact upon the environment, the agent, and the task (Endsley & Jones, 1997, p. 17). Once the agent comprehends the pertinent data within the situation, the agent can project the immediate next actions or the impact of the situation elements on their own next actions (Endsley & Jones, 1997, p. 17). In order to seek out the data necessary to comprehend the situation, however, the agent must project possibilities and probabilities (usually through some form of planning that provides an understanding of the task), comprehend the impact of those possibilities along with the likelihood of the associated probabilities, and then determine a means of seeking out the necessary data. Endsley ’s three levels of SA are therefore interdependent — an agent cannot achieve Eevel 1 SA without at least some measure of Eevels 2 and 3.

10

Endsley separated the decision cycle from “task/system factors” and “individual factors” that influence and affect an agent’s performance of the decision cycle (Endsley, 1995). The task/system factors that Endsley derived resulted from the focus of her work on military aviation and the effect of aircraft interaction on pilot SA. This has particular cross-applicability to the UTACC project because of the similarities between the pilot/aircraft interaction and the Marine/UxS interaction. Aside from the distinct difference between roles as operator versus collaborator, the system factors are still a valid construct to account for the impact of the system (UxS) on SA.

3. Smith and Hancock’s Perceptual Model

Smith and Hancock approached their model of SA from a different perspective than both Endsley and Eracker. What Eracker and Endsley called SA, Smith and Hancock defined as knowledge about the environment interpreted through the lens of the external task (Smith & Hancock, 1995, p. 138). What Eracker and Endsley called situation assessment. Smith and Hancock referred to as the behavior generated by SA that acquires task-relevant information from the environment. Situation assessment is the “agent’s solution to the problem of knowing those cues and demands in the environment that enable it to take action that aligns with the dicta of the arbiter of performance” (Smith & Hancock, 1995, p. 141). They used Neisser’s (1976) perception-action cycle as the framework for their model of SA and added what they termed the “invariant,” as shown in Eigure 3 (Smith & Hancock, 1995, p. 141).

11

Figure 3. Perceptual Model of SA. Source: Smith & Hancock (1995).

Smith and Hancock focused on the invariant as the driver of their SA model because interaction between the agent and the environment is necessary for SA to exist:

[The invariant is] the structure of the agent’s adaptation to the environment: It forms the linkage among information, knowledge, and action that produces competent behavior. Specifically, the invariant codifies the information that the environment may make available, the knowledge the agent requires to assess that information, and the action the knowledge will direct the agent to take to attain its goals. (Smith & Hancock, 1995, p. 141)

They derived the invariant from their view that SA requires the intersection of the agent and the environment during an externally driven task as depicted in Figure 4 (Smith & Hancock, 1995, p. 138). They used an example of commercial air traffic control (ATC) to make their point.

Experienced air traffic controllers had the requisite self-awareness to recognize either a lack of or loss of knowledge and adapt to it in order to increase their state of knowledge to a level sufficient to execute their task (Smith & Hancock, 1995, p. 142). By defining SA as the driver that depends on the invariant, and not a state of knowledge, they account for the situation where an agent’s knowledge is low, but the agent’s awareness of his or her current state of knowledge compared to the state of the environment is high. Thus, SA “not only supports the construction of the picture but also

12

guides the assessment of its integrity” (Smith & Hancock, 1995, p. 142). According to their definition, an agent with a low knowledge state could still be said to have good SA if he or she is aware of a lack of knowledge, the impact of that lack on his or her task performance, and the behavioral adaptations necessary to overcome that lack of knowledge.

Figure 1. An approach to defining situation awareness the presence of a normative arbiter of performance in the (SA) through explicit recognition of the centrality of ex- agent's task environment. The arbiter specifies for the temally oriented consciousness. The central (horizontal) agent task-relevant constraints and criteria for perfor- line provides an arbitrary distinction between exogenous mance. Adaptation to the environment requires the agent and endogenous orientations of consciousness and rep- adopt the arbiter s specification of constraints and resents a distinction between SA and introspection. performance variables. Cues and demands are stimuli

that unfold in the environment. The agent's internal con- straints are those that shape its intentionatity.

Figure 4. Intersection between Environment and Agent within the Consciousness. Adapted from Smith and Hancock (1995).

The UTACC project can benefit from Smith and Hancock’s model by using it to design and assess robotic team members on their understanding of mission and task intent. Robots that provide feedback when they need information but are unable to acquire it are more useful than robots who discount information needs that they cannot support. This will drive requirements for the design of sensors to support mission needs

13

instead of limiting mission capabilities based on the situation assessment capabilities of a particular robot’s sensor suite. On the other side of that coin, evaluations of robotic team member designs can inform commanders of what mission sets they are capable of supporting when in use.

At a deeper level, the identification of environmental data requirements in order to accomplish a given task is not a trivial undertaking. Most human members of the military require years of training and experience to develop the schemata necessary to accomplish operational missions. Robotic team members that utilize advanced intelligence and machine learning in order to adapt to unknowable situations may require similar time and training to achieve the same level of schemata. The benefit will be that each robot can learn from other robots’ experiences, thus shortening the training needs of all similar robots.

Experienced human team members understand the limitations of their different sensors and use a combination of means to gain a picture of the environment. Robotic team members will have to do the same, but they must understand their own limitations, and be able to reason and correlate similarities and differences between input means in order to do so. This will likely be the more difficult task than simply assessing the state of knowledge at any particular time.

D. COACTIVE DESIGN

Coactive