Roger Smith
Mystech Associates
7900 Sudley Road, Suite 500
Manassas, Virginia 22110
ABSTRACT
This paper will discuss the development and application of an After Action Review (AAR) system for military training exercises such as Reforger, Central Fortress, Prairie Warrior, Team Spirit, and Ulchi Focus Lens. Attention will given to the role of software reuse in the development of the system. This will illustrate the speed and cost savings that can be accomplished by using this form of rapid implimentation. AAR systems are used to capture and evaluate the performance of military units participating in the exercise and to indicate the simulation's ability to provide a realistic environment for training these units. Prior to the development of automated systems exercise data was collected and manipulated manually. This severely limited the scope of information that could be extracted and presented during the course of the exercise. Detailed analysis had to be performed in the months following an exercise with the final report being provided to the commanders at a later date. This time` delay greatly diminished the usefulness of the information and eliminated any opportunity to learn from and improve upon performance during the training event. Automated AAR can provide almost immediate feedback to the training audience allowing them to mod- ify their behavior multiple times during the event and thus improve their performance.
1. TRAINING ENVIRONMENT
The US Army's Training doctrine (FM 25-100) describes training as a cyclical process. The cyclical analogy is used because training, whether it be for military personnel or personnel associated with any other type of organization, is seen as a continuous process. The training process is divided into 4 parts. These parts include developing objectives, planning, execution, and assessment of which feedback is a part. As defined earlier, feedback is a key aspect of the training process because at this point the person being trained is given information that allows him or her to assess how well their performance is meeting the recognized objectives. With this information in hand either the person being trained, the trainer, or both can take actions that will sustain or improve performance. Obviously, without appropriate feedback little corrective action can be taken.
While taking action to improve performance is the desired behavior that can be seen and measured. Feedback serves an equally important function in the cognitive processes as well. Specifically, helping to reinforce learning. Certainly the type of feedback, in terms of being positive or negative, has certain implica- tions from a Skinnerian point of view. It is assumed for our purposes that all feedback is positive. Given that assumption, it is widely recognized by cognitive psychologists that feedback is an essential ingredient to the internal learning processes of a person. One of the main benefits being that the person gains a sense of accomplishment and realizes progress (Hill, 1963).
Another condition of feedback is timing. This is generally classified as immediate or delayed. Each type has its place in the training arena. Traditionally delayed feedback is the predominate form used by the training community. Delayed feedback is generally the results of an evaluation instrument (e.g. a test or quiz). The evaluation can be written or some form of practical exercise (e.g. a driving test). This form of feedback provides the person being trained information on how well they have mastered a skill or body of knowledge. Since it comes after the training event there is no time to take corrective action and benefit from re-evaluation. Delayed feedback does not preclude practice during training but practice isn't always actively monitored. Therefore, any feedback given during a practice session may not be comprehensive enough to help the person or group being trained. If the trainee does not attain the establish standards for success then time may be wasted waiting for another training event. On the other hand, structured immediate feedback gives the training audience an opportunity to learn as they proceed. More importantly the trainees can see the results from changes in their performance. In an environ- ment like the military, where field training opportunities are expensive and infrequent, it is important that the training audience be given every opportunity to learn and grow as they proceed. Continuous feedback allows the persons being trained to confirm what they know or can do effectively, make changes in areas where deficiencies are identified, see the results of those changes, and repeat the process. The training event is then a cycle that is dynamic instead of a fixed set of sequential events. The benefits from using immediate feedback are that it improves the use of time, money, and other scarce resources.
Given that immediate feedback has numerous benefits why isn't this form of feedback more widely used? There are certainly many reasons but, tradition and technical limitations are two of the leading causes.
Tradition is simply that trainers have adopted practices widely used by academia. The primary form of feedback in a classroom environment is some form of test or paper. A learning event takes place, students are tested, they get the results of that test, and then move on to the next learning event. This same sequence is generally used by most trainers. The process certainly has value but is limited.
The technical limitations are another set of issues. The relatively recent innovations in computers and telecommunications have provided tools that can make a dynamic training process possible. Considering an environment like a field training exercise where the training audience is large, diverse, and geographi- cally spread out it becomes readily apparent that a number of obstacles must be overcome. The most significant obstacles are collection of pertinent data, analysis, and dissemination of useful information to the training audience. The TACSIM After Action Review User System (TAARUS) applies significant software and hardware technologies to provide a robust solution to these limitations.
2. AFTER ACTION REVIEW APPROACH
TAARUS empowers exercise analysts to provide immediate feedback during training events (figure 1). The lessons learned can then be used by the training audience to modify their actions and improve their proficiency in a cyclic manner during an exercise.
All data must be presented to the audience in a format which can be easily absorbed by non-technical war-fighting personnel. TAARUS tracks the events, time-lines, and forces such that the information can be displayed in map, graph, table, and summary forms which can be tailored to reach a General Officer level audience or a Company commander. In the intelligence environment, significant events include the placement of ground sensors, tasking of air sensors, ground truth picture, intelligence perception of truth, and terrain environment. TAARUS is able to use this information to generate movement tracks of significant units, FEBA locations through time, flight paths of air sensor platforms, the effectiveness of the sensors employed, and the evolving intelligence picture as it should have been perceived by the trainees.
Some of the key measures of effectiveness (MOE's) are:
When tasking aircraft to collect intelligence, operators must account for the operation of each mission within the overall intelligence plan. It is easy to lose sight of this in the desire to maximize the effectiveness of individual missions during specific periods of time. Since air assets are limited, it is important to manage the use of these to provide continuous coverage of enemy activities. However, during the heat of battle operators sometimes resort to "launching the fleet". This results in a glut of duplicate information during a single time period (figure 2). A resulting consequence is that assets are not available during later periods to monitor enemy activities. Additionally, the glut of reports produced by so many aircraft missions clogs the message delivery system and results in delayed knowledge of crucial data. The simulations accurately represents these occurrences and force the operators to live in the environment created by their actions. Unfortunately, in the past, these delays were wrongly attributed to limitations of the simulation. Without AAR tools to capture and display the events and their effects it was impossible to demonstrate the true cause of the problem while it was occurring. With TAARUS it is possible to show this information as it is taking place, thus creating a learning oppor- tunity for the training audience. A similar problem has been the placement of multi-vehicle sensors in order to provide triangulation of the source of RF energy. With automated tools the locations of the equipment can be plotted and measured to demonstrate when these are improperly deployed to operate according to their design specifications. These situations provide learning opportunities which have not been available to the training audience in the past.
Such a huge volume of data is generated by training simulations that it is difficult for an AAR analysts to locate and report on all significant events. To aid in finding key pieces of information TAARUS includes an instant query tool called Quick80 (figure 3). This allows the user to quickly scan all information in the database at a very high level providing clues to the existence of significant events which the analysts can then isolate for more detailed investigation. Alone, the Quick80 tool can also be used to provide a top level picture of the intelligence activity occurring in the exercise, a view that can be useful at the General Officer level.
TAARUS has a map interface into the relational databases (figure 4). This allows the user to plot the locations, echelons, types, orientations, and emitter characteristics of any of the units being detected. This allows the comparison of the true situation with what was perceived and reported. Sensor and player performance can be measured and opportunities to improve these presented. The maps contain Defense Mapping Agency (DMA) terrain, features, and raster map data. This is used to perform Intelligence Preparation of the Battlefield (IPB) functions just as they are done by the analysts being trained by the exercise. Working in a similar environment with similar capabilities the AAR operator can communicate with the training audience in a format that naturally matches their frame of reference.
Though maps, graphs, and windows can be used to provide excellent summaries of the data collected, it is often necessary to study the raw information being generated by the simulations. This is available in the TAARUS database and can be accessed by the analyst. The system contains a windowed interface which allows the user to extract and organize raw data from any combination of tables containing any combination of fields. This can be used to enhance the exercise events and cast a light on them which is not obvious in other forms. The extraction is often done with automated tools which combine the COMINT, ELINT, IMINT, and ground truth pictures at a given time for a given trainee. The information is then compared side-by-side, though it is actually stored in diverse locations and formats.
Finally, TAARUS is able to capture the image of any map, graph, table, window, or screen and store it for inclusion in an electronic slide show. AAR's may be conducted with a projector controlled by the TAARUS computer. This eliminates the need to create paper copies which must be stored and transported. It also overcomes limitations imposed by the time consuming process of transmitting images to a printer. When the AAR is conducted with the computer, questions can be explored which were not anticipated at the time the presentation was prepared. Since all data is immediately available it is possible to create map images and graphs on the fly in response to requests from the AAR audience.
3. COMPUTING ENVIRONMENT
Existing intelligence simulations, such as the Tactical Simulation (TACSIM), have not contained tools to aid in reconstructing the events of the simulation or their effects. Analysts have been forced to perform manual searches of printouts and data files looking for information of interest which then had to be collated and organized by hand in order to prepare an AAR briefing. This type of analysis is so time consuming that all but the highest priority requests had to be delayed or denied. Without an inordinate staffing level it was impossible to provide the type of analysis required. Even that which was undertaken could not be performed in real-time because of human limitations.
When TAARUS was first connected to TACSIM the automated flow and parsing of data significantly enhanced the functionality of the simulation without impacting the performance of the host computer (figure 5). Using a distributed client/server structure TAARUS became an integral part of the TACSIM simulation even though it operates on a separate computer using totally different data structures.
Due to the time and funding available to the TACSIM Project Office, much of the system was created from reusable software already available in the government or public domain. Since TAARUS analyzes the performance of intelligence cells, it was decided that the capabilities of these cells must be resident in the AAR tool. This provides the common look and feel of the data presented to trainees. To accomplish this the Warrior Intelligence Workstation was acquired from the Joint Prototyping Office. This provided the map, user interface, and report parser foundation upon which simulation specific tools were added. Additionally, all system enhancements performed under the Warrior program could be immediately incorporated into the TAARUS system.
Several public domain packages were used as the engines of the graphs, slide show, and image manipulation capabilities. The TACSIM office only had to fund the integration of the tools and slight adaptations (figure 6). Since the software is freely available to the public TACSIM is able to avoid the use of commercial packages which are expensive, must be purchased for each location, and often carry annual maintenance or licensing fees. These strategies resulted in lower costs and aggressive development time-lines.
4. CONCLUSION
The power of automated AAR systems is clear to military simulation sponsors. The requirements for these systems will continue to grow, even though the finances to add new capabilities is static or shrinking. The strategy used to create TAARUS from GOTS and public domain packages will allow the growth of the system at extremely low costs while providing proven components known to be useful and reliable.
REFERENCES
Aronson, Dennis T. and Briggs, Leslie J. (1983). Contributions of Gagne and Briggs to a Prescriptive Model of Instruction. In C. Reigeluth (Ed.), Instructional-Design Theories and Models: An Over- view of their Current Status, Hillsdale, New Jersey: Lawrence Earlbaum Associates.
Briggs, Leslie, J. (1977). Designing the Strategy of Instruction. In L. Briggs (Ed.), Instructional Design (4th ed.), Englewood Cliffs, New Jersey: Educational Technology Publications.
Cohen, V. B. (1983). Criteria for the Evaluation of Microcomputer Courseware. Educational Technology, 23(1), 9-14.
Hill, F. W. (1963). Learning; A Survey of Psychological Interpretations. Scranton: Chandler Publishing.
Headquarters, Department of the Army. (1985). Training: Field Manual 25-100. Washington DC: US Government Printing Office.
Common Operating Environment Working Group. (1992). Department of Defense Human Computer Interface Style Guide. Washington DC: US Government Printing Office.
Department of Defense. (1988). Defense System Software Development, DOD-STD-2167A. Washington DC: US Government Printing Office.
TACSIM Project Office. (1992). TACSIM After Action Review User's System: User's Manual. Unpublished Document.
United States Army Test and Experimentation Command. (1989). TACSIM Operations Manual Volume I: Simulator Operations. Unpublished Document.
AUTHOR BIOGRAPHIES
MAJOR GARY W. ALLEN is currently serving in the Army Acquisition Corps as the Simulation, Training, and Instrumentation Command Project Director for TACSIM. During his tenure as an Military Intelligence officer Major Allen has held a variety of positions that includes responsibility for intelligence analysis, production, and training. His military training includes MI Basic and Advanced Courses, Defense Intellegence Agency's Post Graduate Intelligence Program, and the Command and General Staff College. Major Allen holds a Ph.D. in Educational Technology from the University of Kansas, an MS in Telecommunications Management and BA in Communications from the University of Colorado.
ROGER D. SMITH is a Principal Simulation Engineer with Mystech Associates. He is responsible for developing simulations and tools to support the training missions of US and Allied forces. These have included air and ground combat models, intelligence collection and analysis algorithms, after action review systems, and system management tools. He has an M.S. in Statistics from Texas Tech University and a B.S. in Applied Mathematics from the University of Southern Colorado.