Background Air Force Use of RPAs and Basing Locations The Air Force operates several types of RPAs: the MQ-9 Reaper; RQ-4 Global Hawk; and RQ-170 Sentinel. The MQ-9 Reaper RPA community has about four times the number of pilots and eight times the number of sensor operators assigned as compared to the next largest RPA community (the RQ-4 Global Hawk). Additionally, the MQ-9 Reaper RPA provides persistent intelligence, surveillance, and reconnaissance and strike capabilities against high-value, fleeting, and time-sensitive targets. It is operated by an aircrew that includes an officer pilot and enlisted sensor operator. See figure 1. The Air Force RPAs operate remote split operations, which divides the control of the RPA among geographically separated units. Remote split operations employ a launch and recovery ground control station unit aircrew who controls the RPA’s take-off and landing at an overseas operating location while a crew based in the continental United States (i.e., the Mission Control Element unit) flies the RPA the remainder of the mission via electronic links. Remote split operations result in fewer personnel deployed overseas, consolidates flying multiple aircraft from one location, and as such, simplifies command and control functions as well as the logistical supply challenges for the weapon system. RPA operations include Active Duty and Air National Guard personnel and locations. Figure 2 shows the location of bases involved in RPA training and MQ-9 Reaper RPA operational locations with the active-duty sites bolded. Demand for RPA Capabilities Over nearly two decades, the number of combat lines and flying hours for RPAs has grown substantially. Specifically, in 2008, the Air Force flew 33 RPA combat lines but in 2015, the number had increased to 60 RPA combat lines. A combat line is the measure of the capability to provide near-continuous 24-hour flight presence of an RPA over a specific region on Earth, to include time flying to and from a specific target area. In doing so, the RPA can provide air action against hostile targets that are in close proximity to friendly forces, gather intelligence, or, if necessary, employ its weapons to strike identified targets. Additionally, the number of combat flying hours has also increased from calendar year 2000, as shown in figure 3 below, and reached 4 million cumulative combat hours in March 2019. In March 2016, General Herbert J. Carlisle, then-commander of Air Combat Command, testified to the Senate Armed Services Committee’s Subcommittee on Airland that the RPA enterprise has been “a victim of its own success” with “an insatiable demand for RPA forces” that was taxing the capability of the community. To meet the demand for RPA pilots, the Air Force has pursued efforts to increase the number of RPA pilots. For example, the Air Force trained traditional manned-aircraft pilots to fly RPAs and placed graduates of manned-aircraft pilot training into RPA training rather than in advanced manned-aircraft training. In 2010, the Air Force created a dedicated RPA pilot career field (i.e., 18X specialty code) and developed a training program for pilots who specialize in flying RPAs. In December 2013, there were 1,366 Air Force RPA pilots, of which 249 were dedicated RPA pilots (18 percent). Six years later, in December 2019, the number of total Air Force RPA pilots had grown to 1,768, with 1,127 of those being dedicated RPA pilots (64 percent). Training Process MQ-9 Reaper RPA pilots and sensor operators complete multiple phases of training designed to generate combat mission capable aircrews within approximately a year of starting training. First, the pilots initially attend RPA Flight Training in Pueblo, Colorado, and then Undergraduate RPA Training at Randolph Air Force Base, Texas, which includes instrument qualification in simulators and an RPA fundamentals course. Second, they complete MQ-9 Initial Qualification Training at the formal training unit at either Holloman Air Force Base in New Mexico, March Air Reserve Base in California, or Hancock Field Air National Guard Base near Syracuse, New York. Finally, they are assigned to an operational squadron, where they complete unit-specific Mission Qualification Training that can vary in length. According to officials at two RPA bases, their respective Mission Qualification Training was taking between six to 10 weeks or as much as 17 weeks to complete. MQ-9 Reaper RPA sensor operators go through a similar pipeline. They complete courses on aircrew fundamentals and the basics of being a sensor operator at Lackland Air Force Base, Texas, and Randolph Air Force Base, Texas, respectively. Then, they complete training at the MQ- 9 Reaper RPA formal training unit at Holloman Air Force Base, New Mexico; March Air Reserve Base, California; or Hancock Field, Syracuse, New York. Finally, they complete unit-specific Mission Qualification Training in the operational unit at which they are assigned after graduation. Figure 4 shows the MQ-9 Reaper RPA aircrew training pipeline. The Air Force Has RPA Pilot and Sensor Operator Staffing Shortages and Does Not Track Its Progress toward Implementing Its Combat-to-Dwell Policy as Planned The Air Force does not have enough RPA pilots and sensor operators to meet its staffing targets, and it does not track its overall progress to access and retain sufficient quantities of RPA personnel that is needed to implement its combat-to-dwell policy as planned. More specifically, the Air Force has not consistently met its accession targets for RPA pilots and sensor operators and has had fewer RPA pilots and sensor operators than it has needed for most years between fiscal years 2016 through 2019. The Air Force has offered financial retention incentives to RPA pilots and sensor operators; however, it does not directly measure RPA pilot and sensor operator retention rates and retention concerns exist. Moreover, the Air Force does not track the overall progress being made from its accession and retention efforts to maintain a sufficient quantity of RPA pilots and sensor operators needed to implement as planned its combat-to-dwell policy—a policy intended to better balance RPA units’ time in combat operations with time spent away from those operations to accomplish other activities such as training. The Air Force Has Experienced Staffing Shortages in RPA Pilots and Sensor Operators The Air Force Has Not Consistently Met Accession Targets for RPA Pilots and Sensor Operators The Air Force met its accession targets for its RPA pilots in only one year during fiscal years 2015 through 2019 and it did not meet any of its sensor operator accession targets during those years. However, this is not a new trend. In 2014, we reported that the Air Force did not achieve its accession targets for RPA pilots in fiscal years 2012 and 2013 and recommended that the Air Force develop a tailored accession strategy for RPA pilots to help ensure that it can meet and maintain required staffing levels to meet its mission. The Air Force concurred with the recommendation and took steps to address accession issues for RPA pilots, such as having officers with RPA pilot experience serve at the U.S. Air Force Academy as instructors and as ROTC detachment commanders and instructors at several large, nationally recognized universities, thus giving attention to the career field among future Airmen. Because of these actions to address RPA accessions, the Air Force met the intent of our recommendation. Since then, however, the Air Force has not consistently met its annual accession targets from fiscal years 2015 through 2019, as shown in figure 5. As shown in figure 5, for the 5-year period between fiscal years 2015 and 2019, the average accession target fill rates for pilots and sensor operators were 95 and 88 percent, respectively. Air Force officials told us that they do not believe the RPA pilot career field is facing an accessions problem and thus there is no need to offer an accession bonus because the overall population of RPA pilots has been steadily growing year after year. These officials attribute the trend to the appealing RPA mission. Participants in 12 of 14 focus groups we conducted agreed that the ability to affect front line combat operations and missions every day was a positive aspect of the job. For sensor operators, Air Force officials told us that the number entering active-duty service reflects the number who had finished Basic Military Training and their first RPA-specific training course. These numbers would have been higher but Air Force officials stated they have determined that about 11 percent are disqualified during Basic Military Training sensitive skills screening. This screening involves identifying individuals upon entry into the service with behavioral or mental health issues and is used for, among other things, determining a trainee’s job classification and qualification for sensitive occupations. According to Headquarters Air Force officials, the 711th Human Performance Wing at Wright Patterson Air Force Base, Ohio, has ongoing research to help better identify the right types of airmen for RPA positions beyond the vocational aptitude battery test given to determine how qualified an enlistee is for certain occupations. They said that they expect the results of that research to be disseminated in early fiscal year 2021. The Air Force Generally Has Had Fewer RPA Pilots and Sensor Operators Than It Has Needed since 2016 According to Air Force data, the service has had fewer RPA pilots and sensor operators as compared to both their respective requirements and authorizations for almost the entire time between fiscal years 2016 through 2019. More specifically, the number of RPA pilot and sensor operator requirements has increased every year in support of the Air Force’s plan to create a new wing by 2024 that is needed to implement the combat-to-dwell policy. These Air Force requirements represent minimum essential resources needed to accomplish approved missions and functions that are valid, unconstrained, and realistic. After establishing the number of required positions, the Air Force fills these required positions to the extent possible based first on the number of those positions funded by Congress (i.e., authorizations) and then the number of trained and qualified personnel available to assign to those positions. Since fiscal year 2016, the overall number of authorized and assigned Air Force RPA pilots and sensor operators has increased. However, for a majority of the time in fiscal years 2016 through 2019, the Air Force’s number of assigned RPA pilots and sensor operators were less than both of their respective authorizations and requirements, as shown in figures 6 and 7. The overall number of assigned RPA pilots has increased; however, this trend has not been enough to meet the increased number of authorized positions in this RPA career field. For example, for RPA pilots, there was a 22-percent gap between authorizations (1,168) and assigned (908) in August 2015 which was similar to the 20-percent gap between authorizations (1,652) and assigned (1,320) in September 2019. The Air Force’s Rated Officer Retention Analysis report for fiscal year 2019 states that each of the four rated groups (pilots, combat system officers, air battle managers and RPA pilots) ended fiscal year 2019 in a deficit. Current projections indicate that the pilot deficit will continue into the near future. The report went on to say that while the number of assigned RPA pilots actually grew in fiscal year 2019, increases in the requirements for this career field reduced or negated the effect of the increase. Additionally, there was less than a 10 percent gap between the number authorized and assigned sensor operators during fiscal year 2016. However, by September 2019, a gap of 28 percent had developed (1,277 authorizations versus 919 assigned). The Air Force Has Provided Financial Incentives to Retain RPA Personnel but Does Not Directly Measure RPA Pilot and Sensor Operator Retention Rates and Retention Concerns Exist To encourage the retention of RPA pilots and sensor operators, the Air Force has provided financial incentives for many years. For example, the National Defense Authorization Act for Fiscal Year 2017 authorized RPA pilots to receive aviation incentive pay up to $1,000 a month and an aviation retention bonus up to $35,000 to those who are willing to extend their service. In addition, the Air Force has offered a number of financial incentives to RPA sensor operators. At various times in January 2010 through November 2019, RPA sensor operators were eligible for monthly aviation incentive pay, critical skills incentive pay, or special duty assignment pay to address retention issues and have occasionally been eligible for Selective Retention Bonuses. In November 2019, the Air Force offered a Selective Retention Bonus to RPA sensor operators who were eligible to reenlist and had between 17 months to 6 years of military service. To measure long-term retention trends among pilots other than RPA pilots, the Air Force calculates two retention metrics—the Cumulative Continuation Rate and the Total Active Rated Service rate. However, the number of RPA pilots (i.e., Air Force Specialty Code 18X pilots) is still too few to have enough data to calculate reliably these standard retention metrics since the career field was not established until 2010. Officials at Headquarters Air Force and Air Combat Command told us that to calculate the Total Active Rated Service metric, the Air Force would need about 20 years of data; however, the RPA pilot career field is too new to have that amount of data. These RPA pilots have a 6 year Active Duty Service Commitment, which begins at the end of their undergraduate RPA training at Randolph Air Force Base. According to Air Force officials, the first group of 18X pilots’ service commitments ended in fiscal year 2019. Senior leaders at an RPA base we visited said that due to the newness of the RPA pilot 18X career field, the Air Force does not currently have enough historical data to help predict retention trends going forward. They also noted that until the combat-to-dwell policy is implemented, it is unknown what effect it will have on RPA personnel retention. According to Air Force officials, the Air Force tries to retain about 60 to 65 percent of those who have completed their initial service commitment and are eligible to be retained. However, this target is based on the average aviation retention bonus acceptance rates (i.e., the percentage of pilots accepting the retention bonuses) for healthy and established career fields where the number of required positions are not substantially increasing and which are able to meet between 95 to 100 percent of their staffing requirements. However, as previously discussed, RPA pilot requirements have increased about 74 percent in the 5 years from fiscal years 2015 through 2019. Therefore, these Headquarters Air Force officials stated that use of the 60 to 65 percent target may not be an appropriate target for RPA pilot retention. In the case of RPA pilots, if the Air Force met that target, Air Force officials said the service would still be understaffed due to the growing requirements, so the retention target would need to be higher. Further, they stated that while aviation retention bonus acceptance rates are leading indicators of retention, they are not measures of actual retention rates and there are limitations to using this approach. For example, one limitation is that pilots may choose to stay in the Air Force but not take the aviation retention bonus to exercise more control and flexibility over their career. In these cases, actual retention would be higher than the aviation retention bonus acceptance rate suggests. According to the Air Force’s annual Rated Officer Retention Analysis reports we reviewed, the combined aviation retention bonus acceptance rates for RPA pilots both with and without previous manned aircraft experience completing their initial service commitment were approximately 55 percent in fiscal year 2016, 64 percent in fiscal year 2017, and 60 percent in fiscal years 2018 and 2019. Our comparison of the aviation retention bonus acceptance rates for RPA pilots with previous manned aircraft experience to those without that experience suggests that the pilots without that experience have consistently had lower bonus acceptance rates, as shown in table 1. As far back as April 2014, we reported that there were indications the Air Force could be facing challenges retaining RPA pilots in the future. Despite the existence of incentive payments, pilots in seven of the 10 focus groups we conducted at that time indicated that retention of RPA pilots was or would be a challenge. We recommended that the Air Force develop a retention strategy that was tailored to the needs and challenges of the RPA pilots to help ensure the Air Force could meet and retain required staffing levels to meet its mission. The Air Force took some steps to address RPA pilot retention, such as expanding RPA operations to an additional base to increase assignment choices and decreasing the number of combat lines that RPA aircrews were flying to reduce their workload. Further, in July 2018, officials said that the Air Force established a new division at Headquarters to serve as a focal point for overseeing RPA personnel matters for the service. Because of these actions to address RPA retention, the Air Force met the intent of our recommendation. However, in our current review, we found indicators of concern regarding RPA pilot retention. For example, officials in varying leadership positions in the Air Force raised concerns about RPA pilot retention. Air Combat Command officials stated that they assume that about 30 percent of RPA pilots each year will have to be replaced due to attrition. Senior leaders at one RPA base that we visited told us that not having dwell time as a break from constant combat operations negatively impacts RPA personnel resiliency and retention. They said that to get a break from combat operations, RPA personnel turn to the Air National Guard or separate. They noted that people join the Air Force to see and do things, not to be exposed to constant combat operations in less than appealing locations. Further, according to RPA officials, personnel stated in exit interviews that they wanted more temporary duty opportunities, deployments, exercises, and other opportunities for better career development. Similarly, senior leaders at another location we visited said that the lack of training and leadership opportunities affects retention. They noted that there are hundreds of pilots at Creech Air Force Base, but only one wing commander, and this has a chilling effect given the limited leadership opportunities available. With regard to RPA sensor operators, Headquarters Air Force officials stated that the Air Force does not have an RPA-specific sensor operator retention goal, but rather it generally aims to retain about the same amount as other career enlisted aviator career fields have historically retained, which is about 70 percent. However, according to a February 2017 memorandum, the RPA sensor operators experienced a steady decline in retention since 2012. This memorandum requested Special Duty Assignment Pay for RPA sensor operators stating that airmen in this career field were placed under enormous personal and professional demands. It also stated that in a 2-year sample, 2014-2016, the Air Force Personnel Center reported a 31 percent reenlistment decrease for first term RPA sensor operators, a 7 percent decrease for second term RPA sensor operators, and a 16 percent decrease for career RPA sensor operators. Specifically, the memorandum said that in 2016 the reenlistment rates for RPA sensor operators were 44 percent, 54 percent, and 74 percent for first-term, second-term, and career RPA sensor operators, respectively. In comparison, these rates were 19 percent, 22 percent, and 16 percent lower than the average rate across all Air Force Career Enlisted Aviators. The Air Force approved this Special Duty Assignment Pay for RPA sensor operators effective in November 2017. Additionally, effective October 2018 and again in July 2019 and November 2019, RPA sensor operators were eligible to receive Selective Retention Bonuses. Coinciding with the start of these financial incentives in fiscal year 2018, Air Force data showed increases in RPA sensor operator reenlistment rates as compared to fiscal year 2017 reenlistment rates (see table 2). While Air Force data show improvements in RPA sensor operator reenlistment rates, officials we spoke with shared concerns about retention-related issues specifically regarding sensor operators. For example, a senior leader at one RPA base we visited said that there is an acknowledged retention problem within the sensor operator community citing one of the factors being the perception among sensor operators that private contractors pay more than the Air Force. An Air Force document justifying the Selective Retention Bonus states that contractors are targeting experienced RPA sensor operators for six-figure salaries of greater than $100,000 per year. Similarly, a senior leader at one RPA base we visited stated that contractors are paying sensor operators 2 to 4 times as much as the Air Force does, essentially making the Air Force a pipeline for RPA personnel to become government contractors. Moreover, participants in each of the senior RPA sensor operators (i.e., E5-E9) focus groups that we conducted told us that they thought the retention bonuses and financial incentives were too small to matter in their retention decision- making. In a questionnaire we administered to the 105 participants across the 14 focus groups, nearly half (19 of 41) of the sensor operators responded they were “somewhat dissatisfied” or “very dissatisfied” with their total compensation versus 20 percent (13 of 64) of pilots who responded they were “somewhat dissatisfied” or “very dissatisfied.” The Air Force Does Not Track Its Progress in Implementing Its Combat- to-Dwell Policy within Its Projected Timeframe The Air Force does not track its overall progress of accessing and retaining sufficient quantities of RPA pilots and sensor operators needed to achieve its goal of implementing the combat-to-dwell policy in fiscal year 2024. Specifically, in a February 2018 briefing to Congress, the Air Force stated it planned to fully implement the combat-to-dwell policy in fiscal year 2024. Headquarters Air Force officials stated that in order to meet this 2024 goal, the Air Force is working to increase the number of trained RPA pilots and sensor operators through its accession, training, and retention efforts because they said it cannot implement the combat- to-dwell policy if it lacks sufficient quantities of available personnel. Several senior leaders at each of the locations we visited discussed the importance of achieving and sustaining a sufficient level of staffing that is needed to implement the dwell policy. One senior leader emphasized that the Air Force made “getting to dwell” its cornerstone promise. Officials stated that pilots and sensor operators are currently only able to accomplish training that can be done while completing combat missions because the RPA personnel are currently flying 24/7 combat missions. The January 2017 combat-to-dwell policy emphasized the need for the implementation of dwell time within the RPA community to allow these units to focus on either combat operations or training, but not both at the same time. This policy states that it is essential for preventing future risk to the mission and preserving the combat capability of the RPA force. Headquarters Air Force officials stated that they were hopeful that implementing the combat-to-dwell policy would improve quality of life and reduce burnout among RPA personnel by allowing them to take a break from combat operations to give them time to rest and train. Officials acknowledged that poor quality of life conditions for RPA personnel negatively affects retention. According to an Air Force instruction related to the RPA community, it is important to build a sustainable and healthy force and retention affects virtually all aspects of the Air Force’s effort to meet its goal of attaining the proper number of aircrew personnel. Further, it states that understanding the connection between the accession of new recruits, the training and production requirements of new aircrew members, and the ability of units to absorb newly trained aircrews into the structure and operations of the forces is critical to maintaining a healthy aircrew force and to achieve Air Force goals. However, the Air Force does not know its overall progress toward achieving its goal of having sufficient quantities of RPA pilots and sensor operators to implement the combat-to-dwell policy in fiscal year 2024 as planned. Thus far, Headquarters Air Force officials said that the Air Force has been focused on retaining as many RPA pilots and sensor operators as possible in an effort to meet the increasing staffing authorizations. The Standards for Internal Control in the Federal Government states that management should track achievements and actual performance, compare to plans, goals and objectives and analyze significant differences. Specifically, officials explained that it does not have a comprehensive metric (or set of metrics) which allows them to track changes in the number of its RPA pilots and sensor operators from its combined accession and retention efforts over a projected timeline. This prevents the Air Force from being able to compare its progress against its goal of having sufficient numbers of RPA pilots and sensor operators to fully implement the policy as planned by fiscal year 2024. The Air Force RPA officials stated that the Air Force does not have a metric (or set of metrics) that measures a “glide path to health and stability of the RPA workforce” by balancing both accessions and retention of RPA personnel in order to know when changes might be needed over time to achieve the goal of implementing the combat-to-dwell policy. Without such a metric (or a set of metrics), it is unclear whether the Air Force is on track to have enough RPA pilots and sensor operators to achieve implementation of its combat-to-dwell policy or to know if adjustments are needed to its accession and retention efforts or to the policy’s implementation timeframe. Taking such action is critical for the Air Force to be able to position itself to address long-standing RPA pilot and sensor operator shortages and documented challenges in the management of these communities through its combat-to-dwell policy. Absent such action, a key component of the Air Force’s workforce will not be well-positioned to meet its mission for the nation. The Air Force Has Not Fully Identified the Number of Instructor Positions Needed and Has Experienced Training Unit Staffing Shortages The Air Force Has Not Fully Identified Its Pilot and Sensor Operator Instructor Positions Needed at Its Holloman Air Force Base Formal Training Unit The number of active-duty RPA pilot and sensor operator instructor positions required at the Holloman formal training unit are understated and do not reflect the current training instructor needs. More specifically, the number of instructor positions needed were developed using a 2009 program of instruction with a length of 49 training days and were never updated to reflect changes to the syllabus length, which as of July 2019, was 83 training days. Air Force documentation showed that if 100 percent of the formal training unit’s currently identified active-duty instructor positions were filled, they could provide only 47 percent of the total course instruction currently identified. To provide the rest of the course instruction, the formal training unit relies heavily on contractors. Air Force information shows that, as of July 2019, contractors provided 53 percent of instruction, active-duty personnel provided 27 percent, and 20 percent remained unaccomplished (i.e., not provided). The Standards for Internal Control in the Federal Government states that management should use quality information to make informed decisions to achieve its objectives. Quality information is, among other things, current, complete, and accurate. Further, a 2017 report to Congress on the implementation progress of the Air Force’s actions to ensure a sustainable RPA operational force stated having maximum instructor staffing was critical to generating new RPA pilots. However, the Air Force continues to use the out-of-date, inaccurate, and incomplete number of active-duty RPA pilot and sensor operator instructor position requirements that were originally developed based on the 2009 program of instruction. Without using quality information, the Air Force does not fully know the number of active-duty RPA pilot and sensor operator instructor positions necessary for sufficiently training RPA aircrews. As such, it may not be fully addressing the challenges affecting the training unit’s staffing and ability to produce the needed number of aircrews to support the continued demand for RPAs and the implementation of its combat-to-dwell policy as planned. The Air Force Has Experienced Staffing Shortages at Its Holloman Formal Training Unit since Fiscal Year 2016 Since fiscal year 2016, the Holloman formal training unit has been unable to meet the authorized instructor position staffing levels even though the numbers of those positions are based on an out-of-date number of training days from the 2009 program of instruction that underestimates actual instructor requirements. In 2015, top senior Air Force leaders developed the Get Well Plan, and the Secretary of the Air Force and other top senior leadership helped develop the plan’s two goals to staff 100 percent of the positions for (1) instructors at the RPA pilot school and (2) combat RPA pilots. In the March 2017 report to Congress, the Air Force again emphasized that maximum instructor staffing was critical to generating new RPA pilots and that it had achieved this goal as planned and it would stabilize and sustain the Get Well Plan’s goals into the future. We found that both the number of RPA pilot and sensor operator instructors assigned peaked at the end of 2016 and early 2017 in accordance with this Air Force goal. However, the assigned numbers of both RPA pilot and sensor operator instructors have not stabilized or been sustained and have fallen since that time as shown in figures 8 and 9. Specifically, authorized RPA pilot instructor positions within the three RPA training squadrons at Holloman Air Force Base (i.e., the 6th, 9th, and the 29th squadrons) were filled at 75 percent (110 of 147) as of September 2019. That fill rate is almost 20 percent less than the highest fill rate for these positions in March 2017 (137 of 147, or 93 percent). Similarly, authorized RPA sensor operator instructor positions within these same training squadrons as of September 2019 were filled at 58 percent (82 of 141), down from the highest fill rate of 91 percent (128 of 141) in November 2016. A training official explained that the inability to maintain the level of staffing, even when considering it was an underestimation of the true requirement, is an example of the issues experienced in the RPA community. He stated that when RPA pilots and sensor operators at squadrons leave the Air Force that means there are fewer of them overall available to conduct the missions and to be sent to the formal training unit to serve as instructors. Fewer instructors at the training unit means a greater workload on the instructors already there, which affects the morale of the instructors and may result in those individuals leaving the Air Force. It also limits the ability of the formal training unit to meet the expectations of producing newly trained aircrews that are supposed to fill the staffing need at the squadrons. Overall, this cycle contributes to the challenge the Air Force faces in being able to retain and produce RPA pilots and sensor operators. Moreover, the gap in instructor staffing is compounded by a majority of instructors arriving at the Holloman formal training unit not having prior operational squadron-level instructor experience, according to training officials. According to an Air Force instruction regarding RPA training, any aircrew member designated for instructor duties at a formal training unit should already be an instructor in the applicable aircraft. However, for example, at Holloman’s formal training unit, officials told us that for the training session from August 2019 to May 2020, 17 of 25 of the new incoming instructors did not have previous squadron-level instructor or evaluator experience. In these instances, they said the new instructors would need additional training to qualify them fully to teach certain classes. According to training officials, being an instructor at a formal training unit is not the same as being an instructor at an operational squadron. For example, in an operational squadron, an instructor is expected to take an individual that is fully qualified in the aircraft and get them up to speed on the squadron’s specific mission and to assist in increasing the squadron’s overall level of efficiency through continued supervised training. At the formal training unit, however, instructors are laying the foundation for new aircrew students that are not familiar with the aircraft, its operation, or its various mission sets. Officials stated that because the formal training unit is receiving inexperienced instructors rather than fully qualified ones, the training unit must provide more upgrade training to these student instructors to qualify them to teach any classes. While the instructors are going through the upgrade and any other training needed to become fully qualified, they are filling an instructor staff position but not fully contributing to the development of new RPA pilots or sensor operators. Air Force training officials acknowledge that staffing at its Holloman formal training unit is a concern and that they need more instructors. They said that shortening the length of training was one approach to addressing the instructor gap and, in June 2019, the commander of the 19th Air Force (Air Education and Training Command) directed syllabus modifications. According to training officials, the modifications suspended about 15 percent of the training and thereby, shortened the length of the course. These modifications are scheduled until the end of October 2020 unless deemed necessary to extend them into fiscal year 2021. The Air Force Has Not Fully Implemented the Initiatives It Developed to Address Quality of Life Issues Affecting the RPA Community and Long-Standing Concerns Remain In 2015, the Air Force developed over 140 initiatives to address quality of life challenges facing its RPA units but has not fully implemented them. While the Air Force has been aware that the RPA community faces such issues as work-related physical and mental ailments, lack of base services, and other challenges to its quality of life, long-standing concerns we have identified previously, as well as others, remain. The Air Force Has Not Fully Implemented the Initiatives It Developed to Address Quality of Life Issues Affecting the RPA Community The Air Force’s Air Combat Command established the Culture and Process Improvement Program (CPIP) in 2015 to identify and address stress and quality of life issues within the Air Force’s MQ-1 Predator and MQ-9 Reaper RPA communities. This effort collected nearly 2,500 inputs from the RPA community through surveys and in-person engagement. Following this input, the Air Force developed over 140 initiatives to address concerns in eight different areas, such as missions, quality of life, locations and basing options, and training. These initiatives varied widely in scope and specificity and they addressed the RPA enterprise, such as pilots, sensor operators, intelligence personnel, and maintainers across active-duty personnel and the Reserve component. In February 2018, the Air Force briefed Congress, reporting that 57 percent of CPIP initiatives were complete and 43 percent were ongoing. According to Air Force officials, examples of initiatives completed include: expanding RPA combat operations to Shaw Air Force Base, South Carolina, to provide additional assignment options; establishing an advanced weapons instructor course specifically for redesignating MQ-9 Reaper RPA squadrons from “Reconnaissance” to “Attack;” establishing a medal to specifically recognize the contributions of personnel that operate and support the RPA enterprise; and, authorizing RPA aircrews to log combat time when flying an aircraft within designated hostile airspace, regardless of the aircrew’s physical locations. The CPIP report finalized just over a year later in June 2019 states that the Air Force had achieved “an almost 90 percent solution” and the most significant of the initiatives had been accomplished. It went on to say that there were 17 initiatives remaining open at that time and that the Air Force would no longer track those initiatives because they had reached the point of diminishing returns. Additionally, the office established to track the CPIP initiatives was closed because Air Combat Command officials told us that the office is no longer needed and all remaining initiatives have been staffed to other offices of primary responsibility. However, in our review, we found examples of quality of life initiatives labeled complete where the objective had not yet been fully achieved. Examples we found include: an initiative to create a new MQ-9 RPA wing to be led by an RPA pilot was labeled with a status of “complete” even though Headquarters Air Force officials confirmed that no new MQ-9 Reaper RPA wing has yet been created; an initiative to have aircrews’ shiftwork schedules rotate every 4 to 6 months; however, each of the squadrons at the RPA operational bases we visited had a shift work schedule that rotated for 5 to 8 weeks; an initiative to grant appropriate clearances to allow medical and chaplain personnel into all RPA operational areas; however, at one location we visited, medical officials and a chaplain we spoke with said that they do not have the required clearance levels to meet with RPA personnel within their secured facilities; two initiatives to improve spousal opportunities, although one vaguely stated that the “Air Force should think big and think flexible as it needs to consider society’s shift to the two-income family” and the other called for providing better family services and support. However, we found that while these services may exist at RPA bases, they are not always accessible to RPA personnel or their families for a variety of reasons, as we discuss below; an initiative to provide childcare support for workers performing 24/7 operations, although we found childcare was not available at certain facilities we visited; and, an initiative to make Creech Air Force Base its own installation, add a Missions Support Group, and improve base infrastructure and services. Creech did receive its own command authority and is no longer an auxiliary facility under Nellis Air Force Base and a Mission Support Group was established in July 2019. However, its plans to create officer and non-commissioned officer housing and an additionally medical facility are not expected to be completed until between fall 2021 and fall 2022, according to a Creech official. According to Air Force officials, an initiative marked as “complete” means that the Air Combat Command CPIP office had completed its portion of the initiative and another Air Force entity had taken it over for further action as necessary and may still be in process. Therefore, the 57 percent of initiatives that the Air Force reported to Congress in February 2018 as completed and the “almost 90 percent solution” discussed in the June 2019 CPIP final report may not present a transparent account of what has been completed and what remains to be accomplished. Reporting planned tasks as “complete” as the Air Force did could create perception gaps regarding the effects of CPIP. Interviews we had with senior leaders at multiple bases yielded concerns that CPIP is effectively over without accomplishing key objectives and that CPIP is going to be perceived as a failed promise by the Air Force. Quality of Life Challenges Affecting the RPA Community are Long Standing and Still Continue Along with the CPIP initiatives developed in 2015 as discussed above, academic studies published since 2010 and our previous 2014 report on RPA job dissatisfaction identified challenges facing the RPA community. For example, in April 2011, a study by researchers at the U.S. Air Force School of Aerospace Medicine found that there are several important operational stressors to consider when assessing the health and well- being of RPA operators. More specifically, the researchers noted, for many operators that participated in the study, the most commonly cited stressors associated with occupational stress included, but not limited to, the following: (1) long hours and low manning; (2) frequently changing shift work and shift changes; (3) geographically undesirable locations; (4) limited base resources and rural settings; and (5) human-machine interface difficulties such as poor ergonomics and temperature control of work stations. The study concluded that it stood to reason such stressors could lead to both physical and psychological distress when faced on an unending basis. Three years after the issuance of that study, in April 2014, we reported that RPA pilots faced multiple, challenging working conditions, including work shifts that frequently rotate, long hours, and increased workloads. More specifically, we reported in 2014 that In seven of the 10 focus groups conducted at that time, RPA pilots said continuously rotating to new shifts disrupted their ability to spend time with their family and friends and caused sleep problems. They said that these changes to their sleep schedules resulted in significant fatigue both at home and when they returned to work. In seven of the 10 focus groups conducted at that time, RPA pilots described working long hours because, for example, they had to perform administrative duties and attend briefings in addition to flying their combat shifts. High work demands on RPA pilots limit the time they have available for training and development and negatively affects their work-life balance. During the course of our current review, we heard various positive comments about how RPA pilots or sensor operators like the RPA mission and being able to contribute on a daily basis to combat operations. However, as discussed below, we also found examples of how long-standing challenges that others and we reported about years ago regarding the physical and mental health of RPA personnel and the availability of base support services continue to exist. Physical and Mental Health Concerns Shift Work and Sleep Issues In 12 of the 14 focus groups we conducted, participants stated that the frequent rotations are a key challenge of shift work and that their schedules rotated approximately every 5 to 8 weeks. However, members of the Human Performance Team at Creech Air Force Base stated studies have shown that it is better for individuals to stay on shifts for longer periods of time, such as 3 to 4 months, to allow their circadian rhythms to adjust. Additionally, focus group participants told us that rotating shift work is difficult for RPA personnel’s relationships. Participants in 13 of the 14 focus groups indicated that shift work has negatively affected their family or social life. Additionally, rotating shifts and the limited time with family creates a dilemma on weekends for personnel, especially for those on the midnight shift that covers roughly midnight to 8 a.m. These individuals must decide whether to maintain their work sleep schedule which limits time with family, or instead to align with their family’s sleep schedule which limits their ability to adapt to the work schedule. Some comments from participants include “I destroy my circadian rhythm to spend time with my kids” and “Shift work is disruptive to lives. It is hard to be tied into the community. Shift work can be really isolating.” Crew rest is compulsory for aircrew members prior to performing any aircraft operations. Aircrew members are individually responsible to ensure they obtain sufficient rest during a crew rest period. If crew rest is interrupted, individuals should immediately inform appropriate leadership and will either begin a new crew rest period or not perform flight duties. According to health officials at one of the bases, though, it is well known that RPA aircrew members often do not accurately report how much rest they get. Participants in one focus group agreed with this statement and said that they do not want to be restricted from flying and affect the mission and cause the work to fall on other squadron members. Participants in 12 of our 14 focus groups that we conducted stated that it is difficult to get adequate sleep. Sample participant comments include: “I can’t sleep anymore. Before the military, I could get 10 hours of sleep. Now it’s like 2-4. You’re physically and mentally exhausted.” “I feel perpetually tired. I haven’t felt healthy in years.” “We did an internal survey of how much sleep people on nights for months at a time were getting, and it was like 3-4 hours. And they are flying combat for 8-12 hours at a time.” Back, Eyes, and Other Physical Issues In 12 of 14 focus groups, participants said the working environment is harmful to health in areas such as the neck, back, eye, and hearing. Participant comments included: “I’ve been losing hearing …over the last 6 years from computer fans, air conditioning units, the use of multiple communication devices, etc.” “Just sitting in the seat for 8, 10, or 12 hours affects our posture. It is bad on our backs. I didn’t have lower back problems, and I work out a lot, but I started having lower back problems.” “My eyesight has been getting worse.” See figure 10 for an example of a pilot flying a simulated mission in an RPA cockpit. During our site visits for this review, participants in 14 focus groups that we conducted said that maintaining fitness was difficult. They said they are not motivated to work out as they are frequently exhausted after flying long shifts and then completing other extra duties as well. Further, participants in 11 of 14 focus groups told us that nutrition is difficult for RPA crews. For example, participants said that they consume energy drinks, soda, and sugary foods to stay awake during the midnight shift. Studies have shown negative psychological effects on RPA aircrews. An Air Force study from 2010 of the psychological attributes critical to the performance of RPA sensor operators noted it is important that RPA sensor operators be aware prior to training that they would be targeting and destroying enemy combatants. It stated that it was likely that some candidates might choose not to become sensor operators once they fully understand their role in precision-strike operations. These motivational attributes were not deemed critical to performance, but were deemed critical to retention and job satisfaction. Participants in 10 of our 14 focus groups we conducted said that some crew members—either themselves or others—did not initially understand what the job entails, such as killing. One focus group participant noted “the first time you know what you’re getting into emotionally is the first day of training at Holloman, which is too late because you already have wings.” Participants in 13 of 14 focus groups we conducted stated that witnessing or causing violence has a negative psychological impact but two-thirds of our survey respondents (66 of 105) said that the Air Force has not assessed their level of stress and fatigue related to their role as an RPA pilot or sensor operator. A study published in 2018 described how RPA aircrew members are affected by their own actions in combat as well as by connections with either people who they target or support on the ground regardless of the physical distance separating them. One focus group participant commented “F-16s drop and then go. For RPA aircrews, we get in and we are there for 20 hours. We watch who we employ weapons on, then get the battle damage assessment, including seeing body parts…on the ground.” Availability of Base Services Issues RPA personnel stated that their base’s services are not consistently available to RPA aircrews rotating shifts to conduct missions 24 hours every day or to their families as they live in remote locations. Collectively, participants in all 14 focus groups we conducted expressed concerns about the availability of services such as medical services, childcare, spouse and family support services, and base locations and housing. Some level of health care is provided at each RPA base we visited, but the extent to which these services are available varies. For example, the Cannon Air Force Base mission briefing we received in June 2019 noted some “sustainability challenges” such as the base’s inadequate availability of specialty medical care. The briefing noted that the base had made over 2,000 referrals related to 10 areas of specialty medical care. Additionally, because these referrals were to facilities outside the local area, the base had incurred about $500,000 in travel reimbursements for this medical care—the highest of all Air Force locations—and about $21 million in TRICARE expenses per year, according to officials. Further, we found examples during our site visits of health services without adequate staffing. For example, during our visit to Shaw Air Force Base in May 2019, a medical technician stated that Shaw had two medical technicians for the RPA community though staffing documents state they are supposed to have six medical technicians and two doctors. At Creech Air Force Base, we visited the medical and dental facility and learned that a psychologist position had been unfilled for 9 months as of our visit in August 2019. We also found that the hours of available medical services are limited and not convenient for shift workers such as RPA aircrews. For example, officials at Creech stated occupational therapy is offered only once a month, optometry twice a month, and nutrition on an as-needed basis. In addition, Creech has two family health personnel, a behavioral health officer who is available every Wednesday and Friday, and one flight surgeon who comes over from Nellis Air Force Base is available twice a week. A 2018 internal assessment done for Creech leadership estimated that 20,714 man-hours are wasted each year due to personnel needing to obtain medical services, the equivalent of losing 11.5 people in a given year. To address health issues, Creech Air Force Base has a Human Performance Team that includes chaplains, religious affairs airmen, a psychologist, a mental health tech, and a physiologist. While team members are physically located at Creech, they told us that they are also responsible for RPA units at all the bases under the same wing, including Creech, Ellsworth Air Force Base, South Dakota; Whiteman Air Force Base, Missouri; and Shaw Air Force Base, South Carolina. Further, at Shaw Air Force Base, a religious affairs airman made similar comments about serving a large variety of military personnel, not just the RPA community and a chaplain at Cannon Air Force Base said that he can be assigned responsibility for up to as many as 2,000 to 3,000 people at a time. Childcare is not limited for 24/7 shift workers at certain facilities although a CPIP initiative called for childcare support for workers performing 24/7 operations, citing the Missile Care childcare program offered at Minot Air Force Base. To this end, the Air Force established two programs, RPA Care and RPA 2 Care. The RPA Care program provides additional care outside the normal work hours at no additional cost to members who are already purchasing full-time care from the Child Development Center. However, in 12 of 14 focus groups we conducted, participants said that they found childcare services were of low quality or limited for 24/7 shift workers. For example, Cannon Air Force Base has two Child Development Centers, but they operate Monday through Friday from 6 a.m.to 6 p.m., and focus group participants noted a long waiting list for admission. At Creech Air Force Base, there is no childcare on base and at Shaw Air Force Base, participants said it was difficult finding available childcare to aid RPA personnel working shiftwork. For example, one RPA aircrew member was permanently assigned to the day shift because of childcare issues. Spouse and Family Support Issues RPA personnel have complained about the issues associated with working at remote location, such as the Creech Air Force Base, Nevada, and Cannon Air Force Base, New Mexico, locations. In 9 of 14 focus groups, participants made various comments regarding the limited spousal opportunities and family support issues such as the following: “I got orders to Cannon…. The problem is I’ll be bringing my wife there who has no job opportunities. There will be a lot of military spouses competing for jobs. I’ve already decided I’ll leave at the end of my contract and then will go to the Guard. I’ve told my wife I’ll get out because I don’t want to hurt her quality of life.” “I loved the mission at Cannon, but the facilities and area and schools are absolutely terrible.” “I’m fed up with Cannon and this area in general.” RPA bases vary in housing available for personnel with Cannon and Creech Air Force Bases reporting inadequate housing situations. At Cannon, officials stated that lack of dormitory space was forcing first-term Airmen off base. During our visit in June 2019, Cannon housing officials provided a report that stated that the shortfall in dormitory space continues to put Airmen and the Air Force Special Operations Command mission at risk. The report said that the locations off base where first-term Airmen can afford to live are usually in the worst crime-ridden parts where there is a far greater propensity for trouble. This can create morale issues and a distraction from the mission, according to the report. Additionally, Creech Air Force Base does not have any permanent on- base housing. At Creech, unaccompanied first-term Airmen must live in the dormitories on Nellis Air Force Base, which is approximately 50 miles away. The remoteness of Creech Air Force Base and the lack of basic services offered only at Nellis Air Force Base creates an unusual level of stress brought on by the added time, effort, and expense Creech Airmen experience that those at almost every other continental United States installation do not. In fact, a 2018 internal assessment for Creech leadership calculated that a junior airman who must live at Nellis Air Force Base would have a one-way commuting time of 63 minutes if they drive a personal vehicle or 105 minutes if they take the shuttle. To help address the housing and access to medical facilities, Creech Air Force Base senior officials said that a plan to create officer and non- commissioned officer housing and a medical facility on the northwest side of Las Vegas has been approved, but it is not expected to be completed until between fall 2021 and fall 2022. Many of the RPA workforce issues we identified at the time of our 2014 review continue to exist today. These workforce issues include the challenges to the RPA workforce’s quality of life due to stressful working conditions, including work shifts that frequently rotate, long hours, and increased workloads. In 2017, we recommended that the Air Force should monitor the extent to which its RPA human capital efforts are achieving the Air Force’s overall programmatic goals. The Air Force had not implemented this recommendation as of February 2020. Because long- standing RPA quality of life and workforce management issues affecting RPA personnel continue to exist, we believe that this recommendation is still valid and would aid the Air Force in its efforts to address many of the challenges facing this career field. Therefore, we are not making any additional quality of life related recommendations. Conclusions A healthy RPA workforce is one that balances supply with demand and addresses quality of life conditions to motivate and sustain performance and retention. Successful efforts to assess, train and retain RPA pilots and sensor operators would allow the Air Force to grow sufficient quantities of its RPA workforce to meet its goal of implementing its combat-to-dwell policy. While the total number of Air Force RPA pilots and sensor operators has increased between 2015 and 2019, the number of positions required to meet the constant demand is increasing at a faster pace. Additionally, the Air Force has not achieved its accession targets for pilots and sensor operators for most of those years. Moreover, the inability to use standard retention metrics due to the newness of the RPA pilot career field is hindering the Air Force’s ability to determine accurately if sufficient quantities of RPA personnel are remaining in the service to grow its RPA workforce. Further, the Air Force currently does not have a comprehensive metric (or set of metrics) to track the overall progress toward having sufficient numbers of RPA personnel through its accessions and retention of RPA personnel to meet its prescribed timeline for implementing its combat-to-dwell policy. This policy is intended to balance the time RPA units spend in combat with non-combat activities, to provide relief from those combat operations that it has conducted constantly for many years, to improve the quality of life of these RPA aircrew members. Without a metric, it is unclear whether the Air Force is on course to achieve implementation of its combat-to-dwell policy. As such, the Air Force cannot know if adjustments are needed specifically to that policy and its implementation timeline or to its overall personnel management efforts to access, train and retain sufficient numbers of RPA personnel. Further, the Air Force previously prioritized having maximum instructor staffing at the training units to help increase the production of new RPA aircrews. However, the number of instructor positions required at the RPA formal training unit at Holloman Air Force Base is out-of-date and does not reflect what is needed to teach the current training curriculum. Additionally, this formal training unit has consistently experienced staffing shortages since fiscal year 2016. As such, without updated information, the Air Force does not know the number of instructor positions necessary for sufficiently training RPA aircrews and it may not fully address the challenges affecting the training unit’s staffing and ability to produce the needed number of aircrews to support the continued demand for RPAs and the implementation of the combat-to-dwell policy as planned. The Air Force developed initiatives with its 2015 Culture and Process Improvement Program to address quality of life issues and other challenges affecting the RPA community, but has not fully implemented them. We also identified workforce management challenges in our previous work. We believe that our prior recommendation that the Air Force monitor its human capital efforts would help address these challenges. We believe the Air Force should implement our prior recommendation to aid the Air Force in its attempts to improve the quality of life issues that still exist within the RPA community. Recommendations for Executive Action We are making the following two recommendations to the Secretary of the Air Force. The Secretary of the Air Force should ensure that a comprehensive metric (or set of metrics) is established to track the progress of its combined accession and retention efforts to obtain sufficient quantities of RPA pilots and sensor operators needed to achieve its objective of implementing the combat-to-dwell policy as planned. (Recommendation 1) The Secretary of the Air Force should ensure that the number of instructor positions needed at the RPA training unit at Holloman Air Force Base is updated by applying more complete, accurate and timely information to better reflect the training curriculum and instructor needs. (Recommendation 2) Agency Comments and Our Evaluation We provided a draft of this report to DOD for review and comment. In written comments reproduced in appendix III, the Department of the Air Force partially concurred with our first recommendation and concurred with our second recommendation. In concurring with our second recommendation to ensure the number of instructor positions needed at the RPA training unit at Holloman Air Force Base is updated, the Air Force noted that it has requested an updated study to determine the appropriate number of instructor positions. With regard to our first recommendation—to establish a comprehensive metric (or set of metrics) to track the progress of its combined accession and retention efforts—the Air Force noted that it already has efforts to monitor accession, production, and retention for RPA pilots and sensor operators. Additionally, it expects that standard retention metrics used in other rated career fields will provide increased utility as the RPA career field matures. The Air Force acknowledges in its comments, however, that these efforts could be better integrated to allow for greater analysis, to include tracking progress in meeting the combat-to-dwell policy by 2024. We continue to believe that in developing a specific metric (or set of metrics) the Air Force would be in a better position to evaluate the status of its combined accession and retention efforts to obtain the proper number of RPA personnel to achieve its combat-to-dwell implementation goal. We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, and the Secretary of the Air Force. In addition, this report will be available at no charge on the GAO website at http://www.gao.gov . If you or your staff have any questions regarding this report, please contact me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV. Appendix I: Focus Group Methodology To obtain the perspectives of Air Force remotely piloted aircraft (RPA) pilots and sensor operators regarding training, availability of services and support to RPA personnel and their families; quality of life issues; retention issues; and other challenges facing the RPA career field, we analyzed participants’ comments from 14 focus groups at three different RPA operational locations. These locations were: Shaw Air Force Base, South Carolina; Cannon Air Force Base, New Mexico; and Creech Air Force Base, Nevada. We selected Cannon and Creech Air Force Bases because they have the largest population of RPA operators in Air Force Special Operations Command and Air Combat Command, respectively. In addition, we selected Shaw Air Force Base to obtain the perspectives of RPA pilots and sensor operators working at a base with newly established RPA operations since 2018. To obtain a balance of perspectives from RPA pilots and sensor operators with varying levels of experience and responsibilities, we conducted focus group sessions with active-duty MQ-9 Reaper RPA pilots and sensor operators who were divided by their occupation, Air Force Specialty Code, and rank at the selected locations. Specifically, we used the following categories as shown in table 3 for the formation of the focus groups. The 14 focus groups we held ranged in size from five to 11 participants across the three sites with 105 total participants. We conducted five focus groups at Shaw Air Force Base; four focus groups at Cannon Air Force Base; and five focus groups at Creech Air Force Base. Of the 14 focus groups, eight focus groups were with RPA pilots and six focus groups were with RPA sensor operators. These sessions involved structured small-group discussions designed to gather in-depth information that is not easily obtained from other methods. We requested that our point of contact at each location gather approximately 8 to 12 participants to attend the five pre-defined focus groups. We conducted focus groups with RPA pilots and sensor operators separately because they have different roles and responsibilities and to encourage active participation and minimize the risk of participants being the same group as immediate supervisors. We segmented our groups by this characteristic in order to compare and contrast their perspectives on training, retention, and quality of life issues and to identify meaningful similarities and differences. Participants in the focus groups were not randomly selected by using a probability sampling method, but recruited by unit leadership based on shift availability and correspondence with the characteristics we requested. Because scheduling availability was the primary factor affecting participation, coupled with the fact that questions for focus group sessions were not shared in advance, we considered the risk of leadership selectively picking participants to be minimal. Methodologically, focus groups are not designed to (1) demonstrate the extent of a problem or to generalize results to a larger population, (2) develop a consensus to arrive at an agreed-upon plan or make decisions about what actions to take, or (3) provide statistically representative samples or reliable quantitative estimates. Instead, they are intended to generate in-depth information about the reasons for the focus group participants’ attitudes on specific topics and to offer insights into their concerns about and support for an issue. A facilitator who used a standard script and list of questions to guide the discussion and encourage participants guided the focus group participants to share their thoughts and experiences. We confirmed at the start of each session that participants met the inclusion criteria for the respective group. Due to the low numbers of 18X pilot participants at the O3-O5 rank and 11U/12U pilot participants at Cannon Air Force Base, we conducted a focus group of the available participants together instead of separately. Additionally, at Creech Air Force Base, we encountered three situations where participants were currently full-time Reserve pilots, but because all had former active-duty experience and dismissing them would result in too few participants in the group, we allowed them to stay in the focus groups in order to have a sufficient number of participants. This situation occurred in the O1-O2 18X pilot focus group, the O3-O5 18X pilot focus group, and the E5-E9 1U0XX sensor operator focus group. The core questions that the GAO facilitator asked during each of the focus groups are listed in table 4. During the focus group meetings, three GAO members independently took separate sets of detailed notes to document the participants’ comments. Afterward, each member’s notes were compiled into one final official record documenting the comments made in each of the focus groups we conducted. Then, these records were consolidated into one database to be used for coding each comment and to facilitate the team’s content analysis of all the comments. To identify common categories and themes from the participants’ comments across all focus groups, the team met, reviewed and discussed the official record of each of the 14 focus groups. From that meeting, the team identified 43 categories across seven areas of inquiry; see table 5 for a list of the categories and themes. Using the categories and themes identified, the team conducted a pre- test by having two groups of two coders independently code an identical subset of the comments to determine their levels of coding consistency and accuracy before attempting to code all 1,848 individual recorded comments. After the pretest, the two groups split the list of comments in half and each coder independently coded the comments contained in their list into the categories and themes under which the coder believed the comment fell. Once completed, the coders within each group met to discuss any discrepancies in each of their coding and to make any necessary adjustments in the coding. Where discrepancies could not be resolved between coders, an independent third team member determined which code would be used. Once the coding of all 1,848 comments was finalized, the team’s methodologist prepared a report that presented all comments that fell within each of the categories and themes. The team used this information as the basis for frequency tabulation and qualitative analysis of focus group comments. In addition to discussing the RPA pilots’ and sensor operators’ perspectives in a focus group setting, we administered a questionnaire to each participant at the end of each session before the participants were dismissed. All participants completed the questionnaire. A GAO methodologist with a social science background and knowledge of small group methods and survey administrations reviewed the focus group script and the questionnaire. In addition, we pre-tested both the focus group protocol and the questionnaire on our first site visit to Shaw Air Force Base and both were used again at the remaining RPA locations, Cannon and Creech Air Force Bases, without any changes. Appendix II: Reports and Studies on Air Force Remotely Piloted Aircraft Personnel The Department of Defense (DOD), the military services, and organizations outside DOD have produced reports and studies that addressed issues associated with Air Force remotely piloted aircraft (RPA) personnel, including the following: Armour, Cherie, and Jana Ross. “The Health and Well-Being of Military Drone Operators and Intelligence Analysts: A Systematic Review.” Military Psychology, 2017. Bryan, Craig J., Tanya Goodman, Wayne Chappelle, Lillian Prince, and William Thompson. “Subtypes of severe psychological distress among US Air Force remote warriors: A latent class analysis.” Military Psychology, 2018. Campo, Joseph L. “Distance in War: The Experience of MQ-1 and MQ-9 Aircrew.” Air and Space Power Journal, 2015. Chappelle, Wayne L., Kent McDonald, Lillian Prince, Tanya Goodman, Bobbie N. Ray-Sannerud, and William Thompson. “Symptoms of Psychological Distress and Post-Traumatic Stress Disorder in United States Air Force “Drone” Operators.” Military Medicine, 2014. Chappelle, Wayne, Emily Skinner, Tanya Goodman, Julie Swearingen, and Lillian Prince. “Emotional reactions to killing in remotely piloted aircraft crewmembers during and following weapon strikes.” Military Behavioral Health, 2018. Chappelle, Wayne, Julie Swearingen, Tanya Goodman, Sara Cowper, Lillian Prince, and William Thompson. Occupational Health Screenings of U.S. Air Force Remotely Piloted Aircraft (Drone) Operators. Report, Wright-Patterson Air Force Base, OH: Air Force Research Laboratory, 2014. Chappelle, Wayne, Kent McDonald, and Raymond King. Psychological Attributes Critical to the Performance of MQ-1 Predator and MQ-9 Reaper U.S. Air Force Sensor Operators. Report, Brooks City-Base, TX: Air Force Research Laboratory, 2010. Chappelle, Wayne, Kent McDonald, Billy Thompson, and Julie Swearangen. Prevalence of High Emotional Distress and Symptoms of Post-Traumatic Stress Disorder in U.S. Air Force Active Duty Remotely Piloted Aircraft Operators (2010 USAFSAM Survey Results). Report, Wright-Patterson Air Force Base, OH: Air Force Research Laboratory, 2012. Chappelle, Wayne, Kent McDonald, Lillian Prince, Tanya Goodman, Bobbie N. Ray-Sannerud, and William Thompson. “Assessment of Occupational Burnout in United States Air Force Predator/Reaper “Drone” Operators.” Military Psychology, 2014. Chappelle, Wayne, Tanya Goodman, Laura Reardon, and Lillian Prince. “Combat and operational risk factors for post-traumatic stress disorder symptom criteria among United States Air Force remotely piloted aircraft “Drone” warfighters.” Journal of Anxiety Disorders, 2019. Chappelle, Wayne, Tanya Goodman, Laura Reardon, and William Thompson. “An analysis of post-traumatic stress symptoms in United States Air Force drone operators.” Journal of Anxiety Disorders, 2014. Cooke, Nancy J., Kristen Barrera, Howard Weiss, and Claude Ezzell. “Psychosocial Effects of Remote Operations.” In Remotely Piloted Aircraft Systems: A Human Systems Integration Perspective, by Nancy J. Cooke, Leah J. Rowe, Winston Bennett, Jr. and DeForest Q. Joralmon. West Sussex: John Wiley & Sons, 2017. Goodman, Tanya, Lillian Prince, Wayne Chappelle, and Craig Bryan. A Reassessment of Risk Factors and Frequency of Suicide Ideation Among U.S. Air Force Remote Warriors. Report, Wright-Patterson AFB, OH: Air Force Research Laboratory, 2018. Hardison, Chaitra M., Eyal Aharoni, Christopher Larson, Steven Trochlil, and Alexander C. Hou. Stress and Dissatisfaction in the Air Force’s Remotely Piloted Aircraft Community. Santa Monica, CA: RAND Corporation, 2017. Hijazi, Alaa, Christopher J. Ferguson, Harold Hall, Mark Hovee, F. Richard Ferraro, and Sherrie Wilcox. “Psychological Dimensions of Drone Warfare.” Current Psychology, 2017. Martin, Kiel M., Daniel J. Richmond, and John G. Swisher. “Sustaining the Drone Enterprise: How Manpower Analysis Engendered Policy Reform in the United States Air Force.” INFORMS Journal on Applied Analytics, 2017. Martin, Matt. “Remote-Split Operations and Virtual Presence: Why the Air Force Uses Officer Pilots to Fly RPAs.” 18th International Symposium on Aviation Psychology. Dayton, 2015. Ouma, Joseph A., Wayne L. Chappelle, and Amber Salinas. Facets of Occupational Burnout Among U.S. Air Force Active Duty and National Guard/Reserve MQ-1 Predator and MQ-9 Reaper Operators. Report, Wright-Patterson Air Force Base, OH: Air Force Research Laboratory, 2011. Terry, Tara L., Chaitra M. Hardison, David Schulker, Alexander C. Hou, and Leslie Adrienne Payne. Building a Healthy MQ-1/9 RPA Pilot Community: Designing a Career Field Planning Tool. Santa Monica, CA: RAND Corporation, 2018. Wood, III, Joe, et al. Prevalence of Posttraumatic Stress Disorder in Remotely Piloted Aircraft Operators in the United States Air Force. Report, Wright-Patterson Air Force Base, OH: Air Force Research Laboratory, 2016. Wood, III, Joe D, et al. “Relationship Between Spiritual Well-being and Post-traumatic Stress Disorder Symptoms in United States Air Force Remotely Piloted Aircraft and Intelligence Personnel.” Military Medicine, 2018. Appendix III: Comments from the Department of Defense Appendix IV: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments Brenda S. Farrell, (202) 512-3604 or farrellb@gao.gov In addition to the contact named above, key contributors to this report were Lori Atkinson, Assistant Director; Rebecca Beale, Brad Crofford, Caitlin Cusati, Felicia Lopez, Terry Richardson, Ophelia Robinson, Pamela Snedden, and John Van Schaik. Related GAO Products Unmanned Aerial Systems: Air Force Pilot Promotion Rates Have Increased but Oversight Process of Some Positions Could Be Enhanced. GAO-19-155. Washington D.C.: February 7, 2019. Unmanned Aerial Systems: Air Force and Army Should Improve Strategic Human Capital Planning for Pilot Workforces. GAO-17-53. Washington D.C.: January 31, 2017. Unmanned Aerial Systems: Actions Needed to Improve DOD Pilot Training. GAO-15-461. Washington, D.C.: May 14, 2015. Air Force: Actions Needed to Strengthen Management of Unmanned Aerial System Pilots. GAO-14-316. Washington, D.C.: April 10, 2014.