Background Overview of FEMA’s Disaster Workforce The federal disaster workforce is designed to scale up or down depending on the timing and magnitude of disasters. Specifically, FEMA has the authority to augment its permanent full-time staff with temporary personnel and deploy non-FEMA staff members when needed. FEMA has historically relied on both permanent and temporary staff members to respond to presidentially declared disasters. FEMA’s disaster workforce is organized according to position categories, employee types, functions, and job titles. Every FEMA employee is assigned to one or more of four position categories. Staff assigned to incident management positions deploy to disaster sites to administer federal emergency response and recovery programs. Staff assigned to the other three position categories—incident support, ancillary support, and mission essential—provide support services to deployed incident management staff, as well as to FEMA more generally. For example, incident support staff assist with disaster operations at the regional or national level, while mission essential staff maintain basic agency operations, such as payroll and information technology. FEMA has several different employee types that operate out of the agency’s national headquarters, regional offices, and joint field offices at specific disaster locations. Each of the different employee types hold one or more of the four position categories described above. Permanent full-time employees are steady-state federal employees that support FEMA’s mission areas and operations on a daily basis. Cadre of On-Call Response/Recovery Employees (CORE) are a type of temporary full-time employee hired to directly support response and recovery efforts related to disasters for a 2- to 4-year term. These positions may be renewed if there is ongoing disaster work and funding is available. Incident Management COREs are a type of CORE employee that maintain a regular state of readiness to provide emergency-state support and can be deployed up to 300 days a year in mission areas. Incident Management Assistance Teams are rapid-response teams of CORE employees that deploy to disaster sites with little to no notice and remain at disaster sites for unspecified amounts of time, depending on mission needs. Members generally receive 4-year appointments. Reservists are on-call employees who work intermittently as required during incident management operations. Reservists must be available to deploy as needed on 24 hours’ notice at all times during their 24 month appointment. FEMA also has the authority to augment its disaster workforce with temporary employees. This includes local hires, Surge Capacity Force volunteers, and FEMA Corps members. FEMA further augments its workforce with technical assistance contractors, who are specialized contractors hired to perform specific responsibilities. See figure 2 for more information on FEMA’s employee types. As shown in figure 3, reservists made up the largest portion of FEMA’s deployed workforce during peak deployments for the 2017 and 2018 disaster seasons. In 2017, reservists made up about 32 percent of FEMA’s deployed workforce, followed by COREs, permanent full-time staff, and local hires. In 2018, reservists made up about 44 percent of FEMA’s deployed workforce, followed by local hires, COREs, and permanent full-time staff. Organizational Structures for Incident Management Staff FEMA’s incident management workforce is organized into 23 cadres. Cadres are groups organized by operational or programmatic functions. They are composed of full-time equivalent and intermittent staff members who perform incident-related duties during disaster response. For example, the Public Assistance cadre administers financial assistance to state, local, tribal, and territorial communities for debris removal, implementation of emergency protective measures, and permanent restoration of infrastructure. FEMA’s incident management workforce performs functions to support its response, recovery, and mitigation missions. Each cadre supports at least one of these missions, and some cadres perform functions across more than one. Cadres also generally deploy to an incident at varying points in the response and recovery phases, depending on their functions. For example, FEMA officials said that the Logistics cadre, which coordinates and monitors all aspects of resource planning, movement, and order tracking, typically deploys staff to an incident before the Hazard Mitigation cadre, which supports risk reduction activities later during the recovery phase. See figure 4 for an example of peak deployment by cadre during Hurricane Florence and appendix II for a description of each cadre and their primary duties. FEMA manages the staffing, training, and deployment of its cadres at the national level. FEMA employees whose primary responsibilities are incident management and disaster response, such as Incident Management CORE and reservists, are generally considered national assets and may be deployed to a disaster anywhere in the country, regardless of permanent duty station. FEMA organizes incident management positions into four tiers denoted by increasing levels of leadership managerial responsibilities and further categorizes senior leaders and officers by level of disaster complexity experience. See figure 5 for more information on FEMA’s position tiers and titles. All FEMA incident management employees have a primary title, which specifies their principal roles and responsibilities, and may also hold subordinate titles for additional roles and responsibilities that the employee can perform. Incident management staff members have one primary incident management title and may have multiple subordinate titles. FEMA may also assign specialties—categories used to identify a specific measured (documented or credentialed) skill, task, experience, or certification that may enhance performance of an associated position—to certain staff. For example, a full-time equivalent staff member who works day-to-day in FEMA’s Office of Policy and Program Analysis could hold a primary incident management title as a Facilities Manager in FEMA’s Logistics cadre and a subordinate title of Logistics Specialist in the same cadre, and may be certified to operate certain types of forklifts. FEMA Has Mechanisms in Place to Qualify and Deploy Staff to Disasters and Faced Staffing Shortages during the 2017 and 2018 Disaster Seasons FEMA Designed and Implemented a System to Ensure Standards for Disaster Workforce Qualifications and Capabilities FEMA designed and implemented the FEMA Qualification System in 2012 to standardize and manage the agency’s incident workforce capabilities through prerequisite experience, training, and demonstrated performance. FEMA uses the system to track requirements for incident management positions and the proficiency level of staff members in those positions. According to the 2019 FEMA Qualification System Guide, training and demonstrated performance are foundational elements of the system. Required qualification system training consists of courses designed to build competency in specific position responsibilities and is offered in a variety of settings, such as the Department of Homeland Security (DHS) Center for Domestic Preparedness or at a joint field office, and through a variety of mediums, such as in a classroom, online, or on the job. Demonstrated performance involves validation of the ability to successfully and independently perform specific tasks. According to FEMA, employees conduct required training concurrently with demonstrated performance so that training builds on previous experience and coursework. After FEMA assigns an incident management position to staff, they are issued an electronic position task book, which lists and tracks the training and demonstrated performance requirements for that position. Tasks in the position task book need to be signed off by a coach-and- evaluator—an individual that is trained and designated as a subject matter expert by their cadre to evaluate one or more FEMA Qualification System positions. After staff members have worked with a coach-and- evaluator to complete the tasks and trainings included in their task book, they may submit it to cadre management as part of their qualification application package. Submitted qualification packages go through a number of rounds of review before a final decision is made. First, FEMA’s Qualification System Branch conducts an initial review to validate qualification package completion and requirement waivers, among other things. The branch then forwards the qualification package to cadre management for review. Cadre management reviews employees’ applications for all positions, including specialists and first-line supervisors, and a Qualification Review Board also reviews employees’ applications for leadership, upper management, and middle management positions. See figure 6 for an overview of FEMA’s qualification system process. FEMA Has a Process to Deploy Its Workforce to Disasters A regional or national Incident Management Assistance Team is generally among the first FEMA units to arrive on the ground at a disaster site to, among other things, set up federal facilities, establish a joint field office, and coordinate with officials from the impacted region and other relevant federal, state, tribal, territorial, or local partners. If there are staffing shortages among regional full-time equivalent staff members, FEMA leadership in the region where the disaster occurs may submit a deployment request for additional incident management staff members through the Deployment Tracking System. After the Incident Management Assistance Team stands up a joint field office, the Federal Coordinating Officer assumes authority over all emergency response and recovery efforts for the disaster, which includes filling staffing needs. To determine the number and type of incident management personnel needed in each position to meet FEMA requirements for any given disaster, the Federal Coordinating Officer coordinates with regional leadership, the joint field office’s Chief of Staff, and cadre management. The basis of this determination is a variety of factors related to the nature and scope of the disaster. For example, Individual Assistance and Public Assistance needs are based in part on preliminary damage assessments to determine the level of program assistance that may be required. To fill identified staffing needs, field leadership uses a standard process to request specific FEMA Qualification System titles and proficiency levels. Once a standard deployment request is approved, the Deployment Tracking System identifies staff members that match the requested positions, skill sets, and qualification status using a preprogrammed, automated process. The Deployment Tracking System then notifies staff members selected in a rotational order based on length of time since their last deployment, among other things. If an employee declines a deployment request, the Deployment Tracking System automatically sends a request to the next staff member with that incident management position title on the deployment order list. Standard deployment requests are filled by deploying employee types in the following order: 1. Incident Management COREs 2. Reservists 3. Full-time equivalent employees other than Incident Management At the incident, the Federal Coordinating Officer and other field leadership staff are responsible for overseeing coordinating the responders working for FEMA. Generally, after response operations and programs are initiated, staffing needs may change. At this point, field leadership may decide to demobilize some personnel deployed in certain cadres. The decision to do so is based on workload, complexity of operations, and disaster needs. FEMA Faced Staffing Shortages in Key Cadres during the 2017 and 2018 Disaster Seasons According to FEMA’s 2017 Hurricane Season After-Action Report, FEMA did not meet its annual staffing target for disaster personnel during the 2017 hurricane season. FEMA uses force structure and force strength targets to estimate staffing requirements for incidents and analyze the number of staff the agency has available against these targets. FEMA establishes a longer-term target for the number of incident management staff needed to meet mission needs, called force structure, and tracks the actual number of incident management staff who can deploy at a point in time, which it calls force strength. FEMA uses its force strength measure to track progress towards meeting staffing goals set out in the force structure target and also sets interim targets each fiscal year for reaching the longer-term force structure target. In 2015, FEMA’s Workforce Management Division conducted a review of FEMA’s workforce in coordination with the 23 cadres and adopted a force structure target of 16,305. According to FEMA, this target was established based on a number of considerations, including potential incident levels and historical staffing data for incident management staff deployed to different level events. The agency’s force strength at the end of fiscal year 2017 was 11,656. On average, reservists had the largest gap between force strength and established annual targets. For example, at the end of fiscal year 2017, FEMA’s force strength for reservists was 6,793, which was 15 percent short of its target of 7,982 for that year. In 2019, FEMA’s Workforce Management Division completed a similar review of its incident management workforce and updated its force structure target to 17,670 incident management personnel, which it aims to achieve by 2025. This new target was established using an updated methodology based on information on historical disasters and deployed incident management staff, along with input from each cadre’s management and other officials with expertise on staffing patterns throughout disasters. According to FEMA’s 2017 Hurricane Season After-Action Report, FEMA faced shortages across over half of its cadres when disasters made landfall or began during the 2017 season, and we found that FEMA faced similar shortages during the 2018 disaster season. For instance, according to FEMA’s deployment data, 18 of 23 cadres operated with 25 percent or fewer staff available to deploy when Hurricane Maria made landfall shortly after Hurricane Irma hit Florida, including the Individual Assistance, Logistics, and Hazard Mitigation cadres. See figure 7 for more information on cadre staffing levels at the start of major disasters during the 2017 and 2018 disaster seasons. In addition, many staff members who showed availability to deploy declined when requested to do so. For example, according to FEMA officials, the austere conditions in Puerto Rico and fatigue from previous deployments to hurricanes Harvey and Irma contributed to the high declination rate for Hurricane Maria deployment requests. In addition, FEMA officials stated that permanent full-time employees may not consistently update their availability or may be unavailable to deploy because of their steady-state responsibilities. Further, reservists may decline deployment requests because of the lack of employment protections, which can create difficulties with leaving their jobs to work intermittently in disasters. See table 1 for the declination rates for eight major disasters during the 2017 and 2018 disaster seasons. According to FEMA officials, the agency’s staffing shortages may have originated in part from policy changes in 2012. Specifically, officials said that a large number of incident management staff left the agency after changes were made to the agency’s reservist program and qualification system for disaster personnel in 2012. For instance, officials told us many reservists with years of experience and technical skills left FEMA when the reservist program increased the number of days they were required to deploy or when many were assigned to positions in the qualification system with lower levels of responsibility than what they previously held in order to meet force structure targets. FEMA took various actions to address the staffing shortages during the 2017 and 2018 disaster seasons and used new approaches to augment its workforce. For example, in 2017, FEMA reported that it coordinated the deployment of 2,740 Surge Capacity Force volunteers from eight DHS components. DHS also expanded the Surge Capacity Force to other federal agencies outside DHS for the first time in 2017, including 34 federal departments and agencies in the program, increasing the Surge Capacity Force by 1,323 employees. Additionally, FEMA used local hires to augment its workforce and expedited the local hiring process in response to hurricanes Harvey, Irma, and Maria, resulting in the onboarding of 4,095 local hires from August to November 2017. The Federal Coordinating Officer who initially managed the Puerto Rico joint field office instituted a goal of having a staff composed of 90 percent local hires for recovery efforts. According to the official, investing heavily in local hires and converting them to COREs will help build FEMA’s disaster workforce for long-term Puerto Rico recovery efforts and any future disasters that may occur in the region. As mentioned previously, FEMA also conducted a review of its incident management workforce in 2018 to determine the force structure needed to effectively respond to disasters moving forward. FEMA officials we spoke with said the agency has taken several steps to meet its new force structure, such as establishing a program management office that is dedicated to achieving the agency’s staffing targets. Cadre management officials we spoke with said that FEMA has hiring initiatives in place or planned to help meet the staffing needs established from the review and noted that it will take time for new staff to develop the skills and experience to meet mission needs in the field. FEMA Did Not Provide Reliable and Complete Staffing Information to Field Officials during Disasters and Lacks Mechanisms to Assess How Effectively It Deployed Staff FEMA’s qualification and deployment processes did not provide reliable and complete information on staff skills and abilities to ensure its workforce was effectively deployed and used to meet field needs during the 2017 and 2018 disaster seasons. In addition, FEMA lacks mechanisms to assess deployment outcomes or the extent to which it deployed the right mix of staff at the right time to meet mission needs. FEMA’s Qualification and Deployment Processes Did Not Provide Reliable Staffing Information to Ensure Its Workforce Was Effectively Deployed and Used in the Field FEMA field officials in our focus groups and interviews said they experienced a number of challenges with the reliability of information from FEMA’s qualification and deployment processes and systems during the 2017 and 2018 disaster seasons. Specifically, these officials reported that qualification status was not an accurate indicator of ability to perform, which affected disaster assistance delivery and created difficulties with ensuring the right mix of staff were deployed and effectively assigning responsibilities at disaster sites. Officials also reported other challenges with identifying and leveraging staff skills, including lack of information about specialized abilities and expertise. In response to its experience with recent disaster seasons, FEMA has taken or planned some actions to improve its qualification and deployment processes. However, these actions have not been fully implemented and do not fully address the information shortcomings that field officials experienced, as discussed below. Field Officials Reported Qualification Status Was Not a Reliable Indicator of Staff’s Ability to Perform Their Positions in the Field FEMA’s qualification and deployment processes and systems do not provide accurate and complete information about staff members’ abilities to ensure field leadership and managers get staff with the right skills at the right time or to most effectively employ and leverage the staff that are deployed to support FEMA’s missions. As discussed earlier in this report, field leadership use the Deployment Tracking System to request staff based on mission needs. The system uses an automated process to select who to deploy from a list of available staff by position and qualification status, and relies on the FEMA Qualification System to identify staff members who are qualified in their positions and those who are trainees. Qualified staff members are expected to be able to successfully and independently perform the duties of their position. However, as shown in table 2, our focus groups with incident management staff and interviews with field and regional officials indicate that disaster personnel experienced significant limitations with qualification status in the FEMA Qualification System matching performance in the field. Very few found that it was a good indicator of staff abilities. For example, participants in two of 14 focus groups described positive experiences with qualification status as an indicator of staff abilities; while, in all 14 groups, participants stated that staff members who were designated as qualified in the system did not have the skills or experience to perform effectively in their positions. Officials in 15 of our 29 field and regional office interviews had similar experiences. For example, Individual Assistance managers in one of the joint field offices we visited said that they had 20 staff members who were designated as qualified but not capable of performing basic tasks, such as knowing how to access the program’s information system. Hazard Mitigation managers from the same joint field office said that about half of their staff who were designated as qualified could not proficiently perform their job duties. Participants in our focus groups and field leadership and managers we interviewed cited numerous operational challenges that resulted from qualification status not being an accurate indicator of staff abilities. Specifically, they stated that (1) staff designated as qualified who lacked the skills and knowledge to perform their positions negatively affected disaster assistance delivery, staff workload, and morale and (2) the unreliability of qualification designations hindered their cadre’s ability to staff disasters with the right mix of staff at the right time and effectively assign responsibilities. Table 3 provides examples of the challenges they experienced. Participants in our focus groups also cited a range of challenges with position task books and the qualification process that in their view contributed to qualification status not being an accurate indicator of staff proficiency. For example: Position task book tasks. In 12 of our 14 focus groups with FEMA incident management staff, participants said the tasks in the position task books did not fully reflect the skills or competencies needed to perform positions. For example, a participant in one focus group said that the tasks in her book did not incorporate sufficient soft skills, such as the ability to communicate with sensitivity and empathy and other interpersonal skills, which are important because staff in her cadre often interact with disaster survivors who have suffered great losses. Coach-and-evaluator process. Participants in 12 of our 14 focus groups also raised concerns with how coach-and-evaluators endorsed tasks, such as lack of consistency and objectivity with signing off on tasks. These issues included coach-and-evaluators signing off on large numbers of tasks too quickly or easily, which participants in 12 focus groups said occurred. Some participants who functioned as coach-and-evaluators said they felt pressure from staff to endorse tasks because reservists receive salary increases when they get qualified. Participants also told us that cadre management may push for staff to be qualified to meet qualification rate targets. A participant in one of our supervisory-level focus groups said he felt pressure from both these sources and admitted to signing off on tasks even though he did not feel the staff member could proficiently perform them. He said that the staff member was qualified in the FEMA Qualification System and later deployed to a smaller disaster, where she was the sole person responsible for her functional area and unable to do the job. Difficulties completing position task books. Participants in all 14 of our focus groups also raised various challenges with completing their task books. These challenges include a lack of available coach-and-evaluators to sign-off on tasks; lack of opportunities to deploy or perform certain tasks; and being unable to complete all the training courses in their task books because classes were unavailable, full, or conflicted with mission needs; among others. As a result, staff members who are able to perform their positions may not be designated as qualified in FEMA’s qualification system. Field Officials Cited Challenges with Using FEMA’s Qualification and Deployment Processes to Fully Identify and Use Staff Skills and Experience Participants in our focus groups and leadership and managers in our field and regional office interviews reported other challenges with being able to fully identify and use staff skills and experience during disasters. For example: Position titles not fully reflecting staff abilities. FEMA allows staff to have one primary position title in which they are qualified or have an open task book (trainee or candidate status). Officials in 14 of our 29 field and regional interviews and participants in eight of our 14 focus groups with incident management staff raised concerns with this policy. Specifically, officials noted that many employees have experience and expertise in multiple cadres or programs within a cadre, but their full abilities are not reflected in FEMA’s qualification and deployment systems. As a result, field leadership and managers may not be able to fully identify and use the available skills and experience of FEMA’s workforce during disasters, which can limit FEMA’s capacity and flexibility to meet disaster needs, especially when demand is high. For example, one regional official said the Deployment Tracking System has Operations Section Chief as her position title but does not capture her ability to deploy as an Individual Assistance Branch Director, another position in which she has considerable experience. Overly broad position titles and lack of information on specialized skills. In addition, participants in our focus groups told us that some cadre position titles are overly broad (five of 14 groups) and that FEMA’s qualification and deployment systems do not include information on specialized skillsets and experience that would be useful for making deployment and staffing decisions (10 of 14 groups). Officials in 14 of our 29 field and regional interviews raised one or more of these same issues. For example, Logistics managers in one of the joint field offices we visited said that the Logistics Specialist title is too general and captures the majority of warehouse personnel without specifying the actual responsibilities they are able to perform. They noted that, as a result, management needs to query staff members when they arrive to help determine their skills and, in many cases, assign responsibilities by trial and error. According to officials, this can create a safety hazard because some responsibilities require specific skills, such as the ability to operate a certain type of forklift. They also noted that while the Deployment Tracking System allows cadres to input specific skillsets, such as forklift certification, this field has not been consistently filled in. Limitations with fully capturing permanent full-time employee and CORE qualifications. In seven of our eight focus groups with permanent full-time employees and COREs, participants stated that it is not a priority for them to complete their task books because they have little or no incentives to be designated as qualified in the FEMA Qualification System. For example, some participants noted that unlike reservists, their pay and professional development is not directly tied to their qualification status or position. Another participant said that he has been deployed to many disasters and does not have any tasks in his task book endorsed because he is focused on meeting mission needs and does not care enough about being qualified in the system to take the time to complete his task book. Some regional and field officials also raised the same issues. For example, Response Division managers in one of the regions we selected for interviews said that some of the best talent at FEMA, such as staff members with years of experience who work full-time in positions similar to their incident management titles, have never opened or completed a task book because there is no incentive for them to do so. As a result, FEMA may not be fully capturing the qualifications and skills of permanent full-time employees and COREs. FEMA Has Taken Actions to Help Improve Its Qualification and Deployment Processes, but These Actions Do Not Fully Address the Key Challenges Field Officials Identified FEMA has taken a number of actions intended to help address some of the challenges with its qualification and deployment processes that hindered its ability to provide accurate and complete staffing information to field officials. FEMA headquarters officials acknowledged the challenges we identified with the FEMA Qualification System and noted that the system is still evolving. Key efforts to improve the reliability of qualification designations include: Qualifying staff members who could proficiently perform their positions. During the 2017 hurricane season, FEMA took steps to qualify staff members who were not designated as qualified in the FEMA Qualification System but could proficiently perform the duties of their position. For example, according to the agency’s after-action report for the hurricane season, FEMA temporarily changed qualification procedures during the season to more rapidly qualify employees who had demonstrated their skills outside the traditional process. FEMA headquarters officials stated that this helped qualification designations better reflect the skills and abilities of these staff members. Other actions that FEMA has taken to help qualify staff include allowing cadre management to waive certain tasks or training, allowing specified tasks to be signed-off on during training exercises, and, as discussed later in this report, conducting a pilot on deploying staff to specifically serve as coach- and-evaluators during disasters. Revising position task books. FEMA headquarters officials said they began reviewing task books in spring 2017 to help ensure that tasks are measurable and better align with the knowledge, skills, and abilities needed to perform positions. Officials said this project was completed in November 2018 and the revised task books have been implemented. They noted that this will help streamline the qualification process, allow for more objective evaluation, and help ensure tasks better reflect the skills needed on the job. According to FEMA officials, they plan to continue to work with the cadres to ensure task books align with the skills and competencies required to complete disaster missions. Enhanced coach-and-evaluator training. FEMA revised its training for coach-and-evaluators to provide more guidance and tools for how to accurately evaluate staff and improve the integrity of the evaluation process. Specifically, in October 2017, FEMA updated the coach-and- evaluator training class and added material on, for example, techniques for evaluating with integrity, types of observation, and documenting task performance by including comments in the task books. All staff members must pass the class by performing a capstone exercise and taking a written exam before being able to serve as a coach-and-evaluator. Additional controls in the qualification process. Since 2017, FEMA has established additional controls to provide more oversight on the qualification process. For example, headquarters officials said that as part of the qualification review process, officials may review the qualification packages to check how many tasks were endorsed during a given period of time. If the number is unusually large, they may note it for cadre management to consider when making qualification decisions. This step was incorporated in the new FEMA Qualification System Guide that was issued in August 2019. The guide also includes changes to the Qualification Review Board process, such as requiring candidates for leadership and upper-level management positions to attend the review in person and answer questions about their deployments, training history, and task book completion. FEMA has also taken some initial actions and considered options to better identify and use staff skills and experience in the field. For example, FEMA headquarters officials said they are aware that limiting staff to one primary position or one open task book may not fully capture their experience and abilities and are exploring ways to address it. However, they stated that this is a complex issue and allowing staff to hold multiple primary positions could affect the time it takes for staff to complete task books and, on a broader level, pay scales, career progression paths, and training budgets. They also noted that this could create complications with how to deploy staff if multiple cadres need positions filled during times of scarce resources. FEMA headquarters officials told us that staff can be deployed in positions other than their FEMA Qualification System positions but acknowledged that because these other positions are not systematically recorded in the Deployment Tracking System, leadership would need to be aware of staff skills and abilities through informal means. Further, FEMA headquarters officials said that as part of its review of the incident management workforce, they have revised the position titles for certain cadres, which they noted could potentially result in the titles better reflecting staff roles and responsibilities. Officials added that they need to balance the enhanced staffing information that more specific titles provide with the flexibility of broader titles, particularly for entry-level positions. FEMA has also included information on assigning specialized skills to staff in the Deployment Tracking System in its new FEMA Qualification System and deployment guides. While FEMA has taken some steps to improve its qualification and deployment systems, its efforts primarily affect the qualification process moving forward and do not fully address field officials’ experiences with not having reliable information on staff qualifications and skills to effectively use the available workforce to meet mission needs. For example, the changes to the position task books, coach-and-evaluator program, and FEMA Qualification System guide do not affect staff members who have already been qualified in the system but cannot perform their duties, and as discussed later in this report, FEMA currently does not have an effective performance evaluation system in place to identify and address skill deficiencies. In addition, the agency has not taken actions to address the challenges with identifying staff who can serve multiple incident management positions and fully capturing the qualifications of permanent full-time employees and COREs. Also, headquarters officials stated that FEMA has revised some of its position titles, but not all the cadres that reported challenges with overly broad titles had adjustments made to their positions. Further, this initiative is in the early stages of implementation and it is too soon to assess whether the revised positions will provide better information to field officials on staff members’ specific responsibilities. Further, the lack of reliability of qualification status as an accurate indicator of staff skills and abilities has been a persistent issue with the FEMA Qualification System. For example, we stated in our 2015 report on FEMA workforce management that in five of 11 focus groups with permanent full-time employees and COREs, participants cited concerns about the implementation of the FEMA Qualification System, and some observed employees whose training and experience did not reflect the position and qualification level to which they were assigned. Also, in a 2016 report on the reservist workforce, the DHS Office of the Inspector General stated that in five of the seven disaster deployments included in their review, joint field office staff encountered problems obtaining capable reservists with position titles under the FEMA Qualification System. These officials said that reservists sometimes lacked the experience and training to perform their duties, and as a result, were reassigned to positions outside their system titles. One of the purposes of the FEMA Qualification System is to ensure consistency in skill identification and deployable assets for positions across the agency. In addition, FEMA’s 2018-2022 Strategic Plan states that the agency should guarantee that the FEMA Qualification System maximizes the ability of FEMA staff to use their skills and talents to deliver the best outcomes possible. However, as discussed above, FEMA experienced challenges with achieving these objectives. In addition, Standards for Internal Control in the Federal Government directs management to use quality information to achieve the agency’s objectives. It states that, as part of designing control activities for human capital management, management should continually assess the knowledge, skills, and ability needs of the agency to help achieve organizational goals. According to the standards, only when the right personnel for the job are on board and are provided the right responsibilities, among other things, is operational success possible. In addition, according to The Standard for Program Management, program monitoring, reporting, and controls include the development of plans to respond to identified issues. It also states that program management should include timeframes and milestones for achieving program benefits and obtaining feedback from stakeholders to better understand the concerns related to the program and impact of the program. Given the complexity of FEMA’s workforce and the persistent issues with the reliability of qualification designations and other challenges with identifying the knowledge, skills, and abilities of its staff, FEMA would benefit from developing a comprehensive plan—with timeframes and milestones—to address issues with the quality of information its qualification and deployment processes and systems provide to field officials. Such a plan would also benefit from the inclusion of perspectives from field leadership who depend on the information. FEMA officials acknowledged the staffing information challenges we identified and noted that they have not developed a plan to address them because the issues are multifaceted—changes in policy can potentially affect numerous areas of the workforce—and they had been focused on other initiatives, such as revising force structure targets and streamlining the qualification process. However, they said that such a plan would be useful. Developing a plan to address the challenges that hindered FEMA’s ability to provide reliable and complete information about staff skills to field leaders and managers would better enable the agency to use its disaster workforce as flexibly and effectively as possible to meet mission needs in the field. FEMA Does Not Have Mechanisms to Assess How Effectively Its Disaster Workforce Was Deployed to Meet Field Needs FEMA does not have mechanisms to assess the extent to which its deployment strategies met mission needs in the field during disasters. FEMA’s Deployment Guide states that for the agency to fulfill its preparedness, response, recovery, and mitigation missions, it must be able to effectively and efficiently deploy its responders through a process that sends the right people to the right place at the right time with the right qualifications. FEMA has measures and collects data related to staffing levels and availability, such as comparing cadre force strength to annual targets, comparing staff qualification rates to targets, determining the percent of staff in each cadre that show availability in the Deployment Tracking System, and tracking the number of staff deployed to disasters. However, none of these measures or data directly demonstrate deployment outcomes or how effectively FEMA deployed available staff to meet mission needs. Headquarters officials said that, among other things, they generally have looked at the number of staff members that were deployed to disasters, as well as declinations, to assess the extent to which they were able to meet staffing needs. They noted that this assumed the number, type, and timing of staff deployments matched field needs. However, our focus groups and interviews with field officials indicate that this was not generally the case. For example, in all 17 of our focus groups, participants experienced challenges with the staffing, skill, or experience levels of the deployed workforce, such as having too few staff members with the right technical skills to perform their missions efficiently and effectively. Further, in 12 of the 17 focus groups we conducted, participants said that there were challenges with the timing of deployments, such as staff from certain cadres being deployed too early or redeploying staff from key positions when the mission need was still high. In most of our interviews with field leadership and managers, officials described similar challenges with the number, skill level, or timing of staff deployments. Participants in our focus groups and field officials we interviewed said they make every effort to meet mission needs despite challenges with staff deployment, but noted that these challenges with deployment outcomes not meeting field needs can increase staff workload and delay disaster assistance, among other impacts and inefficiencies. Our work on strategic human capital management states that effective geographic and organizational deployment strategies can enable an organization to have the right people, with the right skills, doing the right jobs, in the right place, at the right time by making flexible use of its internal workforce. Additionally, Standards for Internal Controls in the Federal Government states that management should establish and operate monitoring activities to continually monitor the internal control system, evaluate results, and remediate any deficiencies identified on a timely basis. As part of remediating deficiencies, the standards advise management to report and evaluate issues that were identified as a result of the monitoring and take corrective actions to address them. As discussed earlier in this report, field leadership request staff based on cadres’ anticipated needs using estimates of the severity of damage and the nature and scope of the disaster, among other factors. However, FEMA headquarters officials told us their data systems cannot determine the extent to which field deployment requests were met during disasters. In addition, these officials noted that they have not established other mechanisms to assess deployment outcomes because this is extremely complex and they are considering how best to do so. They noted that they have been working with in-house data science experts to consider what kinds of measures and metrics they could design to assess deployment outcomes, but they did not have any concrete proposals or time frames for when this might be completed. Without mechanisms to assess deployment outcomes, FEMA officials in headquarters lack critical information to monitor and evaluate the extent to which its deployment policies and strategies effectively placed staff with the right skills in the right place at the right time to meet mission needs in the field. As a result, FEMA may miss opportunities to identify when corrective actions are required to better deploy its workforce to meet field needs, such as adjusting the timing and staging of deployments, and the amount of staff deployed. FEMA Staff and Managers Experienced Challenges with Staff Development Efforts Intended to Enhance the Skills and Competencies Needed During Deployments We found significant shortcomings in FEMA’s ability to ensure staff development—which consists of courses, on-the-job-learning, and coaching and mentoring—for the skills and abilities needed in the field. Specifically, although the current approach to developing staff includes efforts to provide training courses, opportunities for on-the-job training and mentoring, and a performance evaluation system, each of these elements has limitations as implemented, and they are not effectively coordinated to help ensure systematic and comprehensive staff development. Staff and managers cited certain recurrent challenges with staff development in focus groups and interviews, such as (1) limitations on the ability to take useful classroom training, (2) challenges providing or receiving on-the-job training and mentoring, (3) inconsistent use of performance evaluations, and (4) difficulty with ongoing development when not deployed to a disaster. Some Staff and Managers Cited Challenges with the Ability to Take Useful Classroom Training One way staff members develop skills and competencies is through completing required courses in their position task books. However, in 10 of our 17 focus groups, participants discussed barriers to taking courses through FEMA’s qualification system that in their view would help them better perform their jobs, such as being unable to take courses that are not in their position task books or if they are already qualified in their positions. Officials in 11 of the 29 field and regional interviews we conducted raised the same issue. FEMA headquarters officials stated that staff are generally required to obtain cadre management approval before they can register for incident management-related courses that are not specifically listed in their position task books, but staff told us it can be difficult to receive approval because of funding limitations. For example, a Hazard Mitigation official at one joint field office we visited described a situation where a staff member wanted to take a course on mitigation and engineering techniques for coastal construction that would have benefitted the work the person was doing, but was not able to get approval. Participants in our focus groups also told us that staff deployed to a position other than their FEMA Qualification System title had been unable to take courses related to the work they were doing. Moreover, staff members said the FEMA Qualification System limits training opportunities for those already qualified in their positions. For example, some staff members said that once they had completed their position task book, they were sometimes unable to get training that included new information on updated policies or procedures specific to their work. An official in one of the FEMA regions we selected for interviews said that some staff members in the region who were qualified would have preferred to be designated as trainees in the FEMA Qualification System because it would allow them to take relevant courses. In March 2020, FEMA officials told us the agency has recently taken actions to make it easier for cadres to send staff to courses that are not required in their position task book or for positions where the person is qualified. Finally, participants in our focus groups with permanent full-time staff members reported challenges with being able to take courses to develop their incident management competencies. These participants told us it is challenging for them to take disaster-related courses while performing their steady-state work. They said this is because there is no budget for localized disaster-related courses in their offices and it can be difficult to get approval and take time from their duties to travel for this type of training. Some Staff and Managers Cited Challenges with Providing and Receiving On-the-Job Training and Mentoring during Disasters Focus group participants frequently said developing skills on the job was the most useful type of training they receive. Specifically, participants in 12 of our 17 focus groups said on-the-job training was the most useful kind of training and participants in 13 of the 17 focus groups said this is how they received most of their training. In addition, headquarters officials in the Individual Assistance cadre said one of the benefits of on-the-job training during deployments is that it provides an opportunity for staff to learn and practice their craft in a setting that is difficult to simulate during training. The FEMA Qualification System Guide states that FEMA uses coach- and-evaluators as the primary mechanism for staff to learn the specific skills needed for each position. However, staff members we spoke with said they have difficulties developing their skills through the qualification process. Specifically, in seven of the 17 focus groups, participants told us they did not get feedback or coaching on the job. According to staff in our focus groups, the coach-and-evaluator aspect of the qualification system is not the ideal mechanism to support on-the-job training and development because it often emphasizes the evaluation role over the coaching role. In nine of 14 focus groups, participants told us the position task book process focuses more on completing tasks than on performance, development, or building competencies. Officials in eight of our 29 field and regional interviews reported similar experiences. Some staff who did receive coaching said it was often based on the interest level and time that an individual who was willing to invest and was not done in a systematic or consistent way. Moreover, a commonly cited challenge—in 11 of our 14 focus groups— was the lack of coach-and-evaluators to sign off on position task books. Officials in 16 of our 29 field and regional interviews raised the same issue. Participants in our focus groups said they had difficulties finding available coach-and-evaluators at disaster sites. For example, our analysis of FEMA data found that 36 percent of FEMA’s incident management workforce did not have a coach-and-evaluator at the start of their deployment during the 2017 and 2018 disaster seasons. In addition, according to staff in our focus groups and interviews, coach-and- evaluators at the disaster often do not have time to coach staff. For example, officials at one of the joint field offices we visited said mission needs always come first and coaching and evaluating responsibilities are often not anyone’s priority. In addition to on-the-job training challenges related to the FEMA Qualification System, focus group participants also reported more general challenges with on-the-job training. For instance, multiple supervisors in the Logistics cadre at one joint field office said that in addition to doing their own work, experienced staff members need to spend significant time training others, which competes with performing their mission. Furthermore, participants in seven of the 17 focus groups said providing on-the-job training was particularly challenging at the beginning of a disaster, when the disaster is often hectic and at its busiest. Recovery Division officials in a FEMA regional office said a challenge at the start of the disaster is finding staff members who know what to do and have the time to train those who do not. Staff members also described difficulties with providing and receiving on-the-job training in later phases of a disaster. In one focus group with supervisors, a participant said that once the disaster has reached a pace where they have time to train, staff members are often redeployed. Finally, in 16 of our 29 field and regional interviews, officials said there was a lack of mentoring and sustained staff development across disasters. For example, officials at one joint field office told us that once staff members complete their position task book, they generally do not receive any additional coaching or mentoring in that position. This official stated that reservists have a more difficult time identifying mentors than other employee types because they deploy intermittently and likely have different supervisors and coach-and-evaluators each time they deploy. In addition, FEMA officials said coach-and-evaluators are not meant to serve as mentors. FEMA human capital officials said that different offices can develop their own mentoring programs but these may not be available to all employee types. As a result, not all staff members know to ask for, or expect to receive, mentoring. FEMA headquarters officials acknowledged some of these staff development challenges and described actions they have planned, or are underway, to help address some of them. Specifically, FEMA revised the coach-and-evaluator course in 2017 to place a greater emphasis on the coaching responsibilities of the coach-and-evaluator role. For example, the revised course teaches effective coaching strategies, including how to give effective, actionable feedback. Also, in summer 2019, FEMA conducted a pilot with the National Disaster Recovery Support cadre to deploy a single coach-and-evaluator solely in that position and communicated to cadre management that this individual was not to be used for other disaster-related responsibilities. FEMA officials said this pilot was a success. In evaluating the pilot, FEMA said the coach-and- evaluator was able to devote time to proper training and answering any questions presented. Finally, the agency revised the FEMA Qualification System Guide in August 2019, which included clarifying differences between coaching and evaluating. The revised guide states that, as part of the position task book process, a coach explains, demonstrates, trains, assesses, and documents an individual’s task performance while an evaluator observes, assesses, documents, and endorses an employee’s independent performance of specific tasks. FEMA Officials Reported Inconsistent Use of Performance Evaluations at Disasters Headquarters officials told us that during the 2017 and 2018 disaster seasons, disaster workforce employees inconsistently received performance evaluations when deployed. Performance evaluations at disasters are to be completed on a paper form by a temporary duty supervisor. If the staff member has a coach-and-evaluator, the temporary supervisor may request input regarding progress toward mastering the skills covered by the position task book. The temporary supervisor is supposed to provide that evaluation to cadre management if an evaluation was completed. However, FEMA officials told us there are no mechanisms in place to ensure these steps occur or that the evaluations will be used to help develop staff competencies, and it is not something FEMA officials monitor. Further, FEMA headquarters officials stated there are no controls in place to ensure supervisors rate staff consistently from supervisor to supervisor. These officials told us they are aware of some problems with how the agency conducts performance evaluations for the disaster workforce and are developing changes to address them. For example, in the months prior to the 2017 disasters, the agency began revising its performance evaluation system, but suspended its efforts when that year’s disasters occurred. In 2019, FEMA resumed this initiative and agency officials told us they expect it will be implemented by June 2020. They said the new system will include replacing the paper form with an electronic program that will be integrated into FEMA’s other personnel systems, such as the Deployment Tracking System. Further, in March 2020, FEMA officials told us they are finalizing a directive intended to provide guidance to supervisors at disasters on how they are to provide deployment performance evaluations. In addition, in April 2020, FEMA issued guidance for the administration, implementation, and oversight of a performance management process that will provide reservists with annual performance appraisals. FEMA officials told us this will help ensure that reservist performance appraisals accurately reflect their job performance and assist them in maintaining and improving performance in the future. The agency’s reservist performance management initiative is expected to be completed by January 2021, but officials have not provided specific interim milestones or target dates. Staff and Managers Cited Difficulties with Receiving Staff Development When Not Deployed to a Disaster Many disaster workforce staff members are not likely to get ongoing development directly from their cadre management when they are not deployed. According to data from FEMA, there was one cadre supervisor of record for every 128 reservists and Incident Management CORE staff as of June 1, 2019. During the 2017 and 2018 disaster seasons, this ratio was higher in certain cadres. For example, there was one supervisor of record for every 807 reservists and Incident Management CORE staff as of June 1, 2017 in the Individual Assistance cadre. FEMA headquarters officials told us they are assessing what the right mix of supervisors to reservists should be across the cadres. Further, staff members told us they have difficulties getting ongoing development through hands-on training outside of a disaster. While FEMA headquarters officials told us that cadres periodically conduct mission rehearsal trainings each year to prepare their staff for disasters, they also said not all staff can attend them because cadre management determines which staff to invite. These trainings are designed for staff members to simulate a potential disaster scenario while in a training environment. Finally, FEMA headquarters officials stated that receiving ongoing development for staff who do not deploy frequently, such as reservists, can be a challenge. The only instances when reservists are paid while not deployed occur when they complete 40 hours a year of mandatory training or 32 hours a year coordinating with their cadre. In addition, an individual in one of our focus groups with permanent full-time employees said reservists had difficulties accessing online mandatory training because they did not have a FEMA laptop. A recovery manager in a FEMA regional office told us that it can be challenging to provide staff development for reservists because they are generally sent to the field to do a discrete job and have limited opportunities to develop their skills and competencies when not deployed. As discussed above, FEMA’s disaster workforce reported challenges receiving staff development through the agency’s existing methods, which consists primarily of classroom training, on-the-job training and mentoring, and performance evaluations. While FEMA has taken actions to address some of the challenges staff experienced, opportunities remain to ensure more effective and consistent staff development. Specifically, FEMA does not have a staff development program in place to provide assurance of effective and comprehensive staff development of the skills and abilities needed during deployments. Further, FEMA headquarters officials said it is primarily the responsibility of staff members to find available coach-and-evaluators at disaster sites and the agency has not developed a mechanism to help ensure deployed staff are consistently paired with coach-and-evaluators. In addition, FEMA headquarters has not taken actions to address the challenges we identified with the lack of mentoring for staff deployed to disasters. Further, given that FEMA’s performance evaluation initiatives are not yet implemented, it is too early to assess how effective they will be in enhancing staff development, including whether they will have mechanisms in place to ensure employees receive useful evaluations or the extent to which they will be coordinated with other development activities, such as coaching through on-the-job training. Standards for Internal Control in the Federal Government states that management recruits, develops, and retains competent personnel to achieve the entity’s objectives. This includes enabling individuals to develop competencies appropriate for key roles, reinforcing standards of conduct, and tailoring training based on the needs of the role. It also includes mentoring to develop individual performance based on standards of conduct and expectations of competence that align the individual’s skills and expertise with the entity’s objectives and help personnel adapt to an evolving environment. In addition, we have previously reported that identifying where an agency’s development process is lacking can help address barriers that hinder its ability to achieve meaningful results. We also reported that it is important for agencies to treat continuous learning as an investment in success as it can address employees’ career development issues, skill-specific training needs, and provide managers with opportunities to identify where training and development is appropriate. Effective and consistent staff development is particularly important because FEMA has hired a large number of reservists over the past few years. Our analysis of FEMA data shows that from June 1, 2017 to May 31, 2019, the agency hired over 3,200 reservists, which was 40 percent of the agency’s entire reservist workforce as of June 1, 2019. Creating a staff development program that systematically and comprehensively addresses staff development through courses, on-the-job training and mentoring, performance evaluation, and ongoing developmental opportunities would provide better assurance that staff develop the skills and competencies needed to meet mission needs during field operations and help ensure the best results for disaster survivors. Conclusions The large-scale and concurrent disasters during the 2017 and 2018 disaster seasons highlighted the complex challenges facing FEMA’s workforce. The agency deployed 14,684 and 10,328 personnel, respectively, at the peak of each of these disaster seasons, and the increased demand for its workforce is expected to continue. Without accurate and complete information on the knowledge, skills, and abilities of these staff members, field officials face challenges with efficiently providing disaster assistance, managing staff workload, and assigning responsibilities. FEMA has taken some initial actions to improve the information provided by its qualification and deployment systems, such as establishing additional controls in its qualification process. However, developing a plan to address the information challenges experienced during the 2017 and 2018 disaster seasons would be beneficial to enhance field leadership’s ability to identify and leverage staff skills and, given the persistence of some of these challenges, help ensure they do not continue to affect FEMA’s ability to support mission needs in future disasters. Further, in light of the staffing constraints that FEMA faces, it is important that the agency be able to assess how effectively it deploys available staff to disasters to meet field needs. Developing a mechanism to assess FEMA’s deployment outcomes would provide officials in headquarters with critical information to monitor and evaluate the extent to which its deployment policies and strategies effectively place staff with the right skills in the right place at the right time to meet mission needs and take corrective actions if needed. Finally, creating a staff development program for its disaster workforce that addresses access to training, delivery of on-the-job training and mentoring, use of performance evaluations, and developmental opportunities when not deployed would help FEMA ensure more consistent and comprehensive development of the skills and abilities needed during deployments. Consistent and effective staff development is particularly important to help build the skills of staff who are qualified in the FEMA Qualification System but unable to proficiently perform their duties and develop the large number of staff that FEMA has recently hired to meet its new disaster workforce targets. Recommendations for Executive Action We are making the following three recommendations to FEMA: The FEMA Administrator should develop a plan—with time frames and milestones and input from field leadership—to address identified challenges that have hindered FEMA’s ability to provide reliable and complete information to field leaders and managers about staff knowledge, skills, and abilities. (Recommendation 1) The FEMA Administrator should develop mechanisms, including collecting relevant data, to assess how effectively FEMA’s disaster workforce was deployed to meet mission needs in the field. (Recommendation 2) The FEMA Administrator should create a staff development program for FEMA’s disaster workforce that, at a minimum, addresses access to training, delivery of on-the-job training and mentoring, use of performance evaluations, and consistent developmental opportunities regardless of deployment status. (Recommendation 3) Agency Comments and Our Evaluation We provided a draft of this report to DHS for review and comment. DHS provided written comments, which are reprinted in appendix III and summarized below. In its comments, DHS concurred with our three recommendations and provided a number of ongoing and planned actions that it intends to leverage in addressing them. DHS also provided technical comments, which we incorporated as appropriate. With regard to our first recommendation for FEMA to develop a plan to address identified challenges with providing reliable and complete staffing information to the field, DHS reiterated some of the steps described in this report that FEMA has taken to improve the coach-and-evaluator program. DHS noted that FEMA plans to engage field leaders on these initiatives to develop a plan to address identified challenges. DHS also reported that FEMA plans to increase training offerings and align its curriculum so that FEMA Qualification System status matches workforce capability. DHS anticipates these efforts will be completed by March 31, 2021. While these are positive initial steps, they focus solely on the coach-and- evaluator program and staff training. Our report identified a number of complex and interrelated challenges with the agency’s qualification and deployment processes that hindered FEMA’s ability to provide reliable information to field officials about staff members’ skills and abilities, including their qualifications, specialized skillsets, and experience within and across program areas. As such, in developing the plan we recommended, it will be important for FEMA to take a comprehensive approach and consider solutions that may cut across multiple systems and processes. We will monitor DHS’s and FEMA’s efforts in this area to assess the extent to which they fully implement our recommendation. With regard to our second recommendation for FEMA to develop mechanisms to assess how effectively FEMA’s disaster workforce was deployed to meet mission needs in the field, DHS reiterated the actions described in this report that FEMA took to establish new force structure targets for its incident management workforce. DHS also reported that FEMA plans to convene subject matter experts to develop mechanisms that demonstrate how effectively FEMA’s disaster workforce deploys to meet mission needs in the field, which are expected to be completed by March 31, 2021. When they are complete, we will assess the mechanisms to determine the extent to which they address our recommendation. Regarding our third recommendation for FEMA to create a staff development program, DHS reiterated some of the actions FEMA has taken to develop its disaster workforce that were described in this report. Our report identified recurrent challenges with FEMA’s efforts to develop staff through training courses, on-the-job training and mentoring, and performance evaluations and noted that the agency’s current and planned efforts do not fully address these challenges. In creating the staff development program we recommended, it is important for FEMA to consider how its overall control environment and the initiatives it puts in place are coordinated to ensure staff receive comprehensive and consistent development to build the skills needed during disaster field operations. DHS anticipates that FEMA’s efforts to implement our recommendation will be completed by March 31, 2021. At that time, we will assess the agency’s actions to determine the extent to which they address the intent of our recommendation. We are sending copies of this report to the appropriate congressional committees, the Secretary of Homeland Security, the FEMA Administrator, and other interested parties. If you or your staff have any questions about this report, please contact me at (404) 679-1875 or curriec@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. Appendix I: Objectives, Scope, and Methodology This report addresses (1) how the Federal Emergency Management Agency’s (FEMA) disaster workforce is qualified and deployed, and workforce staffing levels during the 2017 and 2018 disaster seasons; (2) how effective FEMA’s qualification and deployment processes were during the 2017 and 2018 disaster seasons in helping ensure workforce needs were met in the field; and (3) the extent to which FEMA’s disaster workforce receives staff development to enhance skills and competencies to support the agency’s disaster missions. Our review focused on FEMA’s incident management workforce, which is composed of FEMA staff who deploy to disaster sites. We defined the 2017 and 2018 disaster seasons as the time periods from August 23, 2017 through January 31, 2018, and September 7, 2018 through November 25, 2018. The 2017 dates represent the start of the FEMA incident period for Hurricane Harvey through the end of the incident period for the California wildfire season. The 2018 dates represent the start of the FEMA incident period for Hurricane Florence through the end of the incident period for the California wildfires. To address all three objectives, we (1) analyzed documentation and data on incident management workforce qualification, deployment, staffing levels, and development; (2) conducted focus groups with members of FEMA’s incident workforce across a range of employee types— permanent full-time employees, Cadre of On-Call Response/Recovery Employees (CORE), Incident Management CORE, reservists and local hires; and (3) interviewed FEMA officials in headquarters and field and regional offices. We compared the results of our analysis and the information we gathered with Standards for Internal Control in the Federal Government, The Standard for Program Management, FEMA strategic documents and guidance, and our prior reports on strategic human capital management and strategic training and development. Analysis of FEMA Workforce Documents and Data We analyzed documentation on how FEMA’s incident management workforce is qualified, deployed, and developed. Documentation included the agency’s 2017 Incident Management Handbook, 2015 CORE Program Manual, 2017 Reservist Program Directive, 2015 and 2019 FEMA Qualification System guides, 2019 Coach-and-Evaluator Program Directive, coach-and-evaluator training materials, 2014 Incident Workforce Deployment Directive, and 2019 Deployment Guide. In addition, we analyzed FEMA’s 2018-2022 Strategic Plan, 2017 Hurricane Season After-Action Report, and documentation on FEMA’s staffing targets for its incident management workforce. We analyzed data from FEMA’s Deployment Tracking System to determine incident management staffing levels, the number of staff deployed to a disaster, the number of incident management staff that had a coach-and-evaluator assigned, and the ratio of managers to incident management staff. We also analyzed data FEMA provides to the National Finance Center to determine the number of new staff the agency hired. To assess the reliability of the data, we interviewed officials at FEMA headquarters about their data quality control procedures and reviewed documentation about these data systems. For the Deployment Tracking System, we also conducted electronic testing and reviewed the data for obvious errors and omissions. We found the data to be sufficiently reliable for the purposes of this report. Focus Groups with Incident Management Staff Members As shown in table 4, to obtain perspectives on how effectively FEMA qualifies, deploys, and develops its disaster workforce, we conducted 17 focus groups with a total of 129 participants who were deployed in incident management positions during the 2017 disaster season, and in some cases, the 2018 disaster season. The focus group locations were selected based on where staff members who were deployed during the 2017 disaster season were located at the time of our review. We also selected these locations to reflect where the 2017 disasters occurred and to obtain variation in geographic location to the extent possible. Participants were selected using a stratified random sample from a universe of incident management staff members who were deployed to a federally declared disaster during the 2017 hurricane and wildfire season. For each employee type, we conducted separate focus groups with participants in supervisory and nonsupervisory positions so they could speak more freely. We also selected participants to obtain a mix of staff from different cadres and a mix of staff that were qualified and not qualified in the FEMA Qualification System. If selected staff members indicated they could not attend, we replaced them with the next individual on our randomized list who had similar attributes. There were between three to 11 participants in each focus group, with an average of eight in each. These focus group discussions were guided by a moderator who used a structured list of discussion topics. The topics focused on staff members’ perspectives on, and experiences with, the level of staffing and skill sets their team had, how they were trained and developed, and the FEMA Qualification System and its qualification determinations. Supervisors were also asked about their staff’s skill sets, training, and qualification status. Focus group sessions were audio recorded and transcribed. We evaluated the transcripts using systematic content analysis to identify key themes on how effective FEMA’s qualification and deployment processes were in helping to meet field needs and the extent to which staff members received staff development to enhance their skills and competencies. An analyst coded the transcripts and a second analyst validated the coding. Any discrepancies were resolved by both analysts agreeing on the coding of the associated statement by a participant. If needed, a third analyst adjudicated any continued disagreement between coders. The results of our focus group analysis are not generalizable to all incident management staff members. However, they provided valuable first-hand experiences with staffing levels and skill sets during disasters, FEMA’s deployment processes, the FEMA Qualification System and the reliability of its qualification designations, and how well staff were trained and developed. Interviews with FEMA Officials in Field and Regional Offices and Headquarters We conducted site visits to FEMA’s joint field offices in Columbia, South Carolina; Durham, North Carolina; Guaynabo, Puerto Rico; and Tallahassee, Florida, to obtain officials’ perspectives on staffing levels and skill sets, the effectiveness of FEMA’s qualification and deployment processes and systems in meeting field needs, and the extent to which FEMA’s deployed staff receive coaching and development to enhance their skills and competencies. Officials we interviewed at the joint field offices included federal coordinating officers; chiefs of staff; training managers; and managers in the Individual Assistance, Public Assistance, Hazard Mitigation, and Logistics cadres, among others. We also interviewed an official who was previously a federal coordinating officer at a federally-declared wildfire in California. In addition, we interviewed leadership and managers for FEMA regions VI, VIII, and X to obtain the perspectives of regional officials on the topics above. In each of the regions, we interviewed the regional administrator and managers in both the response and recovery divisions, among others. We selected the joint field offices and regions to conduct interviews based on our focus group locations and to obtain variation in geographic location and disaster activity. We conducted systematic content analysis of this work using the same approach we used to analyze the focus groups. The results from this analysis are not generalizable to all field and regional officials, but provide important perspectives from leadership and managers on FEMA’s mechanisms to qualify, deploy, and develop incident management staff. In addition, we conducted interviews with multiple senior officials in FEMA headquarters. For example, we interviewed officials in the Field Operations Directorate and management in the Individual Assistance, Public Assistance, and Hazard Mitigation cadres to obtain information about how FEMA’s incident management workforce and staff in their cadres are qualified, deployed, and developed, and how the Deployment Tracking System and the FEMA Qualification System are used for these purposes. We also interviewed officials in the Office of the Chief Component Human Capital Officer to learn how FEMA trains and develops this workforce. We obtained information from these officials on the actions FEMA has taken to address the challenges we identified through our focus groups, interviews with field and regional officials, and data analysis. We conducted this performance audit from June 2018 to May 2020 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Federal Emergency Management Agency (FEMA) Cadre List and Descriptions Appendix III: Comments from the U.S. Department of Homeland Security Appendix IV: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments Chris Currie, (404) 679-1875 or curriec@gao.gov In addition to the contact named above, Kathryn Godfrey (Assistant Director), Johanna Wong (Analyst-in-Charge), James Cook, Lawrence Crockett, Elizabeth Dretsch, Ricki Gaber, Eric Hauswirth, Tracey King, Ronald La Due Lake, Rebecca Mendelsohn, Amanda Miller, and Adam Vogt made key contributions to this report.