Some Self-Insured Operator Bankruptcies Shifted $865 million in Estimated Liability to the Trust Fund, but Commercial Insurance Coverage Can Help Limit Trust Fund Exposure Of the eight coal mine operator bankruptcies we identified, three resulted in a transfer of estimated benefit liability from the coal operator to the Trust Fund and five did not, according to DOL. Figure 1 shows how many operators were self-insured or commercially insured at the time of bankruptcy, and if responsibility for benefits was shifted from the bankrupt operator to the Trust Fund. Federal law generally requires coal mine operators to secure their black lung benefit liability. A self-insured coal mine operator assumes the financial responsibility for providing black lung benefits to its eligible employees by paying claims as they are incurred. Operators are allowed to self-insure if they meet certain DOL conditions. For instance, operators applying to self-insure must obtain collateral in the form of an indemnity bond, deposit or trust, or letter of credit in an amount deemed necessary and sufficient by DOL to secure their liability. Operators that do not self-insure are generally required to obtain coverage from commercial insurance companies, state workers’ compensation insurance funds, or other entities authorized under state law to insure workers’ compensation. From 2014 through 2016, three self-insured coal mine operator bankruptcies resulted in a transfer of $865 million of benefit liabilities from the coal operators to the Trust Fund, according to DOL estimates (see table 1). DOL estimates for how these bankruptcies will affect the Trust Fund have considerably increased from what DOL had previously reported. In June 2019, we reported that DOL estimated that between $313 million to $325 million in benefit liabilities would transfer to the Trust Fund as a result of these bankruptcies. In January 2020, however, DOL provided updated estimates stating that $865 million in benefit liabilities would transfer to the Trust Fund as a result of these bankruptcies. According to DOL, their estimates increased, among other reasons, to account for higher black lung benefit award rates that occurred from fiscal years 2016 through 2019, and higher medical treatment cost inflation in recent years. Additionally, DOL’s prior estimate for the Patriot Coal (Patriot) bankruptcy did not account for future claims and their effect on the Trust Fund. The amount of collateral DOL required from these operators to self-insure did not fully cover their estimated benefit liabilities. When this occurs, benefit liabilities in excess of the collateral can be transferred to the Trust Fund. For example, the collateral DOL required from Alpha Natural Resources (Alpha) was about $12 million and approximately $494 million of estimated benefit liability transferred to the Trust Fund, according to DOL. The three other self-insured coal mine operator bankruptcies we identified did not affect the Trust Fund. Specifically, Arch Coal, Peabody Energy, and Walter Energy were also self-insured operators, but DOL officials said that their federal black lung benefit liabilities were assumed by a reorganized company or by a purchaser, and therefore did not transfer to the Trust Fund. Insurance contracts or policies to secure operators’ benefit liabilities are required by law to include a provision that insolvency or bankruptcy of an operator does not release the insurer from the obligation to make benefit payments. Additionally, state insurance regulation, insurer underwriting, risk management practices, and state guaranty funds help to protect the Trust Fund from having to assume responsibility for paying black lung benefits on behalf of bankrupt coal operators. Thus, by being commercially insured, the two operators we identified that filed for bankruptcy between 2014 and 2016—Energy Future Holdings and Xinergy Ltd—did not affect the Trust Fund, according to DOL (see fig. 1). Since 2016, several other self-insured operators have also filed for bankruptcy, according to DOL officials, including Cambrian Coal, Cloud Peak Energy, Murray Energy, and Westmoreland Coal. DOL officials said that about $17.4 million in estimated black lung benefit liability will transfer to the Trust Fund as a result of Westmoreland Coal’s bankruptcy. Given the uncertainty of the bankruptcy process in terms of whether liabilities will or will not transfer to the Trust Fund, however, DOL officials said that they could not speculate on how these other bankruptcies may affect the Trust Fund. DOL’s Limited Oversight Has Exposed the Trust Fund to Financial Risk, and Its New Self-Insurance Process Lacks Enforcement Procedures In overseeing coal mine operator self-insurance in the past, DOL did not estimate future benefit liability when setting collateral or regularly review operators to monitor their changing financial conditions. DOL regulations require that collateral be obtained from operators in an amount deemed necessary and sufficient to secure the payment of the operators’ liability. To determine collateral amounts under the former process, agency procedures stated that an operator’s net worth be assessed by reviewing, among other factors, the operator’s audited financial statements and black lung claims information. The amount of collateral was to be equal to 3, 5, or 10 years of the operator’s annual black lung benefit payments made at the time of a coal operator’s self-insurance application depending on its net worth. Specifically, if net worth was $1 billion or greater, agency procedures set collateral equal to 3 years of benefit payments. If net worth ranged from $500 million to $1 billion, collateral was equal to 5 years of benefit payments. If net worth ranged from $10 million to $500 million, DOL set collateral equal to 10 years of benefit payments. Agency procedures did not permit operators with net worth less than $10 million to self-insure. DOL’s former process for determining collateral did not routinely consider potential future claims for which an operator could be responsible. The agency periodically reauthorized coal operators to self-insure by reviewing an operator’s most recent audited financial statement and claims information, among other things. DOL prepared memos documenting these reviews, and communicated with coal operators about whether their financial circumstances warranted increasing or decreasing their collateral. Regulations state that DOL may adjust the amount of collateral required from self-insured operators when experience or changed conditions so warrant, but regular monitoring of self-insured operators was not conducted. In reviewing the most recent reauthorization memos for each of the self-insured operators, we found that while some of these operators had been reauthorized more recently, others had not been in decades. One operator in particular had not been reauthorized since 1988. There were no written procedures that specified how often reauthorizations should occur after an operator’s initial 18-month reauthorization. DOL has other tools available to mitigate financial losses to the Trust Fund. These include revoking an operator’s ability to self-insure; fining mine operators for operating without insurance; and placing liens on operator assets. Based on our review of agency documentation, however, we found instances when officials did not use these tools to protect the Trust Fund, or were hindered from doing so because of an operator’s ongoing appeal or bankruptcy. James River. In September 2001, DOL required $5 million in additional collateral from James River Coal (James River), which would have increased its collateral from $0.4 million to $5.4 million. Although DOL did not receive the additional collateral, it did not revoke the operator’s authority to self-insure, which is a potential option under agency regulations. Further, DOL had not reauthorized James River at any point from August 2001 until it filed for bankruptcy in April 2014. If James River’s ability to self-insure had been revoked, DOL could have potentially prevented the Trust Fund from being responsible for claims based on a miner’s employment from 2001 through 2016, when James River liquidated. Additionally, if the operator had been unable to obtain commercial insurance, the agency could have potentially fined the operator for each day it operated without insurance. Instead, no action was taken during these years and estimated benefit liability of $141 million was shifted to the Trust Fund, according to DOL. DOL officials stated that they do not have records explaining why James River did not provide the additional collateral or why they did not revoke its authority to self-insure. Patriot. In August 2014, DOL required an additional $65 million in collateral from Patriot, increasing its collateral from $15 million to $80 million. Patriot appealed this decision and, in the 8 months that followed before Patriot filed for bankruptcy in May 2015, DOL did not obtain additional collateral, or revoke Patriot’s ability to self-insure because the appeal was still pending. DOL officials said they would not typically revoke an operator’s authority to self-insure during an ongoing appeal. As a result, DOL was hindered from using this enforcement tool. Liens on operator assets can be an effective tool to protect the Trust Fund if an operator defaults on its benefit liability, but DOL officials said they are hindered from using this tool if an operator files for bankruptcy. DOL can place a lien on a coal operator’s assets under federal law if they refuse the demand to pay the black lung benefit payments for which they are liable. In the event of bankruptcy or insolvency, federal law states that the lien imposed shall be treated in the same manner as a lien for taxes due and owing to the United States under certain laws. However, DOL officials said that operators rarely stop paying benefits until after they file for bankruptcy. Once a bankruptcy occurs, DOL officials said that they are generally prevented by the court from placing a lien and taking an operator’s assets in lieu of payment of current and future benefit liability. Under bankruptcy law, DOL officials said that they have no special status over other creditors with outstanding financial claims. DOL officials said that obtaining sufficient collateral is a better way to protect the Trust Fund. DOL’s New Self-Insurance Process May Help Address Problems, but Key Enforcement Procedures Are Needed In July 2019, DOL began implementing a new process for coal mine operator self-insurance that may help to address some past deficiencies, if implemented effectively. Among other things, DOL will require operators to periodically submit financial and claims information, including an actuarial estimate of the operator’s current and future benefit liability. DOL plans to use this information to assess the insolvency risk of each operator. Depending on the results of their analysis, DOL plans to categorize the risk-level of each applicant as low, medium, or high. DOL will then set the amount of collateral required to self-insure by linking the operator’s risk category to a corresponding percentage of the operator’s actuarial estimated benefit liability. DOL policies state that they would require a high-risk operator to secure with collateral 90 percent of estimated benefit liability, a medium-risk operator to secure 45 percent, and a low-risk operator to secure 15 percent. However, in February 2020, DOL officials said they plan to revise these percentages to 100 percent, 85 percent, and 70 percent for high-risk, medium-risk, and low-risk operators, respectively. Coal mine operators that are already authorized to self-insure will be required to submit, among other things, an annual renewal application. DOL plans to use this information to update their insolvency risk analysis. If an operator’s risk category changes (e.g., from low-to medium-risk), DOL plans to send a form to the operator requiring an additional amount or type of collateral. Upon receiving the completed form, and proof that the collateral has been obtained, DOL stated that they will notify the operator that its authority to self-insure has been reauthorized. DOL’s new self-insurance process made important changes, but overlooked other key internal control improvements that are needed to protect the financial interests of the Trust Fund. DOL’s new requirements for setting collateral and for the more frequent review of self-insured operators are key components of internal controls, which call for agency management to implement control activities through policy. However, DOL’s new self-insurance procedures do not specify (1) the duration of an operator’s self-insurance authority, (2) the time frames for submitting renewal applications and supporting documentation, and (3) the conditions under which an operator’s self-insurance authority would not be renewed. Our report recommends that DOL implement procedures for coal mine operator self-insurance renewal that clarifies how long an operator is authorized to self-insure; when an operator must submit its renewal application and supporting documentation; and the conditions under which an operator’s self-insurance authority would not be renewed. DOL agreed with this recommendation and stated that it will ensure letters granting or renewing self-insurance authority will inform operators that their authorization expires in one year and that they must submit renewal information three months in advance of the expiration date. DOL staff are hindered from taking enforcement action during an operator’s ongoing appeal, as previously mentioned. DOL policies state that an operator may request reconsideration if its self-insurance application has been denied or if it believes the collateral required by DOL is too high to secure its benefit liability. However, DOL lacks procedures that specify, among other things, the length of time that operators have to submit supporting information. Further, DOL does not specify a goal for how much time DOL appeals decisions should take. For example, in October 2015, DOL recommended revoking Murray Energy’s (Murray) authority to self-insure due to deteriorating financial conditions. Murray appealed this decision, and DOL officials said they postponed responding to the appeal until their new self-insurance process was implemented. However, Murray filed for bankruptcy in October 2019 and DOL had not revoked its authority to self-insure or requested additional collateral because Murray’s appeal was still pending and DOL was still evaluating how much collateral it would require from the operator under its new self- insurance process. Our report recommends that DOL develop and implement procedures for self-insured coal mine operator appeals that identify time lines for self- insured operators to submit documentation supporting their appeals and that identify a goal for how much time DOL should take to make appeals decisions. DOL agreed with this recommendation and stated that they will ensure letters denying self-insurance will inform operators that they have a 30-day appeal period (limited to one extension), and that DOL has set a goal of resolving all appeals within 90 days of the denial letter. Commercial Insurance Oversight Improvements Are Needed We found that DOL does not monitor coal mine operators that do not self- insure and, thus, must commercially insure their federal black lung liabilities to make certain they maintain adequate and continuous coverage as required by law. In the absence of effective DOL monitoring, we evaluated the potential risk that uninsured operators could pose to the Trust Fund. Specifically, in examining the 13 largest coal mine operators that do not self-insure, we found that some insurers erred in reporting black lung policies and in one instance an operator did not have adequate coverage. We found six operators (parent or subsidiary) that were not insured for the entire 3-year period from 2016 through 2018, according to our review of DOL data. When we discussed our findings with DOL, agency officials had to research each operator individually and in some cases contact the operator or their insurer to find out whether or not they had been covered. DOL concluded that these entities were insured. However, the insurers had not properly reported the federal black lung endorsement on new policies or subsequent renewals, in addition to other reporting issues. One of these six operators also had, inadvertently, not maintained adequate commercial coverage for its mining operations in Texas, and had not self-insured those operations. In this instance, the operator obtained an excess loss policy that only pays claims once they exceed a high threshold and, therefore, is not sufficient by itself to secure the payment of the operator’s benefit liability. Designing processes to achieve agency objectives and respond to risks is a principle of effective internal controls. Without a process to monitor operator compliance with program insurance requirements, DOL risks not identifying a lapse or cancellation of operator coverage. This could result in the Trust Fund having to assume responsibility for paying benefits that would otherwise have been paid by an insurer. Our report recommends that DOL should develop and implement a process to monitor operator compliance with commercial insurance requirements and periodically evaluate the effectiveness of this process. DOL agreed with this recommendation and stated that it will modify existing computer systems to identify lapses or cancellations of commercial insurance coverage, and require operators identified as having lapsed or cancelled coverage to obtain or provide proof of coverage within 30 days. Chairwoman Adams, Ranking Member Byrne, and Members of the Subcommittee, this concludes my prepared statement. I would be happy to respond to any questions you may have at this time. If you or your staffs have any questions concerning this testimony, please contact Cindy Brown Barnes at (202) 512-7215 or brownbarnesc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. GAO staff who made key contributions to this testimony are Alicia Puente Cackley, (Director), Blake Ainsworth (Assistant Director), Patrick Ward (Assistant Director), Justin Dunleavy (Analyst-in-Charge), Alex Galuten, Rosemary Torres Lerma, Olivia Lopez, Scott McNulty, and Almeta Spencer. This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.


Background Counting the nation’s approximately 140 million households is an enormous undertaking requiring such essential logistics as the opening of hundreds of area offices to conduct essential field activities, recruiting and hiring hundreds of thousands of temporary workers to carry out those activities, and developing an approach to training those employees. To help control costs while maintaining accuracy, the Bureau is making significant changes in each of these areas compared to prior decennials. Area Census Offices According to Bureau planning documents, the Bureau intends to use technology to efficiently and effectively manage the 2020 Census fieldwork, and as a result, reduce the staffing, infrastructure, and brick- and-mortar footprint required for the 2020 Census. The three main components of the reengineered field operations are increased use of technology, increased management and staff productivity, and streamlined office and staffing structure. The Bureau’s 2020 Operational Plan states that 2020 Census field operations will rely heavily on automation. For example, the Bureau plans to provide most listers—temporary staff who verify and update addresses and maps—and enumerators— temporary staff who follow up with households that do not respond to the census questionnaire—with the capability to receive work assignments and perform all administrative and data collection tasks directly from a mobile device allowing them to work remotely. Supervisors will also be able to work remotely from the field and communicate with their staff via these devices—precluding them from needing access to a nearby local office. The Bureau’s 2020 Operational Plan states that these enhanced capabilities will significantly reduce the number of offices required to support 2020 Census fieldwork. In the 2010 Census, the Bureau established 12 RCCs and nearly 500 ACOs. The new design for the 2020 Census field operations includes six RCCs with 248 ACOs. Those 248 will be split into two waves, with 39 of the offices opening for Wave 1 by March 2019 to support early census operations such as in-field address canvassing, and the remaining 209 opening for Wave 2 by September 2019. Recruiting and Hiring Recruiting enough workers to fill the hundreds of thousands of temporary positions needed to conduct the 2020 Census is a tremendous challenge. According to Bureau plans before hiring begins, the Bureau needs to assemble an applicant pool in the millions. For the decennial census, Bureau plans indicate the Bureau will need a large and diverse workforce to ensure the accuracy of its maps and address list, and to follow up by phone or in person with households that do not respond to the questionnaire. Making these efforts even more difficult are external factors beyond the Bureau’s control, such as low unemployment rate, which can make it harder to recruit. According to Bureau plans, recruiting of potential employees will be conducted throughout the ACOs’ geographic area, based on projected operational workloads and staffing models developed for 2020 Census operations. Selected candidates will be invited to be fingerprinted and submit selected appointment paperwork prior to attending classroom training. The candidates will be sworn in and hired during the first day of training. The ACO staff model is as follows: one ACO Manager, one Lead Census Field Manager, one Administrative Manager, one Recruiting Manager, one Information Technology (IT) Manager, and Office Operations Supervisors, Clerks, and Recruiting Assistants. For data collection, it is: multiple Census Field Managers, Census Field Supervisors, and Enumerators; specific numbers based on workload; and supervisory ratios to be determined (see fig. 1). Training According to Bureau plans, the 2010 Census approach to training was predominantly instructor-led training with some hands-on training. This primarily consisted of instructors standing in front of a room of trainees and reading training materials to them from a prepared script. For 2020, the Bureau has developed training materials that use a blended training approach including instructor-led training, computer-based training, and hands-on training. This approach is intended to maximize trainee learning and on-the-job performance during the 2020 Census. According to the Bureau’s Detailed Operational Plan for the Field Infrastructure and Decennial Logistics Management Operations, it has developed training materials based on the lessons learned from previous censuses, such as the need to provide computer-based training. The Bureau’s Detailed Operational Plan for the Field Infrastructure and Decennial Logistics Management Operations also states that this innovation to training combines multiple modes of training delivery designed to maximize training outcomes for various types of learning styles: visual, auditory, and hands-on, blending online training methods, instructor-led classroom training, and on-the-job training or role-playing to prepare field staff to effectively fulfill their duties. Blended training is intended to: Provide a standardization of training, limiting the impact of instructor interpretation. Allow for easily updateable training materials in the case of errors or operational changes, minimizing the burden of errata materials. Provide automated assessment tools to enable a more consistent and reliable way to measure learner understanding of concepts. Provide post-training support through easily accessible online manuals and job aids. Training materials are designed to maximize self-paced learning. These accompanying training materials are developed to provide the most up-to- date methodologies for recruiting, onboarding, and training-the-trainer to carry out field data collection activities. The Bureau’s Efforts to Open Area Census Offices Appear on Track, Despite Some Schedule Slippages For the 2020 Census, the Bureau plans to open 248 ACOs. Similar to the 2010 Census, the total number of ACOs for 2020 was derived from the projected workload for field operations based on the number of enumerators needed for nonresponse follow-up. The Bureau allotted a specific number of ACOs to each of its six regional offices. Regions then developed boundaries for the ACO based on seven mandatory criteria that are described in a program memorandum, including that every state have at least one ACO; federally-recognized American Indian areas and military bases (regardless of county, state, or regional boundaries) will be managed by only one ACO; and ACO areas of responsibility will not cross state boundaries (with the exception of Indian reservations and military bases). See figure 2 below for the location of the 248 offices. Requirements for Leased Office Space for Area Census Offices In addition to the criteria used to delineate boundaries for its ACOs, the Bureau also had requirements for the ACO leased space. These requirements, for example, included that the ACO have a certain amount of contiguous square footage depending on the ACO type, and that an ACO not be co-located in a building that also houses agencies with law enforcement responsibilities because of privacy and confidentiality concerns. The Bureau also designated an “area of consideration” for each of its ACOs. According to Bureau officials, the area of consideration, which is a smaller geographic range where they would like to house the office, was based on such factors as access to public transit, general centrality within the ACO work boundaries, and proximity to eating establishments. In some cases, the Bureau had to deviate from its requirements for leased space or initial area of consideration. The decision to deviate from requirements usually arose from a lack of viable options in the real-estate market coupled with the Bureau’s need to meet its time frames. According to RCC staff, any deviations from requirements were presented at weekly staff meetings and then subsequently approved by the Regional Director, and in some cases such as co-location with law enforcement Bureau Headquarters approval was needed. According to Bureau officials, co-location with law enforcement is sensitive because of concerns that census data may be shared with others. Census data are kept confidential for 72 years. However, Bureau officials told us either the law enforcement offices were deemed innocuous, for example, the office housed a public defender or the law enforcement offices operated undercover, whereby no one entering the building would have been aware of their presence. In another case, Bureau officials told us that the Philadelphia region was struggling to find space for its ACO in Frederick, Maryland. When the General Services Administration (GSA) proposed a space in Hagerstown, Maryland, 30 miles away, the Bureau accepted it, though it was outside the initial area of consideration. According to officials at the regional office, the Bureau saved time and money by using a readily available cost-effective option by choosing Hagerstown, Maryland. The Bureau also had to expand the area of consideration for more than 31 percent or 77 of its 248 ACOs. According to Bureau officials, designating an area of consideration was an iterative process based on market availability, and having to expand the area was often necessary to secure space (see table 1). In select cases, the Bureau co-located ACOs in the same building. For example, instead of having one office in North Philadelphia and one in South Philadelphia, Bureau officials in the Philadelphia Region Census Center agreed to accept space in the same building located within the boundaries of the South Philadelphia ACO. The Bureau hired staff for each ACO from the original designated areas and kept the two offices completely separated. Bureau officials provided documentation indicating that this compromise came with considerable cost savings. The Bureau also abandoned other planned requirements in a number of cases to secure space, such as access to loading docks, assigned parking, and freight elevators. When we reviewed selected ACO files at the regional offices to determine whether the files included support for when deviations from space requirements and initial areas of consideration were documented, we did not find documentation. Instead, documentation was in staff emails. Files included a checklist of documents required, such as the signed lease and design intent drawings; however, there was not a requirement that documentation of deviations from space requirements or initial areas of consideration be maintained. Bureau officials at the regional level said that all procedures for handling waivers and expansions of the area of consideration were driven by the RCCs as well as informal guidance that was not documented. Standards for Internal Control in the Federal Government calls for documentation and records to be properly managed and maintained. Based on our suggestion that the Bureau develop a procedure for documenting these deviations in ACO areas of consideration or requirements, Bureau officials sent an email requiring that staff keep documentation (electronic or paper) on deviations in ACO areas of consideration or requirements in the ACO’s lease file folders. In cases where decisions are made via telephone or email, Bureau officials asked staff to write notes and scan emails, and add them to the ACO files. Maintaining this documentation will help ensure the transparency and integrity of Bureau decision-making, and ensure the information is readily available. The Bureau Is Managing Schedule Slippage in Opening Area Census Offices The Bureau experienced some early delays when regions were trying to find space and acquire leases. The Bureau attributed some of these delays to the use of the GSA’s Automated Advanced Acquisition Program (AAAP) process. This procurement process provides building owners and their authorized representatives with the opportunity to offer general purpose office space for lease to the federal government. The AAAP process accepts bids the first week of each monthly cycle. Then the remaining three weeks of the month are used to evaluate submitted offers and identify a potential lessor. According to GSA documents, in tight real estate markets, the first cycle did not always yield a suitable lessor due to lack of available inventory, and the short lease term the Bureau was seeking. Therefore, the Bureau had to wait three weeks until the start of the next cycle to re-open the bidding process. Bureau officials stated that during these 3 weeks, the Bureau regions would conduct additional market outreach and communicate outreach efforts with GSA to find a lessor. According to GSA, they agreed that too much time was elapsing in Wave I trying to receive offers without making any changes to the requirements or areas of consideration. To address this issue for Wave 2, the Bureau stated that GSA provided additional training to the Bureau’s regional staff, increased market outreach which included dedicated support from GSA’s national office, and the development of a strategy to use all of GSA’s tools, such as using GSA’s contract brokers in regions with the greatest number of Wave 2 ACOs. Bureau regional staff also told us they were able to meet leasing milestones in part because of flexibility in their requirements and in the areas of consideration. As of June 2019, there were signed leases for 247 of 248 offices. However, during our review, the Bureau reported that it had missed several construction (meaning renovations such as new electrical layouts, heating, ventilation, and air conditioning) and deployment deadlines. According to Bureau documents, for Wave 1 offices, nine of 39 offices had missed the February 28, 2019 deadline for having furniture and IT equipment; and for Wave 2 offices, 49 of 209 offices missed the February 20, 2019 deadline for having construction drawings complete. According to Bureau officials they are managing each of these delays on an office- by-office basis, and headquarters officials meet weekly with the RCCs to discuss the status of each office. They are also actively communicating with GSA on how to best work with the landlord to meet deadlines. Agency officials also indicated that the schedule deadlines for the later phases of construction allow for more time than may be necessary, allowing them to make up time lost from early delays. For example, at the Concord, New Hampshire ACO, the Bureau plans to make up lost time in construction with actions such as using a fence to divide two office areas instead of adding a wall, and using a “cage” for badging instead of constructing a separate room inside the space. As of June 3, 2019, 38 of 39 Wave 1 offices are ready for business. Seven of 209 Wave 2 offices are still working to finish the milestone of completing construction drawings, which had an original deadline of February 20, 2019. According to Bureau officials, the seven offices without completed construction drawings are being given priority attention by both GSA and the Bureau. We will continue to monitor the opening of ACOs in ongoing work. The Bureau Has Exceeded Its Early Recruiting Goals for the 2020 Census; However, It Faces Some Challenges Going Forward The Bureau Has Exceeded Its Early Recruiting Goals According to Bureau reporting documents, as of June 2019, the Bureau is exceeding its recruiting goals for early operations. This includes field staff for in-field address canvassing where census staff verify address and map information for housing units in selected areas of the country, office staff at the 39 Wave 1 ACOs, recruiting assistants, and partnership specialists. The Bureau had a goal of recruiting approximately 205,000 individuals for its 2020 early operations efforts by the end of June 2019, and plans to recruit between 2.4 million and 2.6 million applicants for all field operations. By comparison, in 2010, the Bureau recruited about 3.9 million applicants. As of June 17, 2019, the Bureau had processed job applications and assessments for approximately 428,000 applicants which represent about 208 percent of its roughly 205,000 recruiting goal. For the 2020 Census, the Bureau plans to hire nearly 400,000 temporary field staff from its applicant pool for two key operations: in-field address canvassing and nonresponse follow-up, where census staff visit households that do not return census forms to collect data in person. In 2010, the Bureau hired approximately 628,000 temporary workers to conduct the address canvassing and nonresponse follow-up field operations. Below is the recruiting and hiring timeline for the in-field address canvassing and nonresponse follow-up operations (see fig. 3). According to Bureau officials, they are recruiting and hiring fewer temporary staff in 2020 compared to 2010, in part, because automation has made field operations more efficient. For example, there is less paper to manage and process as daily payroll records and daily field work assignments are electronic. As a result, productivity has increased and mileage and labor costs have decreased because census field staff do not meet daily with their supervisors, as was the case in 2010. Moreover, the automation of assignment routing to housing units has optimized the time spent by enumerators driving to housing units. During the 2018 End- to-End Test, the Bureau found the productivity for in-field address canvassing had exceeded its goal at all three test sites (see table 2). The Bureau attributes these efficiencies to the automation of work assignments. The Bureau Plans to Use Successful Recruitment Strategies from Prior Censuses while Leveraging Technology For the 2020 Census, the Bureau plans to use some of the same strategies it used to recruit and hire all temporary workers as during the 2010 Census—because those strategies were successful—while also leveraging technology and social media. For example, according to the Bureau, the overarching strategy for hiring enumerators is to hire people who will work in the communities where they live. This strategy provides the Bureau with enumerators who are familiar with the areas where they will be working and who speak the languages of the local community. To recruit staff, recruiting assistants are to work with local partnership staff and use paid advertisements and earned media (e.g., publicity gained through promotional efforts, news reports, etc.). The Bureau plans to also continue to use its recruiting website, http://www.2020census.gov/jobs, which provides information about the various positions, local pay rates, application materials, and job qualifications. Moreover, Bureau officials stated that a diverse multilingual workforce is needed and that the Bureau has tailored its approach to that end. For example, the website includes Spanish language pages and recruitment materials (see fig. 4). Bureau documentation indicates that similar to 2010, the Bureau will continue to use waivers and hiring exemptions to enable well-qualified individuals to work on the 2020 Census who otherwise might not have applied for jobs, particularly in hard-to-recruit areas. These waivers allow the Bureau to temporarily hire federal retirees and individuals receiving public assistance without impacting their benefits, and to hire current federal employees without impacting their job status or salary. As of February 27, 2019, the Office of Personnel Management (OPM) had given the Bureau approval to hire 44 re-employed annuitants for the 2020 Census. The Bureau also had dual employment agreements with 28 federal agencies and commissions. For the 2010 Census, the Bureau had these agreements with a total of 81 federal agencies. To obtain waivers for individuals on public assistance, the Bureau is partnering with the Office of Management and Budget and working with Health and Human Services to obtain waivers for Temporary Assistance for Needy Families and Supplemental Nutrition Assistance Program recipients. The Bureau is also working with tribal governments to acquire similar waivers. In addition to these previously used strategies, the Bureau is planning to leverage technology in its recruiting strategy for 2020. This technology includes the Bureau-developed Response Outreach Area Mapper (ROAM) application, a publicly available online mapping tool that Bureau staff can use to better understand the sociodemographic makeup of their assigned areas. The Bureau plans to use ROAM to identify areas where recruiting could be hard and to develop recruitment strategies such as hiring staff with specific language skills. The new technology also includes the MOJO Recruiting Dashboard (also referred to as MOJO Recruit), which is software for Census recruiting personnel to plan and manage recruiting activities and track recruiting progress. For example, MOJO Recruit includes an interactive mapping feature which lets the Bureau plan recruiting activities and track recruiting status for each census tract. The map draws attention to areas that may be experiencing recruiting problems (see fig. 5). Red indicates areas where the Bureau is less than 50 percent of the way toward meeting its recruiting goal. Yellow indicates areas where the Bureau is 50 to 79 percent of the way toward meetings its goal. Green indicates areas where the Bureau is 80 percent or higher of the way toward meetings its goal. Bureau officials also stated that that they plan to increase the use of social media platforms such as Facebook, Twitter, and Instagram to promote and advertise 2020 Census job opportunities. For example, the Bureau’s 2020 Census Recruitment Toolkit includes social media guidelines, tips, sample posts, and sample email messages to assist recruiting staff in providing information about 2020 Census job opportunities. It also assists recruiting staff with responding to questions and concerns or directing people to the appropriate location for more information about jobs. The Bureau Took Steps to Improve Its Application and Assessment Process for Potential Hires For the 2020 Census, the Bureau revised its application and assessment process to ease the burden on job applicants and to better assist the Bureau in identifying qualified applicants. Job candidates are to apply and take a skill assessment online, as opposed to attending recruiting sessions in person and taking a written test. The Bureau has also streamlined both the application and assessment process by asking fewer questions and requiring only one assessment for all nonsupervisory positions. According to Bureau officials, the 2020 Census job application should take 10 minutes to complete, by comparison the 2010 Census job application took 30 minutes to complete. Moreover, for prior censuses, applicants had to complete one of two 45-minute assessments to determine the appropriate skill set for either working in the office or in the field. For 2020, OPM has approved the Bureau giving one assessment for all five short-term census positions: Recruiting Assistant, Clerk, Office Operations Supervisor, Enumerator, and Census Field Supervisor, thereby eliminating the need to give separate assessments for the office and field positions. Finally, for those considering a supervisor position, a separate supervisory assessment is required. For 2020, this consists of nine questions compared to 29 questions in 2010. According to Bureau officials, this supervisory assessment should take an additional 10 minutes to complete instead of 1 hour, as it did on 2010. For 2020, the Bureau has also changed the assessment questions it asks applicants from situational-judgment questions to biodata and personality questions. In making this decision, during the 2018 End-to-End Test, the Bureau asked situational-judgment questions in the assessment questionnaire, and then administered a set of biodata and personality questions after hiring. The Bureau conducted an analysis of both types of questions and concluded that the biodata and personality questions were a better predictor of job success. Bureau officials told us they will be evaluating their new job assessment processes for 2020, including the use of biodata. Despite the Progress Made in Recruiting, the Bureau Still Faces Several Hiring Challenges The Bureau has identified challenges that exist in some areas, such as: (1) delayed background checks; (2) low unemployment; and (3) language barriers. Delayed Background Checks Employment with the Bureau is contingent upon successfully completing a background check. The Bureau found that the process for four positions (recruiting assistants, office operation supervisors, clerks, and partnership specialists) was taking longer than it expected. These positions require a full background check because employees will have access to the Bureau’s network, they will be issued expensive equipment (e.g., laptops and desktops), and their employment will likely last more than 6 months. For the full background checks, applicants must complete two security background forms—Standard Form 85: Questionnaire for Nonsensitive positions (SF85) through the Electronic Questionnaires for Investigations Processing system (e-QIP) and Optional Form 306: Declaration for Federal Employment (OF306)—and must have their fingerprints processed, in which the Federal Bureau of Investigations conducts a review for any prior arrest or convictions. Once completed, the forms are reviewed by the Census Investigative Services (CIS) where OPM-trained staff make either a favorable, unfavorable, or inconclusive precheck employment determination. According to Bureau officials, certain crimes, for example violent crimes, automatically exclude the applicant from further consideration. If the determination is inconclusive, then CIS is to send the form to the Office of Employee Relations to make a favorable or unfavorable determination. All favorable determinations are then sent to OPM for adjudication with a full background check (see fig. 6). According to Bureau officials, in December of 2018, they began to encounter a backlog of pre-employment background checks Bureau-wide as they began hiring some 800 recruiting assistants and about 1,970 office staff for the first wave of 39 ACO openings. As of March 21, 2019, Bureau officials told us that Bureau-wide there were 7,092 background clearances pending, of which, 4,900 were for field positions. In response to the backlog, Bureau leadership said it created a team to determine the cause of the backlog and started having weekly meetings to prioritize which job positions needed to be cleared first. Bureau officials stated that the delays arose, in part, because a significant number of applicants did not completely or correctly fill out the e-QIP form. This, they said, coupled with the increase in required pre- employment background checks, resulted in a growing backlog of clearances for which the Bureau did not have the resources to clear. In response, in February 2019, the Bureau began to bring on, through a combination of new hires and reassignments, about 130 temporary staff. New staff was assigned to either review the forms for accuracy and completeness prior to being submitted to the CIS office, or help the CIS offices conduct the pre-employment background checks. Additionally, Bureau officials told us that they meet weekly to reprioritize job positions for the clearance process. The CIS office is to process background checks for all Census employees requiring them, including decennial census field staff, decennial census contractors, and staff needed for nondecennial census surveys in headquarters and in the field. According to Bureau officials, the decennial census takes precedence and within the decennial census positions are also prioritized. For example, in January 2019, the 800 recruiting assistants were given priority and now the hiring of 1,501 partnership specialists has been given priority. Bureau officials told us that in December 2018 they were processing 110 background checks a week, and have set a goal that each CIS analyst process 25 pre-employment packages a week. There are 40 analysts on board, giving the Bureau the ability to process 1,000 pre-employment background check packages a week. Bureau officials also told us that they anticipate the clearance process for the positions of enumerator/lister and census field supervisor will not experience the same delays because these positions only require fingerprint processing, which is quicker. According to Bureau officials, these results can be made available within 3 hours. Moreover, although the Bureau has taken steps to address the backlog, the bulk of pre-employment background clearances has yet to be processed and Bureau officials told us that they remain concerned. In the coming months, the Bureau will need to conduct background checks for an additional 3,991 recruiting assistants and about 10,300 office staff for the remaining 209 offices. We will continue to monitor the backlog of background clearances through our ongoing work. Low Unemployment Although the Bureau has exceeded its recruiting goals for early operations, recruiting a sufficient number of job applicants for the job of partnership specialist is a challenge. Bureau officials told us that a robust economy and low unemployment rate have resulted in a smaller pool of applicants for that position. For example, as part of its 2020 Census efforts, the Bureau had planned to hire 1,181 partnership specialists by May 1, 2019 and 1,501 partnership specialists by June 30, 2019, to help increase awareness and participation in the 2020 Census in minority communities and hard-to-reach populations. The Bureau did not meet its goal to hire 1,181 partnership specialists by May 1, 2019. To hire sufficient partnership staff, Bureau officials told us they have an “open and continuous” posting for partnership specialist positions instead of discrete individual job postings, and they are selecting two candidates from each certification list of qualified applicants. Moreover, Census leadership tracks the weekly progress of the partnership specialist positions. As of July 9, 2019, the Bureau’s latest biweekly reporting indicated that it had hired 813 partnership specialists as of June 22, 2019. Moreover, as of July 10, 2019 Bureau officials told us that another 830 applicants were waiting to have their background checks completed. According to Bureau officials hiring data are based on payroll dates generated biweekly, while background check data are tracked internally. Therefore, according to Bureau officials, more current hiring data were not available as of July 10, 2019 to indicate whether the Bureau had met its June 30 hiring goal. Hiring partnership specialists in a timely manner is key to the Bureau’s ability to carry out its planned outreach efforts, especially for hard-to- count communities. In addition, several RCC officials said the pay rate and the low unemployment rate in some ACO locations initially affected their ability to recruit well-qualified staff for office positions. Atlanta RCC officials stated it was challenging to recruit managers in the Gainesville, Florida, area. According to Bureau officials, the pay rate was too low and potential recruits were seeking employment elsewhere. The Bureau increased the managers’ pay rate to be more competitive for the area. Philadelphia RCC officials stated that in rural ACO locations the pay rate is lower and potential recruits would rather travel to the metro areas to get the higher pay rates offered there. The Denver RCC reported that low unemployment rates throughout the regions make recruiting difficult, and that Census enumerators jobs are not as competitive with many other wages offered in the region. The Los Angeles RCC reported having difficulty recruiting local applicants in high-cost areas like Beverly Hills, the San Francisco Bay Area, and Silicon Valley. Bureau headquarters officials acknowledge that some ACO locations have experienced some recruiting challenges, but said that the RCCs were ultimately able to fill the office positions. Headquarters officials stated that their pay rates either match or exceed the competitive pay rate in the majority of the ACO locations. According to Bureau headquarters officials, regional offices that may be experiencing challenges recruiting staff must demonstrate or prove that the pay rate for a specific ACO is causing difficulty recruiting. The Field Division is responsible for approving or denying the request to adjust pay. For the 2010 Census, the Bureau reported 124 requests for pay rate adjustments, of which 64 were approved. The Bureau stated that it will continue to monitor how low unemployment affects its ability to recruit and hire. Language Barriers The Bureau reports that the demographic and cultural makeup of the United States continues to increase in complexity, including a growing number of households and individuals whose proficiency in English is limited. Language barriers could make it difficult to enumerate these households, whose members may have varying levels of comfort with government involvement. Several RCC officials also mentioned that language barriers could impact their recruiting efforts: Both the Los Angeles and New York RCCs reported it is hard to recruit in immigrant communities where residents speak a foreign language or dialects, and often have no organizational infrastructure (such as associations of individuals of the same national origin, print news media, or radio). The New York RCC reported challenges in locating applicants who are bilingual in English and other languages such as Chinese, Russian, Arabic, Korean, Creole, Polish, Portuguese, Bengali, Urdu, Punjabi, Gujarati, Hindi, and Hebrew, as well as Yiddish and African languages. The Atlanta RCC reported challenges related to the diverse language needs (e.g., Spanish, Chinese, Vietnamese, Creole, Portuguese, etc.) in south and central Florida. The Chicago RCC reported recruiting outreach challenges in urban areas, including Chicago, Indianapolis, Detroit, Minneapolis/St. Paul, St. Louis, and Kansas City, that have higher minority and immigrant populations as well as in rural areas with increasing diversity. Bureau officials responded that later this fall, in preparation for their peak operations effort, they will begin to focus recruiting efforts on foreign language recruiting. Specifically, partnership and recruiting staff plan to work with partners and advertise jobs locally (at the grassroots level) in places where persons with these skills are likely to look to ensure they are meeting recruiting goals in those areas. The Bureau Is Following Its Planned Training Approach for 2020, but Has Opportunities to Improve Its Ability to Assess Performance For the 2020 Census, the Bureau is following its plans to use a blended training approach combining technology-assisted training with classroom instruction. According to Bureau planning documents, on the first day of in-person classroom training, the Bureau will provide orientation information and issue devices that trainees will use to conduct census operations. The Bureau plans to use local institutions such as schools, libraries, churches, and fire halls to host training. ACO staff are to coordinate the training location setup, device deliveries to training sites, and manage other logistics for large-scale field staff training. After the first day of training, field staff will spend the next 4 to 6 days (depending on the operation) completing at-home training online using their own personal device at their own pace. This training will include, for example, operation-specific skills, use of the data collection device (smart phone or tablet), and general field processes. Trainees who complete the online portion of the training program will return to the classroom to practice what they learned through role-playing, mock interviews, or live cases (for listing operations) facilitated by managers or supervisors. According to Bureau officials, employees will also have access to just-in- time training materials on their devices for use in the field. The Bureau Took Steps to Manage Some Challenges in Implementing Its Blended Training Approach During the 2018 End-to-End Test The Bureau encountered a number of challenges in implementing and testing its blended training approach, but is taking steps to mitigate those challenges. Specifically, during the 2018 End-to-End Test, the Bureau (1) experienced problems with the proper recording of online training scores for census staff, (2) was unable to test online training for one of its operations because the operation was added late, and (3) encountered challenges with census staff not always having access to the internet, which is required to complete the training. The Bureau Is Taking Action to Ensure the Completion of Training Is Accurately Tracked The 2018 End-to-End Test of address canvassing and nonresponse follow-up training revealed some technical challenges in using the Learning Management System. The Learning Management System is the online training system for the 2020 Census; it contains online training modules and tracks final assessment scores and training certifications. In February 2019, the Department of Commerce (Commerce) Office of the Inspector General (OIG) noted that during the address canvassing operation there was no final assessment scores recorded for 23 trained listers. The Bureau was also unable to provide documentation that another three lister trainees who failed the final assessment had been observed by their supervisor before being permitted to work. Bureau officials said they provided an action plan to the Commerce OIG in April 2019. According to Bureau officials, the action plan has not been finalized because they are incorporating changes to the action plan based on Commerce OIG comments. In December 2018, we reported that roughly 100 enumerator trainees in the nonresponse follow-up operation were unable to transmit their final test scores because the Learning Management System had an erroneous setting. According to Bureau officials, this problem delayed the start of unsupervised work for these otherwise-qualified enumerator trainees by an average of 2 days per trainee, and resulted in the attrition of some who were able to quickly find other work. Bureau officials reported that they have fixed the system setting. Moreover, according to Bureau officials, they have also developed an alternative means to certify training by incorporating the employee final assessment into the final day of classroom training. The Bureau was Unable to Test All Online Training, but Has Plans in Place to Conduct Dry-Runs of the Untested Training According to Bureau officials, Update Leave online training was not tested during the 2018 End-to-End Test due to the late addition of the operation to the 2020 Census design. Officials told us that the Update Leave operation was approved in May 2017, leaving just 10 months for the development team to create and implement software and the systems to support this field operation for the End-to-End Test. This left no time to develop online training that would be ready for the End-to-End Test in March 2018. Therefore, the Bureau classroom-trained headquarters staff instead of temporary field staff for the operation. According to the Bureau’s risk register, the utilization of Bureau headquarters staff did not properly simulate training conditions or staff characteristics in which new employees have no prior knowledge of census operations. Therefore, the 2018 End-to-End Test did not allow for proper training feedback or the capture of lessons learned with regard to temporary staff or the mode of training. According to Bureau officials, the Bureau plans to conduct scheduled dry runs of training in September 2019 to collect feedback and, if necessary, make changes to Update Leave-specific training. The Bureau Has Plan to Address Trainee Access to Online Training In June 2018, we reported that some listers had difficulty accessing the internet to take online training for address canvassing. According to the Bureau, in addition to the Bureau-provided laptop, listers also needed a personal home computer or laptop and internet access at their home to complete the training. However, while the Bureau reported that listers had access to a personal computer to complete the training, we found some listers did not have access to the internet at their home and had to find workarounds to access the training. We recommended that the Bureau finalize plans for alternate training locations in areas where internet access is a barrier to completing training. The Bureau took action and in March 2019 finalized its plans for identifying alternate training locations in areas where internet access is a barrier to completing training. Specifically, Bureau officials told us that in areas of known low connectivity rates, regional staff will identify sites that trainees can access to complete online components of the training. In addition, the Bureau provided us with a training module for identifying training field staff locations that emphasized training sites need to be located in areas with a good cellular connection and also have access to the internet. The Bureau Has Generally Met the Criteria for Selected Leading Practices for Training Development, but Could Better Document Measures of Success Effective training can enhance the Bureau’s ability to attract and retain employees with the skills and competencies needed to conduct the 2020 Census. Our Guide for Assessing Strategic Training and Development Efforts in the Federal Government describes components for developing effective training in the federal government. Our strategic training guide identifies four phases of the training—planning, design/development, implementation, and evaluation. We assessed the Bureau’s training approach and found that it generally aligned with selected leading practices. This report includes the design/development and evaluation phases of training. We did not assess the implementation phase because field staff training had not yet begun during our audit, and we did not assess the planning phase because practices in that phase are more applicable to agency-wide rather than program-specific training development. Design/Development The design/development phase involves identifying specific training and development initiatives that the agency will use, along with other strategies, to improve individual and agency performance. According to the guide, well-designed training and development programs are linked to agency goals and to the organizational, occupational, and individual skills and competencies needed for the agency to perform effectively. Moreover, in response to emerging demands and the increasing availability of new technologies, agencies, including the Bureau for the 2020 Census, are faced with the challenge of choosing the optimal mix for the specific purpose and situation from a wide range of mechanisms, including classroom and online learning as well as structured on-the-job experiences (see fig. 7). In developing its training approach we found the Bureau met all five selected leading practices related to design/development. Specifically, Bureau training aligned with achieving results for the Bureau’s re- engineered field operations. Specifically, the Bureau has a formal online training program that uses the Learning Management System as a control mechanism to provide and record training results for all 2020 Census field staff who take online training. The Bureau’s training program is integrated with other strategies to improve performance such as building team relationships. For example, the training includes modules for supervisors that focus on guiding and motivating employees, communicating effectively, and resolving conduct issues. To ensure the training is properly integrated with device issuance, for larger scale operations, the Bureau plans to stagger training sessions to help ensure there is the necessary support during the first day of training when census field staff receive their devices. The Bureau also plans to use different training delivery mechanisms. For example, the Bureau will use a blended training approach which includes a mix of computer-based and instructor-led classroom training. The Bureau has measures of effectiveness in its course design. The Bureau relied on an in-house training development team that worked with the data collection operations staff to develop learning objectives. We found that that the Bureau has procedures to incorporate feedback. Specifically, the Bureau incorporated lessons learned from previous census tests, such as the refinement of procedures for reassigning work in the field and emphasizing the importance of knocking on doors to find a proxy respondent during the nonresponse follow-up operation. Finally, the Bureau’s training documents contained goals for achieving results for its new training approach. Specifically, the Operational Assessment Study Plan for Recruiting, Onboarding, and Training for the 2018 End-to-End Test contained the following measures of success for training—reduce cost and increase efficiency over what was reported in 2010. Evaluation In developing its evaluation phase for training, the Bureau met five of six selected leading practices and partially met one leading practice. The evaluation phase involves assessing the extent to which training and development efforts contribute to improved performance and results. We have previously found that it is increasingly important for agencies to be able to evaluate their training and development programs, and demonstrate how these efforts help develop employees and improve the agencies’ performance (see fig. 8). Overall, we found that the Bureau has a robust evaluation plan for the 2020 Census that gathers data from multiple sources. For example, The Bureau has a plan to evaluate the effectiveness of training for the 2020 Census. Specifically, operational and assessment study plans set priorities for evaluations and cover the methods, timing, and responsibilities for data collection, including assessment questions, metrics, data sources and expected delivery dates, and division responsibilities. The Bureau has an analytical approach to assess training programs. For example, the Field Decennial Data Collection Training Branch has developed three separate training evaluation surveys which will be administered to field staff through the Learning Management System. The three evaluations provide training feedback after the completion of the online training; after the completion of the classroom training; and near the completion of the operation. According to the Bureau, these assessments will help determine the effectiveness of training. The Bureau incorporated evaluation feedback into planning and design of training. For example, the Bureau held debrief sessions with census workers during the 2018 End-to-End Test and told us they were also incorporating recommendations made by a training vendor. Feedback from the 2018 End-to-End Test is being used to inform training for the 2020 Census. The Bureau incorporates different perspectives in assessing the impact of training. Bureau officials stated that they incorporated feedback from a variety of stakeholders when evaluating the effectiveness of its training during testing, including participant debriefs and evaluations from vendors. As previously discussed, the Bureau used three different surveys at different points in time to evaluate training, and relied on debrief sessions with census managers and staff in the field. Bureau officials said they considered the training methods of another organization. For example, Bureau officials told us they used training vendors that followed requirements, including e-learning content developed by the Department of Defense. However, we found that the Bureau does not have performance goals or measures for training in its corresponding study plan for the 2020 Census. Specifically, we found that in the Detailed Operational Plan for the Field Infrastructure and Decennial Logistics Management Operations for the 2020 Census, the Bureau had planned to include the following success measures: Process Measures that indicate how well the process works, typically including measures related to completion dates, rates, and productivity rates. Cost Measures that drive the cost of the operation and comparisons of actual costs to planned budgets. Costs can include workload as well as different types of resource costs. Quality Measures, such as, the results of the operation, typically including rework rates, and error rates. However, according to Bureau officials they decided not to include the measures from the study plan for training because the study plan was intended to provide descriptive information about operations rather than evaluate them. We have previously reported that a fundamental element in an organization’s efforts to manage for results is its ability to set meaningful goals for performance and to measure progress toward those goals. Thus, without specific performance goals and measures for its new blended training approach that considers cost and benefits when compared to 2010, the Bureau will not be able to determine whether its blended training approach reduced costs or increased efficiency. Moreover, not having goals and measures in place could inhibit the Bureau’s ability to develop meaningful lessons learned from the 2020 Census. Bureau officials agreed and stated they will consider including goals and measures on cost and efficiency in its plans; however, the Bureau has not yet provided us with documentation to reflect the goals and measures it will use to evaluate training, and has no time frame for doing so. Training for in-field address canvassing operation will begin in July 2019. Having performance goals and measures will help the Bureau assess the impact of its new training approach on cost, quality, and resources expended. Conclusions Successfully carrying out the thousands of activities needed to complete an accurate, cost-effective head count on schedule is an enormous and challenging task. However, for those activities we examined, the Bureau appears to be positioned to carry them out as planned, if implemented properly. While Bureau officials acknowledged there were some early delays when regions were trying to find office space and acquire leases, they said that the deadlines for the later phases of construction allow extra time—giving them a chance to make up lost time. Regarding recruiting and hiring, the Bureau was exceeding its recruiting goals for early operations, but identified challenges in areas such as promptly completing background checks, hiring in a time of low unemployment, and overcoming language barriers. Moreover, although the Bureau has exceeded its recruiting goal for early operations, recruiting a sufficient number of job applicants for partnership specialist is a challenge. The Bureau’s continued response to and management of these challenges will be important as it begins recruiting for its peak operation efforts later this fall. The Bureau has generally followed its training plans for 2020, but has opportunities to improve its ability to evaluate training efforts. The Bureau notes that the blended training approach is intended to maximize trainee learning and on the job performance during the 2020 Census. However, 2020 Census documents do not contain performance goals or measures for determining the cost and benefits of the training when compared to 2010. Revising plans to include goals and measures will better position the Bureau to determine how its blended training approach will impact the cost, quality, and resources expended on the 2020 Census. Recommendation for Executive Action We recommend that the Secretary of Commerce direct the U.S. Census Bureau to revise plans to include goals and measures for assessing the cost and benefits of the Bureau’s new blended training approach. These measures might include, but are not limited to, measures of cost, quality, and resources associated with training when compared to 2010. (Recommendation 1) Agency Comments and Our Evaluation We provided a draft of this report to the Secretary of Commerce. In its written comments, reproduced in appendix II, the Department of Commerce agreed with our findings and recommendation and said it would develop an action plan to address our recommendation. The Census Bureau also provided technical comments, which we incorporated. We are sending copies of this report to the Secretary of Commerce, the Under Secretary of Economic Affairs, the Director of the U.S. Census Bureau, and interested congressional committees. The report also is available at no charge on GAO’s website at http://www.gao.gov. If you have any questions about this report please contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. Appendix I: Objective, Scope, and Methodology This report assesses the extent to which the Census Bureau (Bureau) is following its plans for space acquisition, recruiting and hiring, and training. For all of our objectives, we reviewed current Bureau planning documents and schedules, and interviewed Bureau officials including officials at the Bureau’s six regional offices. To assess the Bureau’s progress in opening area census offices (ACO), we obtained and reviewed current Bureau leasing agreement information and construction (meaning renovations such as new electrical layouts, heating, ventilation, and air conditioning) and deployment information. We also gathered information on the General Services Administration’s role in obtaining office space. To determine whether the Bureau is on track, we compared the current status of opening, construction, and deployment of ACOs to the Bureau’s plans, schedules, and timelines, and identified differences for follow-up with Bureau officials. We also reviewed a randomly selected nongeneralizable sample of ACO files at the Philadelphia RCC to determine whether justification was included when changes to ACO locations occurred. To determine the extent to which the Bureau is following its field hiring and recruiting strategy for the 2020 Census, we reviewed Bureau documentation regarding its strategy for recruiting and hiring temporary field staff for the 2020 Census. We also reviewed output and analysis from relevant Bureau human resources systems/databases, such as MOJO Recruit. We interviewed Bureau officials in both headquarters and the field who are knowledgeable about and responsible for recruiting and hiring temporary field staff to determine the extent to which the Bureau is meeting its recruiting and hiring goals, to describe their perspectives on any challenges facing the 2020 Census, and to understand the Bureau’s actions to mitigate any challenges. To understand changes from 2010, we compared the 2010 Census recruiting and hiring plans to those of the 2020 Census to determine differences, and interviewed Bureau officials to discuss what drove these changes. Finally, to determine the extent to which the Bureau has followed its plans for training field staff, and whether this training approach is consistent with selected leading practices, we examined relevant documents and interviewed Bureau officials to determine the Bureau’s planned approach for training, lessons learned from prior Census tests, the extent to which the Bureau is incorporating lessons learned as a result of its own testing, and what changes to training need to be made before the start of 2020 field operations. Additionally, we interviewed Bureau officials responsible for developing training curriculum to understand how training was developed (e.g. what courses to develop, challenges to using technology, etc.). We also reviewed federal guidance and our prior reports, and selected 11 leading practices from GAO’s Guide for Assessing Strategic Training and Development Efforts in the Federal Government (GAO-04-546G) as leading practices for training. Our strategic training guide identifies four phases of the training development process (planning/analysis, design/development, implementation, and evaluation). We assessed the approach against leading practices in two of these phases: design/development and evaluation. We did not assess the implementation phase because field staff training for the 2020 Census had not yet begun during our audit, and we did not assess the planning/analysis phase because practices in that phase are more applicable to agency-wide rather than program-specific training development, and focus on full-time permanent employees rather than temporary employees. Moreover, within the design/development phase and evaluation phase, we did not assess all best practices because some of those best practices were also more applicable to agency-wide rather than program-specific training development, or we had already evaluated such practices as cost. Moreover, this report primarily focuses on training for the address canvassing and nonresponse follow-up operations. We then compared the Bureau’s training approach to those leading practices and identified practices being followed and any differences. We conducted this performance audit from August 2018 to July 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Comments from the Department of Commerce Appendix III: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments Other key contributors to this report include Lisa Pearson, Assistant Director; Timothy Wexler, Analyst-in-Charge; Mark Abraham; Michael Bechetti; Jessica Blackband; James Cook; Rob Gebhart; Kerstin Hudon; Kayla Robinson; and Cynthia Saunders.