Background In the Telecommunications Act of 1996 (the 1996 Act), Congress specified that consumers in “rural, insular, and high-cost areas” should have access to telecommunication rates and services that are “reasonably comparable” to consumers in urban areas. The 1996 Act altered the federal mechanism for funding universal service by requiring telecommunications carriers and other entities providing interstate telecommunications service to contribute to the USF, unless exempted by FCC. The carriers generally pass these costs on to customers, sometimes in the form of a line item on customers’ telephone bills. USF provides financial support to carriers through four different programs, each targeting a particular group of telecommunications carriers or consumers. The high-cost program provides support to both wireline and wireless carriers that provide telecommunications services in areas that carriers would otherwise not serve and where there is no competition from other providers. These are typically rural or remote areas where the customer base is relatively small and the cost of installing infrastructure is high. The high-cost program has been the largest USF program based on disbursements and has been particularly important to rural areas. High- cost support is intended to offset the carriers’ higher costs, thereby allowing them to provide services and rates that are reasonably comparable to those that consumers in lower-cost—generally urban— areas receive. In 2009, Congress required FCC to develop a broadband plan to ensure that every American has access to broadband capability, including a detailed plan for providing this service at affordable rates. In response, an FCC task force issued the National Broadband Plan in 2010, which recommended reforming USF so it could support both telephone and broadband service. FCC’s USF Transformation Order of 2011 emerged in response to this recommendation and provided USF support to carriers for broadband capable networks. The order required carriers that receive support to meet broadband-speed and quality-deployment requirements. Through the USF Transformation Order, FCC adopted a framework to transition high-cost carriers from traditional cost-accounting support to incentive-based support mechanisms, using forward-looking broadband cost models and competitive bidding. FCC’s forward-looking cost models use historical data to project the future financial needs of carriers providing telecommunications services. According to FCC, rate-of-return carriers receive about $2.5 billion in annual support from the high-cost program to support service deployments in these carriers’ 1,078 rate-of- return service areas, which FCC refers to as “study areas.” FCC has allowed rate-of-return carriers to choose, on a voluntarily basis, one the following mechanisms to receive USF support: Traditional cost-accounting support mechanism. This method retroactively provides support to carriers for costs already incurred, based on cost studies, including financial statements these companies provide each year. At the time of our review, according to FCC officials, FCC guaranteed these companies recovery of eligible deployment costs, plus a return of 10.25 percent on regulated investment costs. According to FCC’s OIG officials we interviewed, many carriers contract with telecommunications accountants to navigate the complicated process of determining which costs are reimbursable by the high-cost program and file the associated documentation with USAC. According to FCC, as of September 2019, there were approximately 437 study areas served by rate-of- return carriers receiving support through this mechanism. Model-based support mechanism. This method is aimed at providing a level of support to carriers based on modeled forward- looking costs and revenues of an efficient carrier to serve an area with voice and broadband Internet. According to FCC officials, in developing the model, FCC: had experts peer-review the model’s methodology; demonstrated how different inputs affect model support and sought stakeholder feedback on the reasonableness of how these inputs affected support levels; publicly released the model’s methodology; and used historical deployment cost and revenue data to develop the model’s inputs and assumptions. As of September 2019, FCC officials told us that rate-of-return companies serving 641 study areas were receiving support through this mechanism (or almost 60 percent of all 1,078 rate-of-return carriers’ study areas). FCC determines overall policy and issues the regulations that govern the high-cost program, while FCC’s Wireline Competition Bureau in particular implements FCC’s policies and programs regarding rate-of-return carriers. State governments play a role in implementing the federal high- cost program, as do a not-for-profit corporation (USAC) and an association (NECA). As shown in table 1, FCC, USAC, and NECA have responsibilities for the high-cost program to ensure payments to rate-of- return carriers are made properly. FCC has the following audit and oversight procedures for the high-cost program: Carrier self-certification. Carriers submit cost and line count data directly to NECA. Carrier self-certification is the primary tool for ensuring that carriers use high-cost program support consistent with program rules. USAC uses these data to qualify carriers for the program and also to calculate the amount of support carriers are eligible to receive. Carrier audits. Audits of carriers receiving high-cost program support are the primary tool used to oversee carrier activities, and audits may be conducted by USAC, state regulators, or FCC’s OIG. USAC primarily relies on assessments from the Payment Quality Assurance Program and Beneficiary and Contributor Audit Program that occur after disbursements have been made to detect improper payments, which may include fraud. Carrier data validation process. All cost data that the carriers submit to NECA for purposes of high-cost support are subject to several electronic validations, which focus on ensuring that all required data are reported and that the data ranges are consistent with information reported in previous years. In addition, NECA compares the reported cost data with financial records supporting carriers’ audited financial statements to identify any discrepancies and to require corrections when discrepancies are discovered. Carriers’ broadband deployment verification. Since 2018, USAC has performed carrier broadband deployment verifications by obtaining broadband location data to monitor whether a carrier’s broadband deployment meets FCC requirements. Carriers receive verification reports from USAC that reflect the results of the verification process. Whistleblower process. USAC maintains a whistleblower log that is shared with FCC. Through whistleblower complaints, USAC may identify instances of potentially fraudulent activity. FCC has identified three rate-of-return carriers that received at least $34 million in improper payments from the high-cost program in prior years. Two such cases were described above. In the third case, a rate-of-return carrier self-reported to NECA and USAC what it represented to be the costs and revenues of providing its telecommunications service; as discussed previously, NECA and USAC rely upon the accuracy and completeness of the carrier’s reporting to calculate the carrier’s support. An FCC OIG investigation later revealed that the carrier had manipulated FCC’s accounting rules by including the costs of a nonregulated, commercial mobile radio service in the information it submitted to NECA, thus inflating the amount of high-cost program support the carrier received. FCC eventually determined that the carrier owed the federal government almost $7 million in support overpayments received between 2005 and 2010. A petition for reconsideration is pending. As there is a finite amount of funding for the high-cost program, compensating carriers for improper, ineligible, and inflated costs they claim means less program funds are available for deploying service to the areas the program was designed to serve. Federal internal control standards, along with GAO’s fraud risk framework, OMB guidance, and the Fraud Reduction and Data Analytics Act of 2015 have placed an increased focus on the need for federal program managers to take a strategic approach to managing improper payments and risks, including the risk of fraud. GAO’s fraud risk framework provides comprehensive guidance for conducting fraud-risk assessments and using the results to develop a robust fraud risk management strategy. This framework also describes overarching concepts and leading practices for establishing an organizational structure and culture that are conducive to fraud risk management, designing and implementing controls to prevent and detect potential fraud, and monitoring and evaluating fraud risk management activities. The leading practices in the fraud risk framework are organized into four components—commit, assess, design and implement, and evaluate and adapt—as depicted in figure 1. FCC Adopted Several Funding Reforms to Enhance Carriers’ Accountability, but Not All Reforms Are Mandatory FCC Reforms Are Intended to Improve the Accountability of Rate-of- Return Funding FCC, in various orders, has adopted several funding and other reforms specific to rate-of-return carriers. As described below, the reforms we reviewed were designed to (1) control the carrier and high-cost program expenditures, (2) incentivize efficient broadband deployment, and (3) ensure carriers’ compliance with the high-cost program. Control Carrier and High-Cost Program Expenditures Prior to the 2011 USF Transformation Order, rate-of-return carriers primarily received high-cost support based on their actual costs. Under the old rules, carriers faced no FCC-imposed limits and, according to FCC, had no incentive to be more efficient. FCC adopted the reforms described in figure 2 to control the program’s expenditures. As shown in figure 2 above, FCC’s reform effort related to eliminating support to areas with competition has been ongoing since 2011. According to FCC officials, FCC relied on its broadband deployment data to identify competitively served areas, but we have previously reported that FCC’s broadband deployment data are not always accurate. In August 2017, FCC initiated a proceeding to review the Form 477—the principal tool FCC uses to gather data on communications services, including broadband services—to help inform its policy making. According to FCC, a goal of this proceeding was to enable FCC to collect better and more accurate information on the Form 477. In August 2019, FCC adopted an order based on the proceeding that, among other things, established requirements for collecting geospatial broadband-coverage maps from internet service providers. According to the order, FCC will require the service providers to submit granular maps of the areas where they have broadband-capable networks; FCC intends that these broadband-deployment maps will enable FCC to precisely target scarce universal service dollars to where broadband service is lacking. Incentivize Efficient Broadband Deployment According to FCC, one of the USF’s core principles since 2011 has been to ensure that support is provided in the most efficient manner possible, recognizing that ultimately American consumers contribute to programs like the high-cost program. FCC adopted the reforms described in figure 3 to advance its long-standing objective of adopting incentive-based policies to spur additional broadband deployment, while preserving additional funding in the high-cost program for other reforms. According to FCC, the prior cases of carriers’ abuses of USF support for unrelated purposes prompted FCC to issue more specific rules for compliance and reporting obligations. Accordingly, FCC adopted reforms described in figure 4 to improve accountability and transparency of the high-cost program. FCC’s Model-Based Support Reform May Reduce Fraud Risks, but It Is Voluntary and Not All Carriers Received Model- Based Support Of the reforms we reviewed, one reform in particular—the development of a model-based support mechanism—shows promise in reducing fraud risk, according to stakeholders from federal and state government, industry, and accounting firms we contacted. Stakeholders said the model-based support mechanism is less prone to fraud risks and is a more efficient support mechanism than traditional cost-accounting support. In particular, unlike the traditional cost-accounting-support mechanism, model-based support does not rely on carrier-submitted data to determine support amounts. Instead, the model uses, among other things, a combination of historical cost data and other data, such as expected customer revenue, to determine support amounts. Since there are no data provided by carriers in the process of determining support amounts, there is no means by which carriers can provide falsified information to fraudulently receive excess support. The carriers involved in the previously described improper payments cases were receiving support from the traditional cost-accounting support mechanism. On the other hand, stakeholders told us FCC’s traditional cost-accounting support mechanism is complex and difficult to audit, and that such weaknesses make it prone to fraud risks. For example, USAC officials told us it is time consuming to detect inflated costs associated with carriers’ affiliate company transactions. The traditional cost-accounting support mechanism also requires that carriers separate costs based upon the type of service with which the cost was associated. According to FCC’s OIG officials and representatives from accounting firms we contacted, determining whether a carrier has overly attributed costs to eligible services is difficult. For instance, determining if labor costs are properly being allocated between eligible and ineligible services requires looking at each employee’s timesheet. According to USAC, it also faces challenges auditing traditional cost-accounting support payments due to limited expertise and capacity to address the complexity of the audits. USAC officials noted that this issue has been exacerbated by audit staff turnover. According to USAC officials and some stakeholders we contacted, auditing carriers receiving traditional cost-accounting support is also difficult due to the extensive documentation requirements for this type of support, requirements that often entails hundreds of pages of financial information per carrier. USAC officials told us that a single audit can take over 1,000 hours to complete, and USAC officials told us they only completed 10 audits of carriers that received support on a traditional cost-accounting basis in fiscal year 2018. As previously noted, FCC allows carriers to choose which funding mechanism is best suited for their company. FCC officials told us they developed the model-based funding mechanism in consultation with carriers and industry stakeholders. However, according to FCC officials, the model’s use is not mandatory because some carriers do not believe that the model would accurately reflect their specific costs. FCC officials said the agency does not have plans to assess the accuracy of the model’s cost estimates or require carriers to receive model-based support. FCC officials told us they did not have plans to assess the model. FCC officials told us they had not planned to do so because in May 2019 FCC had just made available model-based support to the remaining legacy carriers, and FCC was still in the process of evaluating next steps. Planning for and conducting such an assessment would enable FCC to demonstrate the validity of the model and its reliability in accounting for the costs of broadband deployment. Federal internal- control standards state that management should use quality information to make informed decisions and evaluate program performance in achieving key objectives. Furthermore, according to FCC’s strategic plan, FCC must ensure its USF programs, including those for the high- cost program, are well managed, efficient, and fiscally responsible, and the National Broadband Plan says that FCC should move rate-of-return carriers to incentive-based regulation mechanisms, such as model-based support. Yet because a substantial number of rate-of-return study areas— 437—continue to receive traditional cost-accounting support, and the carriers that provide service in these areas cannot be effectively audited, significant fraud risks remain for the high-cost program. By assessing the model, FCC would have greater assurance that it is producing reliable cost estimates and be better positioned to determine whether to make its use mandatory. FCC Has Taken Steps to Manage Fraud Risks, but Its Efforts Do Not Fully Align with Leading Practices Managers of federal programs are responsible for managing fraud risks. Implementing effective fraud risk-management processes is important to help ensure that federal programs fulfill their intended purpose and funds are spent effectively. GAO’s fraud risk framework is aligned with federal internal-control standards related to assessing fraud risk. It focuses on preventive activities, which generally offer the most cost-efficient use of resources since they enable managers to avoid a costly and inefficient “pay-and-chase” model, which refers to the practice of detecting fraudulent transactions and recovering funds after fraudulent payments have been made. As discussed previously, our fraud risk framework consists of four components—commit, assess, design and implement, and evaluate and adapt—each of which includes overarching concepts and leading practices for carrying them out. We found that FCC has implemented some policies and procedures related to managing fraud risk for the high-cost program. For example, according to a memorandum of understanding between FCC and USAC, FCC requires USAC to alert, as appropriate, FCC’s OIG and Enforcement Bureau about potential instances of fraud. However, as detailed in appendix II, FCC’s efforts do not fully align with some elements of the fraud risk framework. In particular, we found deficiencies in FCC’s efforts related to the following three overarching concepts and one high-level component: creating a structure with a dedicated entity to manage fraud risk activities (overarching concept within the commit component); planning regular fraud-risk assessments tailored to the program and assessing these risks to determine the program’s fraud risk profile (two overarching concepts within the assess component); and designing and implementing an antifraud strategy for the program (the design and implement component). Creating a structure with a dedicated entity to lead fraud risk- management activities. Leading practices for managing fraud risk include demonstrating management’s commitment to combating fraud and designating an entity to design and oversee fraud risk-management activities. According to GAO’s fraud risk framework, an entity should lead these activities by serving as the repository of knowledge on fraud risks and controls, managing the fraud-risk assessment process, leading fraud- awareness activities, and coordinating antifraud initiatives. According to FCC officials, FCC has steering committees for each of the four USF programs, including the high-cost program. According to FCC officials, the steering committees allow in-depth discussions about each program, including on operational issues such as current spending levels and information technology systems, as well as improper payments and other issues. However, fraud risk is but one of many responsibilities of these steering committees, and they do not fill the role of a dedicated fraud risk- management entity, as called for by the fraud risk framework. In August 2019, FCC officially launched a Fraud Division—comprising existing FCC staff who investigate and prosecute fraud—within its Enforcement Bureau. However, FCC told us the scope of the new division’s operations is limited to investigations, so the Fraud Division does not fill the role of a dedicated fraud risk-management entity. Planning regular fraud-risk assessments tailored to the program and determining the fraud risk profile. An effective antifraud entity tailors the approach for carrying out regular fraud-risk assessments of its programs. According to GAO’s fraud risk framework, the approach should, among other things: fully consider the specific fraud risks the agency or program faces, analyze the potential likelihood and effects of fraud schemes, and document prioritized fraud risks. According to FCC officials, FCC has annually worked with USAC high- cost program staff to identify and assess some risks facing the high-cost program, some of which are fraud risks, but has not planned regular fraud-risk assessments that are tailored to the high-cost program in accordance with GAO’s fraud risk framework. FCC officials also told us that they adopted a tool originally developed by another agency that was used to evaluate risks facing that agency’s loan and grant programs, not just fraud risks. Using that tool as a model, FCC created a risk assessment document that included fraud risk as one of nine categories of risks across the high-cost program. Based on our discussions with FCC officials, however, the document does not constitute a fraud-risk assessment that takes into account changes to the program or operating environment. Furthermore, the risk assessment document does not constitute a fully tailored risk assessment because it does not identify and assess the fraud risks stakeholders we interviewed described as inherent to the high-cost program, detailed below. Risk caused by the complexity of the high-cost program’s cost- allocation rules. Officials from three out of four accounting firms, FCC’s OIG, and a state utility commission we contacted singled out the specific fraud risk caused by what they described as confusing and subjective rules governing the process carriers use to separate eligible and ineligible costs. Risks related to oversight challenges. Stakeholders identified several oversight challenges as significant in that they could contribute to fraud risks for the program, such as: financial mismanagement within carriers that allowed companies to submit potentially fraudulent information to USAC and NECA, and that a telecommunications accountant told us contributed to previous instances of alleged fraud; USAC’s audit personnel challenges that were due to attrition and limited resources and expertise and that were identified by officials from FCC, USAC, FCC’s OIG, and an accounting firm; and deficiencies identified by FCC’s OIG in NECA’s internal controls over payments to carriers, data validation, and the appropriateness of NECA’s role validating carriers’ cost information. In addition, FCC’s OIG officials told us of oversight challenges related to carriers’ reporting, including that it is difficult for USAC to detect when carriers improperly report rates billed for services provided by an affiliate of the company or report incorrect labor rates. Furthermore, we found FCC had not identified and assessed risks to determine the fraud risk profile for the high-cost program, as called for in the fraud risk framework. A fraud risk profile is the summation of key findings and conclusions from a fraud-risk assessment, including the analysis of the types of internal and external fraud risks, their perceived likelihood and effects, managers’ risk tolerance, and the prioritization of risks. FCC officials said they consider the risk of fraud to be low in the high-cost program, and FCC includes fraud risk in its current risk assessment process. Since FCC believes the fraud risk is low for the high-cost program, FCC has not deemed it necessary to conduct a separate fraud-risk assessment of the program. For example, FCC provided us with documentation related to its Enterprise Risk Management activities that identifies risks USAC faces to achieving its corporate objectives. However, while FCC considers fraud risks as part of these activities, the document does not specify the fraud risk tolerance for the program or constitute a fraud risk profile. Without conducting regular fraud-risk assessments to gauge the likelihood and effects of the inherent fraud risks described above, and potentially others, FCC cannot determine or document the program’s fraud risk profile. Furthermore, FCC has no assurance that it has fully considered important fraud risks, determined its tolerance for risks that could be lower priorities, or made sound decisions on how to allocate resources to respond to fraud risks. Not doing so could result in FCC compensating carriers for improper, ineligible, and inflated costs, such as in the previously discussed cases of identified fraud. By regularly assessing fraud risks to determine a fraud risk profile, FCC could better determine the extent to which it has designed and implemented adequate fraud-prevention controls. Designing and implementing an antifraud strategy for the program. Managers who effectively manage fraud risk develop and document an antifraud strategy that describes the program’s activities for preventing, detecting, and responding to the fraud risks identified during the fraud-risk assessment. FCC and USAC have established mechanisms to enhance the oversight of USF programs, mechanisms that can also help mitigate fraud risks for the high-cost program, including: In fiscal year 2016, USAC implemented a risk-based selection method for conducting audits to identify the entities with the greatest risk. USAC forwards potential fraud, waste, and abuse cases to FCC. FCC’s OIG established a hotline that can be used to report potential fraud, and USAC established a Whistleblower Alert mechanism to inform USAC of possible instances of universal service support being misapplied or mismanaged, or when carriers are potentially violating laws, rules, or regulations. USAC shares this information with FCC. FCC and USAC formed a working group tasked with developing a data-analytics tool designed to share USAC high-cost program data with FCC. FCC’s documentation of the tool states that once developed, the tool will help FCC’s Enforcement Bureau in its fraud detection activities across all USF programs. FCC officials described the development of the tool as technically challenging and said there is no established timeline for implementing the tool. FCC officials said that FCC has regular, informal interactions concerning fraud risk with USAC and, to a lesser degree, with NECA and that FCC has confidence that USAC’s improved audit processes are identifying issues appropriately. However, FCC has not specifically designed or implemented an overall strategy to mitigate fraud risks across the high-cost program. An FCC official said FCC believes its existing antifraud activities are adequate. The FCC official said FCC considers the risk of fraud in the high-cost program to be low because USAC audits have revealed that carriers’ financial reporting errors occur at a low rate and therefore do not indicate that a large amount of fraud is occurring. The official said FCC’s fraud risk-management practices are based on federal internal-control standards and are woven into existing FCC mechanisms. Given that FCC has not conducted a fraud-risk assessment that is tailored to the high-cost program and that the deceptive nature of fraud makes it difficult to measure in a reliable way, it is unclear how FCC officials reached the conclusion that the program’s risk of fraud is low. Also, in the absence of an antifraud strategy, FCC has little assurance that it has the specific control activities to prevent and detect high-cost program fraud and that the types of misconduct that previously occurred are not widespread. The improper payment activity discussed previously was caught after USAC provided support to these carriers, and it was not prevented or detected through any strategic fraud risk-management effort on FCC’s part. Furthermore, the FCC’s OIG has expressed significant concerns about such abuses by rate-of-return carriers and about the effectiveness of USAC’s auditing processes in helping prevent improper payments to these carriers. As noted above, USAC’s high-cost program audits can take over 1,000 hours and USAC faces human capital challenges that have diminished its audit capacity. In addition, while NECA’s data validations and reviews of high-cost support provide opportunities to identify input and data-reporting errors, they do not specifically address whether or not the data provided by carriers are reasonable or if the money carriers spent addresses the intended purposes of the high-cost program. Designing and implementing an antifraud strategy that conforms to leading practices would help FCC effectively manage and respond to the fraud risks identified during the fraud-risk assessments. Conclusions Given the continuing importance of deploying telecommunications services in difficult-to-serve areas, effective oversight for rate-of-return carriers is an important component for helping ensure that the high-cost program’s finite funds are used properly to meet the intent of the program. Overall, the traditional cost-accounting mechanism that FCC uses to provide support to a substantial number of rate-of-return carriers is complex, prone to fraud risks, and presents auditing challenges that FCC has not fully addressed. By following leading practices from GAO’s fraud risk framework, FCC could better ensure that it is addressing and strategically targeting the most significant fraud risks facing the high-cost program. Furthermore, FCC’s model-based support mechanism has now been in use by some rate-of-return carriers for several years and stakeholders agree that it is less prone to fraud risks. However, FCC has not assessed the extent to which the model is producing reliable cost estimates. By conducting such an assessment, FCC would have greater assurance that it is producing reliable cost estimates and be better positioned to determine whether to make its use mandatory. Recommendations for Executive Action We are making the following five recommendations to FCC: The Chairman of FCC should ensure that FCC’s Office of Managing Director follows the leading practices in GAO’s fraud risk framework related to a dedicated entity’s management of its antifraud activities, such as serving as the repository of knowledge on fraud risks and coordinating antifraud initiatives. (Recommendation 1) The Chairman of FCC should plan regular fraud-risk assessments tailored to the high-cost program and assess these risks to determine the program’s fraud risk profile, as provided in GAO’s fraud risk framework. (Recommendation 2) The Chairman of FCC should design and implement an antifraud strategy for the high-cost program with specific control activities, based upon the results of fraud-risk assessments and a corresponding fraud risk profile, as provided in GAO’s fraud risk framework. (Recommendation 3) The Chairman of FCC should assess the model-based support mechanism to determine the extent to which it produces reliable cost estimates. (Recommendation 4) The Chairman of FCC should consider whether to make use of the model-based support mechanism mandatory depending on the results of the assessment. (Recommendation 5) Agency Comments We provided a draft of this report to FCC for review and comment. In FCC’s written comments, reprinted in appendix III, FCC described actions it would take to implement our recommendations. FCC also provided technical comments, which we incorporated as appropriate. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Chairman of the FCC, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-2834 or vonaha@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. Appendix I: Key Elements of the Fraud-Risk Assessment Process for Creating a Fraud Risk Profile To help managers combat fraud and preserve integrity in government agencies and programs, we identified leading practices for managing fraud risks and organized them into a conceptual framework called the Fraud Risk Management Framework (the framework). As described in the background section of this report and depicted visually in figure 1, the framework encompasses control activities to prevent, detect, and respond to fraud, with an emphasis on prevention, to help managers achieve the objective of mitigating fraud risks. The second of four framework components—Assess—calls for specific actions managers should take to achieve the objective of mitigating fraud risks. Specifically, managers should plan regular fraud-risk assessments and assess these risks to determine a fraud risk profile. Figure 5 illustrates the key elements of the fraud-risk assessment process that lead to the creation of a program’s fraud risk profile. Appendix II: Assessment of the Federal Communications Commission’s Antifraud Efforts for the Universal Service Fund High- Cost Program We developed a data collection instrument to structure our assessment of the Federal Communications Commission’s (FCC) antifraud efforts for the high-cost program related to the commit, assess, and design and implement components of GAO’s fraud risk framework. For our assessment, we determined the extent to which FCC had implemented the leading practices within each component, as summarized in table 2. We did not assess FCC’s approach against leading practices in the “evaluate and adapt” component of the framework because we determined that FCC had not adopted fraud risk management leading practices within the first three components, and therefore it was premature for us to assess whether FCC was evaluating and adapting its use of these practices. Appendix III: Comments from the Federal Communications Commission Appendix IV: GAO Contact and Staff Acknowledgments GAO Contact: Staff Acknowledgments: In addition to the individual named above, Sally Moino (Assistant Director); Sean Standley (Analyst in Charge); Philip Farah; Camilo Flores; Mark Goldstein; Gary Guggolz; Hannah Hubbard; Josh Ormond; Ben Licht; Rebecca Shea; Andrew Stavisky; and Michelle Weathers made key contributions to this report.


Background In the U.S. commercial airline industry, passengers travel on network, low-cost, and regional airlines. With thousands of employees and hundreds of aircraft, network airlines support large, complex hub-and- spoke operations, which provide service at various fare levels to many destinations. Low-cost airlines generally operate under a low-cost business model, which typically includes providing point-to-point service using fewer types of aircraft. Regional airlines typically operate small aircraft—turboprops or regional jets with up to 100 seats—and generally provide service to smaller communities on behalf of network airlines. Airlines rely on a wide variety of IT systems to schedule and transport passengers; some of these IT systems interface with networks operated by travel-booking sites, other airlines, and the FAA. These IT systems touch all phases of a passenger’s travel experience, including booking, check-in, boarding, and baggage, as well as airline operations behind the scene, including flight planning, crew scheduling, and flight dispatch, according to FAA. In addition, aviation stakeholders explained that airline IT systems operate in a dynamic, data-intensive environment that demands around-the-clock availability and real-time information. In recent years, the introduction of new mobile applications and telecommunications infrastructure has added to the myriad systems and network connections now critical to an airline’s operations. Airlines face challenges in maintaining or enhancing their IT systems. For example, some airlines operate a web of IT systems that were developed over many years as manual systems transitioned to electronic and computer-processed functions. Replacing software and upgrading these older systems, such as reservations and crew scheduling, can be complicated undertakings as airlines serve millions of travelers and need to keep data flowing across their networks. For example, in its financial filings, Southwest pointed to the significant challenges and costs involved in introducing new IT capabilities while managing existing systems. Increasingly dependent on the use of IT systems to run its ongoing operations, the company recently completed a multi-year initiative to transition to a new third-party reservation system through Amadeus, among other investments. In addition, a wave of industry consolidation stemming from airline bankruptcies in the late 2000s has affected airline IT systems, requiring significant sustained focus among airlines on merging different IT infrastructures necessary to support worldwide flight operations without interruption. For instance, we previously found that United struggled to integrate computer and reservation systems following its merger with Continental in 2010, although the airline has subsequently completed this transition, according to airline representatives. Likewise, in 2015 American pointed to its reliance on technology when discussing principal risks posed by the integration of its computer, communications, and other technology systems with those of US Airways following the merger of the two airlines. Additionally, some airlines rely on regional partners or third-party IT providers to help manage certain IT systems, such as reservations, crew scheduling, and flight dispatch, further adding to the variety of systems that airlines depend on to run their operations. Moreover, the airline industry is going through a transformation as it shifts to digital merchandizing and retailing to better serve consumers, a process which requires access to real-time information, according to an industry stakeholder. Finally, the speed of technology evolution has accelerated, making it a constant and iterative process to keep systems refreshed and operating in sync, a situation that poses additional challenges, according to a stakeholder. Passengers may be affected by an airline IT outage in different ways depending, in part, on the type and severity of the outage—for example, whether the outage stems from a software glitch or a hardware failure— and the system affected. (See fig. 1.) Effects can range from standing in line to be checked in by a ticket agent instead of using a mobile application to delayed and canceled flights if a hardware failure forces the airline to ground all of its flights until the system is back online. System failures may have cascading effects across other airline IT systems or operations, as well. For example, an outage in a flight dispatch system could cause hours-long delays for subsequent flights. Likewise, aviation stakeholders noted that crew positioning can hinder recovery from an outage as delayed flight crews “time out,” further extending the effects of an outage. In addition to these effects, passengers and airlines can also face higher costs from delayed or canceled travel, including increased operational expenses facing airlines as crews and aircraft sit idle, as well as indirect costs, such as those faced by travelers as their itineraries are delayed or canceled. FAA and DOT Have Limited Roles in Overseeing Airline IT Systems and Addressing Effects from Outages on Passengers FAA’s Role Is Primarily Initiating Traffic Management Initiatives Requested by Airlines FAA plays a key, but limited, operational role in responding to airline IT outages. As previously noted, FAA is responsible for ensuring the safe, efficient operation of the NAS. Agency officials we interviewed emphasized that airline IT outages have a limited effect on FAA’s management of the NAS because such outages tend to affect the demand for airspace, not its capacity. As a result, FAA officials explained that if flights are delayed or canceled because of an airline IT outage, the NAS is often less congested for those that remain flying. However, in managing the air-traffic control system, FAA is responsible for initiating and administering traffic management initiatives (such as a ground stop) if requested by an airline experiencing an IT outage. For example, an airline might request that FAA initiate a ground stop if the airline is unable to report flight dispatch information to the FAA, such as the weight and balance of aircraft. FAA works with airlines to accommodate flights back into the NAS when the outage is over. Once an airline recovers from an outage, FAA may also need to initiate traffic management initiatives if demand exceeds capacity in the system— potentially causing delays both for the airline that experienced the outage, as well as others. FAA does not routinely collect data about airline IT outages—which fall outside of its management of the NAS, according to agency officials— although it does collect data on NAS operations, which could include some information about these events. Specifically: The National Traffic Management Log (NTML)—the real-time narrative log of NAS traffic management initiatives kept by air traffic controllers—includes information about ground stops or other initiatives such as time the stop was put in place, affected airports, and when the initiative was lifted. Log entries may also include additional information about the outage, if such information is provided to air traffic control by the airline experiencing it. The Operations Network (OPSNET) system, among others, collects operational data, including air traffic operations and delay data to analyze the performance of the FAA’s air traffic control facilities. However, according to agency officials, data on the effects of airline IT outages (including delay and cancellation data related to airline IT outages) are discarded because information about airline-caused flight disruptions do not provide instructive information to FAA about whether the agency is efficiently operating the NAS. FAA does not directly oversee airline IT systems related to reservations, check-in, baggage, and boarding or their use, according to agency officials. These systems are managed by the airlines themselves. For airline IT systems that interface with FAA’s operational systems, such as automated systems used in air traffic control, FAA works with airlines to ensure that any output (i.e., data feeds) interfaces correctly with the agency’s systems. FAA may provide observations to the airline if its IT systems are not providing accurate information, such as if crews are not being correctly scheduled and tracked, fuel plans are not accurate, or flight plans are not correctly calculated and observable. For Passengers, DOT Helps Ensure Compliance with Consumer Protections, Which May Be Triggered by Certain Airline IT Outages DOT’s Office of the Assistant General Counsel for Aviation Enforcement and Proceedings and its Aviation Consumer Protection Division are responsible for helping ensure airlines’ compliance with passenger protection requirements and educating passengers on their rights. Airline IT outages are not specifically addressed by any of DOT’s consumer protection regulations. Rather, when these outages occur, they may trigger broader consumer protections afforded passengers. For example, airlines are required by DOT’s interpretation of the statutory prohibition on unfair and deceptive practices to provide refunds for flights that are canceled or significantly delayed if a passenger declines any rerouting that the airline may offer. In the case of delay, however, what amounts to a significant delay is not defined in this policy, and as discussed below, individual airlines may or may not set their own thresholds. According to agency officials, DOT is currently conducting a review of air carriers’ handling of involuntary changes to passengers’ travel itineraries. DOT also regulates compliance through its tarmac delay rule, which requires airlines to mitigate or avoid consumer harm in the event of a lengthy tarmac delay. In addition to these consumer protection regulations and policies, DOT oversees airlines’ compliance with obligations included in airline contracts of carriage or customer service plans. These contracts and plans must be publicly posted by airlines on their websites. As we have previously reported, DOT helps ensure airlines’ compliance with its passenger protection requirements by educating airlines on new regulations or clarifying existing regulations, responding to airlines’ questions, and reviewing airlines’ consumer service policies. According to DOT officials, the agency encourages proactive reporting of incidents by airlines, such as airline IT outages, including a brief description of the incident and any steps taken by the airline to provide accommodation to affected consumers. DOT also receives and investigates complaints from passengers and uses complaint data to identify which airlines to inspect and whether to begin investigations that may result in fines or enforcement actions. According to agency officials, DOT received 126 complaints that explicitly mentioned a domestic airline IT outage from 2015 through 2017. These complaints involved five such outages. For comparison, in all, the agency received between 17,000 and 21,000 complaints per calendar year during that timeframe, according to DOT’s Air Travel Consumer Report. According to DOT officials, complaints that explicitly mentioned an airline IT outage largely mirror in substance those received for other causes of flight disruptions. (These complaints are discussed in more detail below.) According to DOT officials, no investigations have been carried out focusing solely on airline IT outages, but DOT investigations have included airline IT outages that contributed to violations of DOT’s consumer protection regulations. For example, DOT found that an IT outage affecting Delta’s operational systems, including gate management and flight dispatch systems, caused significant surface congestion and resulted in a violation of tarmac delay regulations. This violation was among those included in enforcement proceedings resulting in a civil penalty and consent order to the airline. Finally, to monitor airline on-time performance and baggage handling and to provide information to consumers, DOT requires certain airlines to report data to BTS monthly, including the causes of flight delays and cancellations. However, the causes are grouped into broad categories and do not specify IT outages as a cause. BTS, which is an independent statistical agency within DOT, publishes summary data from reporting air carriers on the number of domestic on-time, delayed, canceled, and diverted flights on its website. DOT’s Office of Aviation Enforcement and Proceedings also publishes a monthly Air Travel Consumer Report with this information. We discuss these data in greater detail below. Information on Airline IT Outages and Their Effects Is Limited, but Suggests That Outages Result in a Range of Passenger Inconveniences We Identified 34 IT Outages Affecting Almost Every Domestic Airline in Our Review Using a variety of information sources, we identified 34 airline IT outages from 2015 through 2017 affecting 11 of the 12 airlines in our review. No government data, academic literature, or other information source could be used to determine a comprehensive count of airline IT outages, and information is also limited regarding the types, causes, and effects of these incidents. Additionally, airlines do not regularly share detailed data about their IT outages publicly, such as the number of flights or passengers affected or the technical cause of the outage, although general information about these incidents is sometimes provided on their websites and social media accounts or to the press. To identify airline IT outages in the absence of other sources of information, we validated a preliminary list of outages developed through a review of open source information, including media coverage. This preliminary list was validated through a combination of interviews with the airlines and third-party IT providers and a review of publicly available airline information, FAA NTML log entries, and DOT consumer complaints. Through our validation process, airline representatives and others identified additional airline IT outages that had not been reported or acknowledged publicly by airlines or third-party IT providers, reflecting the variation in quantity or quality of information available regarding these events. For example, we found more information about IT outages that had nationwide or multi-day consumer or operational effects because these incidents garnered more coverage—and often an official airline response—as compared to those that were of shorter duration or affected a regional carrier or smaller number of flights, passengers, or airports. Additionally, we found less or incomplete information on outages at third- party IT providers and regional carriers because their effects were dispersed across multiple airlines. We found that the number and severity of flight disruptions associated with the airline IT outages we identified varied widely. About 85 percent (29 of 34) of our identified outages resulted in some flight disruptions, including 5 outages we identified that caused over 800 delays or cancellations. However, we were unable to verify the exact number of disrupted flights caused by each outage. At least 14 outages resulted in a ground stop, some of which lasted for several hours, according to a review of FAA’s NTML logs. We identified seven outages that had no associated flight disruptions, although they inconvenienced customers in other ways. For example, during these incidents customers experienced problems buying tickets online, checking into flights on an airline’s website, or using frequent flier benefits. Because no comprehensive data are available on airline IT outages and their related effects, we could not compare these incidents with the effects on flights caused by other disruptive events, such as severe weather like hurricanes or snowstorms. However, FAA analysis of two of the IT outages that caused over 800 flight disruptions found that the number of delays or cancellations resulting from these outages was on par with or worse than those caused by severe weather in the same months the outages occurred. Likewise, representatives from one airline stated that operational effects from airline IT outages are comparable to severe weather events, although outages occur much less frequently. An aviation industry representative noted that these events are typically unexpected, hindering the ability of airlines to react and recover. By contrast, disruptions from weather may be forecast ahead of time, allowing airlines to prepare for predicted disruptions, including accommodating customers, adjusting flight crews and schedules, and pre-positioning aircraft, according to the same representative. The airline IT outages we identified were caused by a range of IT and infrastructure issues, according to airline representatives we interviewed and official press statements. These issues included hardware failures, software outages or slowdowns, power or telecommunications failures, and network connectivity issues, among others. In several instances, an IT issue in one airline system had cascading effects across other systems not affected by the initial outage. For example, a large volume of online traffic shut down an airline’s website and subsequently disrupted the airline’s reservations and check-in systems. Representatives from six airlines, an IT expert, and four other aviation industry stakeholders pointed to a variety of factors that could contribute to an outage or magnify the effect of an IT disruption. These factors ranged from underinvestment in IT systems after years of poor airline profitability, increasing requirements on aging systems or systems not designed to work together, and the introduction of new customer-oriented platforms and services. Representatives from airlines we interviewed also described some of their IT system investments and risk mitigation efforts undertaken in response to an outage or to address potential disruptions, such as investing in new backup systems or technologies. For example, five airlines have sought to reduce vulnerability by expanding IT operations beyond a single data center or moving them to the cloud, which allows for the delivery of computing services through the Internet. Likewise, two airlines described efforts to ensure connectivity and reduce the effects of IT disruptions by using multiple telecommunications network providers. Several airline representatives and an IT expert said that these airline IT investments are aimed at enhancing overall system functionality as well as revenue. However, the IT risk expert we spoke with noted that carrying out major upgrades to their IT systems can be challenging because these systems are always in use. Additionally, according to stakeholders we interviewed, airlines employ a variety of contingency planning and recovery strategies to respond to unforeseen technical issues, including IT outages. For example, one airline described incorporating routine system testing, artificial intelligence, and outage drills into planning for system disruptions to avoid outages or speed recovery. Airline efforts to increase the resiliency of their IT systems, such as those described above, could prevent or lessen the impact of such outages. BTS Data Broadly Capture Flight Delays and Cancellations BTS data capture the causes of flight delays and cancellations in several broad categories, which do not isolate flight disruptions resulting from airline IT outages and do not reflect the root cause of flight disruptions. As previously mentioned, BTS collects on-time performance data from the airlines, including the causes of flight delays and cancellations. On a monthly basis, certain airlines are required to report at least one cause of delay (in minutes) for each flight delayed 15 minutes or more from the following five categories: air carrier, extreme weather, NAS, security, and late arriving aircraft. Similarly, for each flight that was canceled, airlines are required to report the cause from one of four categories: air carrier, extreme weather, NAS, and security. BTS guidance instructs airlines to report flight delays that are within the control of the airlines in the air- carrier category. Also included in the air-carrier category, according to the guidance, are more than 40 other potential causes of delays or cancellations, such as aircraft maintenance, baggage, terminal operations, and crew matters. As a result, flight disruptions from IT outages are indistinguishable from other airline-caused issues within this category. Additionally, delays caused by airline IT outages may be captured in a category other than air carrier because of how airlines can report the causes of flight delays based on BTS guidance. For example: Multiple causes for a delay. Airlines have the option to report either just the main cause or all the causes for a flight delay as long as the airline consistently applies the same method in its monthly report to BTS. Also, if there is more than one cause for a flight delay that starts at the same time, airlines are required to report the cause that lasted the longest. As a result, delays caused by an airline IT outage may be attributed to other categories if they happen at the same time as other issues affecting an airline’s operations, such as poor weather or airport conditions. Late arriving aircraft delays. Airlines can report a flight delay in the late arriving aircraft category if the previous flight arrived late and caused the next flight (on the same aircraft) to depart late. Airlines are not required to provide additional information on the cause of the delay for the previous flight (air carrier, NAS, security, or extreme weather). As a result, delays from incidents that can cause ripple effects on an airline’s operations, such as an IT outage or severe thunderstorms, may be attributed to the late arriving aircraft category. NAS delays. Airlines can report delays in the control of the FAA, airport operators, or state and local officials in the NAS category, which includes ground stops, flight volume delays, and air traffic control issues, among others. However, BTS guidance does not specify how airlines should report delays caused by ground stops requested by the airlines, including after an IT outage. As a result, these delays may be captured in the NAS category. BTS data are collected to provide general information on the quality of airline performance to consumers and to improve airline scheduling, rather than detailed information about specific flights or events. Consequently, these data provide limited insight into the effects of individual events, including airline IT outages, both because flight disruptions may be captured in more than one category and because the data do not allow for the isolation of effects for affected flights. We reviewed BTS data for most of the airline IT outages we identified and found, for example, that for 3 outages, airlines reported the largest total number of flight delays in the NAS causal category on the day that the airline requested a ground stop because of the outage—rather than in the air-carrier category. In addition, we reviewed BTS data for the 5 outages we identified where the airline involved delayed or canceled at least 800 total flights and found that airlines spread the causes of flight delays and cancellations across several categories, primarily air carrier, late arriving aircraft, and NAS for the first day of these outages. For example, we found that airlines attributed 44 percent of all reported flight delays to late arriving aircraft for these days. (See fig. 4). DOT officials did not see a need for additional reporting requirements on flight delays and cancellations caused by airline IT outages given the effects of such events are not unique when compared to other causes of flight delay and because these incidents involve a small portion of consumer complaints received by DOT. Aviation stakeholders we spoke to told us that airlines track flight disruptions for internal purposes such as managing operations and scheduling. For example, representatives from one airline said that the airline tracks delays and cancellations associated with IT outages and other issues internally to identify patterns and reoccurring issues that need improvement, such as scheduling, staffing, and maintenance. DOT officials noted that obtaining more detailed information on the causes of flight delays and cancellations would require a cost and benefit analysis to determine whether the benefit from collecting the data would exceed the airlines’ cost to report the data. Officials also noted that the agency has undertaken efforts to provide additional information to consumers. Notably, to provide more insight into the underlying causes of delay attributed to late arriving aircraft, BTS began calculating the original causes of delays in the late arriving aircraft category and providing these data on its website in response to a recommendation made by the DOT Inspector General in 2013. Information on the Effects on Passengers Is Largely Anecdotal and Illustrates Varied Passenger Experiences No data are publicly available to quantify with any degree of precision the number of passengers affected by airline IT outages, and only one airline provided this type of information to us. Airline contracts of carriage set the minimum accommodations passengers are entitled to when their flights are delayed or canceled, which could include refunds, rebooking, or other amenities, such as food or meals. However, there is no comprehensive information about the accommodations that were actually received by passengers, and available information is largely anecdotal. Even with respect to the same IT outage, different people may be affected differently. For example, passengers may be affected by the complexity of the NAS and their individual circumstances. According to an airline representative we spoke with, an airline may be able to quickly rebook affected passengers on a different airline for one destination, for example, but may have difficulty rebooking passengers for another destination if other flights are full. Further, while network airlines have hub-and-spoke networks that include a number of route options or frequent service between cities, others—particularly point-to-point or low- cost carriers—may have more limited service, further constraining the ability to rebook individual passengers. Finally, passengers travel for different reasons and their tolerance for disruption can differ, as well, according to DOT officials. Thus, someone flying to visit a friend may have a different tolerance for delay than someone traveling for a job interview, they noted. Airlines are required by DOT to provide refunds for canceled—and significantly delayed—flights if a passenger chooses to cancel his or her trip. Beyond these requirements, however, airlines are not obligated to provide accommodations for flight disruptions such as cancellations and delays unless specified in an airline’s contract of carriage, according to DOT. These contracts govern what, if anything, a passenger is entitled to, although airlines may offer additional accommodations to inconvenienced passengers. Generally accommodations received by inconvenienced passengers could include rebooking on the same airline or alternate travel; refunds or compensation in the form of money or other benefits (e.g., credit for later travel); and amenities such as hotel stays and food, according to their contracts of carriage. Airlines can—and in some cases do—go above and beyond the obligations set forth in their contracts of carriage, as illustrated by some examples below. Accommodations Included in Airlines’ Contracts of Carriage Vary To better understand the accommodations that passengers may have received as the result of airline IT outages, we reviewed airlines’ contracts of carriage for the airlines in our scope with applicable contracts. None of these contracts addressed IT outages directly, but flight disruptions caused by outages would be covered under the broader contract terms addressing cancellations and delays. We found that the contracts vary in terms of what accommodations are provided for, as well as the extent to which airlines have discretion in providing them. For example, while several airline contracts include provisions to provide hotel vouchers, transportation to the hotel, or meals, other airlines—notably several low- cost carriers—do not. Likewise, some airlines establish set time thresholds for when they are obligated to provide a certain accommodation (e.g., after a delay of at least 4 hours), while others do not. Specific accommodations we identified in our review of airline contracts of carriage are discussed below, and table 1 further details some of the variation that we found. Alternate transportation. All nine airlines in our analysis provide for rebooking on their own airline in the event of a flight delay or cancellation such as might be caused by an airline IT outage, although Frontier includes certain airports near a passenger’s original destination as acceptable alternatives in its contract of carriage. Under this exception, for example, Frontier could rebook a passenger on a flight to Tampa if he or she had originally planned to travel to Orlando, or vice versa, in the event of a flight disruption. Three of the airline contracts of carriage we reviewed provide for travel on a different airline—or the use of alternate ground transportation— typically at their discretion, and a fourth airline provides for alternate transportation if a passenger’s flight has been diverted to a different airport. Airline representatives with two low-cost carriers described their unsuccessful efforts to develop agreements with network airlines to facilitate the rebooking of passengers on another airline. Refunds for cancellations. If a flight is canceled and no alternative is available—or if available flights are not acceptable to the passenger— all nine airlines in our analysis provide for refunds, although three airlines may instead reroute passengers to nearby cities. Under their contracts of carriage, airlines typically provide refunds for the unused portion of a ticket in the event of flight disruptions. If, for example, passengers have already completed the outbound portion of a roundtrip ticket, they would receive a partial refund for the unused, return portion, rather than the entire ticket. Finally, three airlines (Hawaiian, Southwest, and United) offer passengers the option of travel credits in lieu of a refund in their contracts of carriage. Refunds for delays. The majority of airlines in our review provides refunds or flight credit for flight delays, although refunds in some cases could be contingent on the absence of an acceptable alternative, such as being rebooked on a subsequent flight or to an alternate airport. As mentioned above, DOT requires airlines to provide refunds for flights that are “significantly delayed” but does not define how long such a delay is and instead relies on a case-by-case determination. Four of the contracts we reviewed establish a specific timeframe for the delay after which a passenger is entitled to a refund, while the others do not establish such a threshold. For example, a passenger flying on Alaska Airlines could request and receive a refund for a flight disruption lasting at least 2 hours, and passengers on Delta are entitled to a refund, if requested, after a 90 minute delay. By contrast, airlines without a defined threshold for a delayed flight have discretion for when passengers would be eligible for refunds, particularly with regard to nonrefundable tickets. Hotel stay. The majority of airlines in our review provide for hotel stays in their contracts of carriage (and ground transportation to the hotel), to varying degrees, although two low-cost carriers (Frontier and Southwest) do not. The contracts of carriage for seven airlines include a hotel stay for passengers inconvenienced by flight disruptions, and of these four stipulate that passengers have to be away from home or from their points of origin or destination; five require that the flight disruption span certain hours (e.g., 10pm to 6am); and one includes credit for a long-distance phone call. Four of the contracts we reviewed include additional provisions for hotel stays (or other accommodations) to passengers with disabilities or other needs. For example, under its contract of carriage, American will provide amenities to maintain the safety and welfare of certain passengers if they are delayed (e.g., customers with disabilities, unaccompanied children, the elderly, or others with special needs or circumstances). Food. Three airlines in our review provide for meals for passengers inconvenienced by flight disruptions in their contracts of carriage. For example, JetBlue’s contract of carriage provides for meal vouchers or pizza for flight delays of 6 or more hours. In addition, airlines may deliver meals or offer other amenities to passengers waiting for delayed or canceled flights, even in the absence of the promise of food in the contract of carriage. In these cases, additional accommodations may be publicly announced on airline websites, by social media accounts, or through statements to the press, or they may be provided directly to individual flights or passengers at the airport. For example, in response to severe thunderstorms in 2017, Delta had pizza delivered to passengers waiting in airports across the Southeast. Monetary compensation or travel credit. Inconvenienced passengers are not entitled to monetary compensation in the case of a flight delay or cancellation in the United States, and none of the airlines in our review includes such compensation in their contracts of carriage. Nevertheless, two airline contracts of carriage include provisions for travel credit—above and beyond a refund—for flight disruptions. JetBlue’s contract of carriage provides for travel credit for canceled or delayed flights with several tiers, depending on the timing of the cancellation or length of the delay. For example, passengers delayed over 6 hours are entitled to $250 credit for future travel on JetBlue. Likewise, Alaska’s contract provides for a discount code for future travel (and a letter of apology) for passengers delayed longer than 2 hours. Although not included in Delta’s contract of carriage, the airline provided $200 in travel vouchers to all customers with flight disruptions lasting at least 3 hours for two of the IT outages we identified, according to airline representatives. Consumer Concerns Stemming from Airline IT Outages As mentioned above, collecting and analyzing passenger complaints is one way DOT helps ensure that an airline fulfills its obligations included in its contract of carriage and customer service plan, as well as any additional accommodations that may be publicly offered. Our review of passenger complaints filed with DOT stemming from airline IT outages found that they included complaints related to the lack of monetary compensation for delayed or canceled flights and refusals to refund other expenses, such as rental cars or missed hotel or cruise reservations, among other concerns. For example, complaints related to a Southwest outage in 2016 included several related to lack of compensation or other amenities, such as food or hotel stays offered by the airline. As noted above, Southwest’s contract of carriage does not provide for compensation, food, or hotel stays in the event of a delay or cancellation. Complaints filed after the Delta outage of 2016 acknowledged receipt of a $200 travel voucher in compensation or a hotel voucher, but pointed to other non-refunded expenses incurred or difficulties in redeeming these vouchers. The three consumer or passenger advocacy groups with whom we spoke raised several concerns with regard to passengers inconvenienced by airline IT outages. Stakeholders we spoke with responded to these concerns and addressed how airlines respond to IT outages. Passengers may not receive the same accommodations. In the absence of requirements for accommodations or compensation, passengers are dependent on whether or not the affected airline chooses to be generous, according to the consumer advocates we interviewed. They also noted that mileage plan or first class passengers may receive more accommodations than others, even when passengers are affected by the same underlying outage, as may be true in other circumstances, as well. Representatives from one airline told us that they attempt to promptly address the needs of all of their passengers but acknowledged that accommodations may vary depending on passenger circumstances, including passenger status (e.g., frequent-flyer program members or VIP travelers). Airline obligations toward affected passengers may be confusing for passengers. According to consumer advocates we spoke with, even if a passenger understands that an airline’s contract of carriage lays out its obligations to passengers affected by an IT outage, these contracts are often lengthy and difficult to understand. As noted above, our review of DOT complaints stemming from airline IT outages found that many passengers expected to receive compensation or other accommodations in response to these events, although such accommodations were not included in contracts of carriage. We reported in 2017 that airlines committed to reviewing their contracts of carriage to see if they could be simplified. Contracts of carriage may not clearly exclude IT outages from force majeure events, according to consumer advocates. Flight disruptions caused by extreme weather, terrorism, and other events that are seen as being beyond the control of the airline are typically treated as special situations in airline contracts of carriage, and as a result, inconvenienced passengers may not receive accommodations that they otherwise might. Consumer advocates voiced concerns that airline IT outages might be treated as events outside the airline’s control (i.e., Acts of God or force majeure events) given ambiguity in how these exceptions are defined. We found that IT outages were not explicitly included among the force majeure events identified in the contracts of carriage we reviewed. In interviews and written statements, representatives with four of the airlines in our review varied in the extent to which they characterized airline IT outages as incidents in the control of the airline, but generally indicated that passengers would be accommodated as if the outages were. Agency Comments We provided the Department of Transportation (DOT) with a draft of this report for review and comment. DOT responded by email and provided technical clarifications, which we incorporated into the report as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of the Department of Transportation, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-2834 or KrauseH@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. Appendix I: Objectives, Scope, and Methodology Our objectives for this report were to: identify (1) the Department of Transportation’s (DOT) and Federal Aviation Administration’s (FAA) roles, if any, in relation to airline IT outages and their effects and (2) what is known about these outages, including the number of flights and passengers affected. The scope of this report focuses on those airline IT systems that affect passenger experiences, including systems related to reservations and check-in, as well as those used by airlines for flight planning and dispatch. Our scope excluded IT systems involved in avionics (such as aircraft navigation systems); in-flight operations (such as passenger WiFi networks); and internal operations (such as company email systems). Our analysis included the 12 airlines that were required to report on-time performance information to DOT’s Bureau of Transportation Statistics (BTS) from 2015 through 2017, including network carriers (Alaska, American, Delta, and United); low-cost carriers (Frontier, JetBlue, Spirit, Southwest, and Virgin America); regional carriers that provide service for partner airlines (ExpressJet and SkyWest); and Hawaiian, which provides a niche service. Given the role of third-party IT providers, we also included Amadeus and Sabre in our scope. To identify relevant DOT and FAA authorities and responsibilities vis-à-vis airline IT outages in several areas, including operations, oversight, and data-collection, we reviewed relevant laws, regulations, policies, and guidance, as well as prior GAO work addressing agency roles. We interviewed DOT officials with BTS, which collects data on airline on-time performance, and the Office of the Assistant General Counsel for Aviation Enforcement and Proceedings and its Aviation Consumer Protection Division, which oversee consumer protections and receive consumer complaints. We also interviewed FAA officials with the Office of the Chief Information Security Officer, which advises the agency on matters relating to IT management and security. Within FAA’s Air Traffic Organization, we interviewed officials with Systems Operations Services, which administers traffic management initiatives including ground stops, and its National Airspace System (NAS) Operations and Office of Performance Analysis. These two offices are responsible for programs related to air traffic control systems and assessing the performance of the NAS, respectively. Through our review of relevant plans and an interview with officials in DOT’s Office of the Secretary, we determined that airline IT systems are not included in federal plans for critical infrastructure protection. According to DOT officials, outages in these systems do not have the potential to reach established thresholds for potential casualties or damages. By contrast, air traffic control systems and airports are included in sector-specific plans addressing critical infrastructure protection in the case of a terrorist attack or other natural or manmade disaster. To determine what is known about airline IT outages, we reviewed DOT data sources, including BTS and FAA performance and operations data, as well as passenger complaints received by DOT in response to airline IT outages from 2015 through August 2018. We also conducted interviews with or received written responses from 11 (of 12) airlines in our scope, and interviewed other stakeholders, including third-party IT system providers Amadeus and Sabre; an IT risk expert (Robert Charette); industry associations, including Airlines for American (A4A), the Regional Airline Association (RAA), and Airports Council International (ACI); and employee union representatives with the Air Line Pilots Association (ALPA). We determined that DOT and FAA data were not designed, and could not be used, to comprehensively identify airline IT outages. To identify airline IT outages in the absence of detailed DOT or FAA data, academic literature, or internal (proprietary) airline data on these incidents, we validated a preliminary list of such outages developed using open source material that included media coverage and publicly available airline sources for outages from 2015 through 2017. Specifically, we searched GAO subscription databases (e.g., ProQuest, Nexis, and EBSCO) to create a preliminary list of 37 airline IT outages from media coverage; performed additional searches of articles and official airline websites to collect more information on and corroborate incidents identified; provided our list of identified IT outages to the 12 airlines in our scope and two third-party IT providers (Amadeus and Sabre) for confirmation; and corroborated 20 of the identified IT outages with FAA’s National Traffic Management Log’s (NTML) log entries and DOT’s consumer complaint data. Through this process, we were able to corroborate 34 airline IT outages from 2015 through 2017, and we are confident that our list of outages includes all of the outages large enough to garner national-level, multi- day media coverage and an official response from an airline executive. While accurate, our list is not comprehensive because three airlines and a third-party IT provider identified additional outages that we did not find in our preliminary search, including one airline that shared information on more than 20 additional outages. We did not include these additional outages in our count to ensure that our methodology was consistent. To account for outages that may have occurred subsequent to our review, we identified an online listing of airline IT outages and validated 9 of 11 of the outages included from 2018 through January 2019 using publicly available airline or airport information or coverage in at least 3 media sources. This list and our validation process provides evidence that airline IT outages continued to occur during this timeframe, but does not match the rigor applied to the identification of outages we identified from 2015 through 2017. As a result, we are not confident that this list identified all of the outages large enough to garner national-level, multi- day media coverage and an official response from an airline executive. Once we had identified airline IT outages through other sources and could look at data for specific dates, we were able to use DOT and FAA data to provide additional insight into flight disruptions (i.e., flight delays or cancellations) and ground stops caused by outages. For example, we requested that FAA conduct analysis on 3 of the 34 outages we had identified to determine what FAA operational data could reveal about the effects of these outages. We selected these 3 outages to reflect a range of flight disruptions for comparative analysis, including variations in size and cause of the outage. We also assessed the extent to which the effects on passengers could be seen in the BTS on-time performance data reported by airlines. For these data, we sought to determine the cause and magnitude of delays and cancellations for each outage. We also reviewed NTML log entries for the dates of known outages to further identify potential information, including incidents of ground stops. Finally, to obtain more information about the potential effects on passengers resulting from these events, we reviewed consumer complaints to DOT stemming from airline IT outages. These complaints were provided to us by DOT’s Aviation Consumer Protection Division and include reference to the associated outage. To understand how airlines accommodate inconvenienced passengers, we reviewed airline contracts of carriage for 9 of the 12 the airlines in our scope. These contracts are the legally binding contracts between carriers and passengers and may include specific provisions such as refund procedures and responsibility for delayed flights, among other things. We excluded two regional airlines (ExpressJet and SkyWest) that operate under the contracts of carriage of their mainline partners and Virgin America, which merged with Alaska in 2018 and no longer has a separate contract of carriage. In addition to the stakeholders mentioned above, we also interviewed consumer or passenger advocacy groups, including representatives with the Consumers Union, the National Consumers League, and Travelers United to identify any concerns regarding consumers affected by airline IT outages. We conducted this performance audit from February 2018 to June 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: GAO Contact and Staff Acknowledgments GAO Contact Heather Krause, (202) 512-2834 or KrauseH@gao.gov. Staff Acknowledgments In addition to the individual named above, other key contributors to this report were Jonathan Carver, Assistant Director; Molly Laster, Analyst-in- Charge; Neha Bhatt; David Hooper; Rich Hung; Delwen Jones; SaraAnn Moessbauer; Emily Mussey; Josh Ormond; Corinne Quinones; Pamela Snedden; James Sweetman, Jr.; and Elizabeth Wood.