Background DOD began the F-35 development program in October 2001 with plans to produce next-generation aircraft to replace aging aircraft in the military services’ inventories. Figure 1 shows the F-35 in flight. The program has developed and is delivering three variants of the F-35 aircraft; the F-35A conventional takeoff and landing variant for the Air Force, the F-35B short takeoff and vertical landing variant for the Marine Corps, and the F-35C carrier-suitable variant for the Navy. The characteristics of the services’ variants are similar, but each service’s variant also has unique operating requirements. For example, the Marine Corps requires that the F-35B be capable of operating from aircraft carriers, amphibious ships, and main and austere operating bases alike, requiring the ability to conduct short take offs and vertical landings. In March 2005, we found that the F-35 program had started development without adequate knowledge of the aircraft’s critical technologies or a solid design. Further, DOD’s acquisition strategy called for high levels of concurrency between development and production, which runs counter to best practices for major defense acquisition programs. In our prior work, we identified the F-35 program’s lack of adequate knowledge and high levels of concurrency as the major drivers of the program’s significant cost and schedule growth, and other performance shortfalls. Since the development program began in 2001, it has been restructured three times with revised cost and schedule estimates. The most recent restructuring was initiated in 2010 when the program’s cost estimates exceeded certain thresholds established by statute—a condition known as a critical Nunn-McCurdy breach. DOD subsequently certified to Congress in June 2010 that the program was essential to national security and needed to continue. DOD then established a new acquisition program baseline in 2012 that added $162.7 billion to the program’s cost estimate and extended the original delivery schedule by 5-6 years. Since then, the program’s cost and schedule estimates, as well as the expected number of aircraft to be delivered, have remained relatively stable, as shown in table 1. Of the F-35’s $406 billion estimated acquisition cost, DOD needs a majority of the funding ($270.3 billion) to purchase aircraft over the next 26 years. Of that future funding, the program plans to spend between $9.6 billion and $14 billion each year through fiscal year 2031. In addition, the program’s sustainment costs to operate and maintain the F-35 fleet over the next 52 years are estimated to be $1.12 trillion. Though the program’s total planned quantities have been relatively stable, the program’s timeframes for procuring these aircraft have changed multiple times. Since the start of development, the program has pushed the procurement of more than half of the total aircraft planned into the future, mostly due to significant concurrency between development and production. Specifically, the program office had originally planned to procure almost 2,000 aircraft by fiscal year 2019. However, according to the current plan, by the end of 2019, the program will have procured just over 500 aircraft. The F-35 baseline aircraft development program was complete in April 2018, when developmental testing concluded. As we reported in June 2018, the program office reported it had met all nine of its capability thresholds—or the minimum acceptable value for each capability—and delivered three of those nine capabilities. However, we also reported that the program has to complete operational testing before DOD can determine if the six remaining capabilities have been delivered. The purpose of operational testing is to assess the effectiveness, suitability, survivability, lethality, and mission capability of the F-35, including the information systems and the air vehicle, in an operationally representative environment. Operational testing includes cyber security assessments, some of which have been conducted. Production of the F-35 began in 2007 while development was in its early stages and before developmental flight testing had started. As a result of this concurrent development, the 357 aircraft delivered through 2018 will need retrofits to fix deficiencies and design issues found during testing. The program’s total estimated cost of concurrency is $1.4 billion. The program office plans for over 500 aircraft to be procured by the time operational testing is completed. Until operational testing is complete, there is a risk that additional problems with the aircraft may be identified. As a result, the concurrency costs of retrofitting delivered aircraft could increase. Operational Testing Has Started The F-35 program started formal operational testing in December 2018 after a 3-month delay. This testing was delayed for two main reasons: (1) to resolve critical deficiencies and (2) to accommodate an unexpected grounding following the crash of an F-35B in September 2018. According to a test official, the program expects to complete testing in December 2019, about three months later than planned due to delays with the simulator that is used for more complex testing. Figure 2 shows the program’s planned end to developmental testing and planned timeframes for operational testing for 2012 and the past four years and the delays the program has realized each year since the program was re-baselined in 2012. The operational testing was delayed for the following two main reasons. Resolution of deficiencies: First, before the program could begin operational testing, it had to resolve critical deficiencies with the aircraft that were identified during development testing. The program categorizes deficiencies according to their potential impact on the aircraft’s performance. Category 1 deficiencies are considered critical and could jeopardize safety, security, or another requirement. Category 2 deficiencies are those that could impede or constrain successful mission accomplishment. In January 2018, the F-35 program had 966 open deficiencies—111 category 1 and 855 category 2. At that time, the program planned to move forward before resolving all of them. In June 2018, we recommended that the Secretary of Defense direct the F-35 program to resolve all these deficiencies before the program’s October 2019 full-rate production decision. According to DOD officials, over the past year, the program has made progress in reducing the number of open deficiencies by resolving, re-categorizing, closing, or combining them. For example, in 2018, the program resolved nearly 50 category 1 deficiencies and re- categorized over 50 others to category 2. As a result, the program received approval from the Under Secretary of Defense for Acquisition and Sustainment to begin formal operational testing with 13 category 1 deficiencies and almost 900 category 2 deficiencies. According to the Program Executive Officer, none of the open category 1 deficiencies are a safety of flight concern, and all of them have operational workarounds. A current example of an open category 1 deficiency is with lines on the F- 35’s landing gear, which can rupture when a tire blows, potentially causing loss of a major aircraft system such as the brakes. Such an event requires some repair work to the landing gear, but contractor officials explained that it is not a safety concern. According to the program office, it is not a safety concern because the current workaround for this deficiency is pilot training to avoid braking on the side of the blown tire. Program test officials said that testing with deficiencies is not uncommon and they will continue to work to address them, but some may not be fully resolved for several years. Unexpected grounding: In October 2018, the F-35 fleet was grounded after the program identified a manufacturing fault with an engine fuel tube—a component in the F-35 engine produced by Pratt & Whitney. The fault was found in an inspection that stemmed from an F-35B crash in September 2018. This was the first crash of an F-35. Of the 23 operational test aircraft, the program replaced the fuel tubes on 18 aircraft by the start of operational testing in December 2018, which contributed to the 3-month delay. This and other key technical risks are described in more detail in appendix III. In addition to starting operational testing and the unexpected grounding, the program and the airframe contractor Lockheed Martin experienced other major events over the past year, as shown in figure 3. For example, the United States completed its first F-35 combat mission in September 2018 when an F-35B successfully hit a target in Afghanistan. The program took steps to mitigate delays to the start of operational testing. For example, the program office, in coordination with DOT&E, received approval to conduct some preoperational testing events starting in January 2018, before the official start date in December. According to DOT&E officials, the outcome of these preoperational test events should count towards the completion of operational testing. This included cold weather testing in Alaska, which took advantage of appropriate weather conditions. Despite the 3-month delay, program officials stated that they consider the F-35 operational test schedule to be adequate for addressing schedule risks, which pertain to unresolved deficiencies and potential problems with the availability of test and support aircraft, ground systems, test ranges, and necessary test models and simulations. According to a test official, as of April 2019, some of these risks have been realized, such as the delay with the simulator, and as a result, the end of operational testing is now planned for December 2019. In addition, there is the possibility of new deficiencies emerging from operational testing. Unresolved deficiencies: Existing or new deficiencies could negatively affect test results. According to DOT&E officials, since the start of operational testing, four new category 1 deficiencies have been identified, bringing the total to 17. According to DOD officials, it would not be unexpected during the course of operational testing for the program to discover additional deficiencies that may require resolution and re-testing. Availability of test and support aircraft: According to test officials, F-16s and F-18s are needed to represent adversaries during F-35 operational tests. These assets may not be available because they also support other test programs. According to officials, the F-35 program does not have control over the availability of these aircraft and must work with the Navy and Air Force to negotiate their use. In addition, the limited availability of F-35 test aircraft, in part due to R&M issues and shortages of replacement parts, may also pose a challenge to completing test events, according to officials. Availability of ground systems: Ground systems required for operational testing, such as the DOT&E developed Radar Signal Emulators, are late in development and may not be available when required. According to DOT&E officials, the emulators imitate modern threat radar capabilities of adversarial nations but their integration with the test range is approximately a year behind schedule. The program is currently using other threat simulators. DOT&E officials stated that they are working to have the radars ready by the spring of 2019, when needed. Availability of test ranges: Test officials at Edwards Air Force Base expressed concern about the availability of test ranges, which the F- 35 program shares with other programs. According to test officials, the F-35 was the fifth in line, in terms of priority, to use the range at Edwards Air Force Base, as of October 2018. DOT&E officials, however, stated that they did not observe any range availability issues during the F-35’s first month of operational testing. Availability of test models and simulations: According to program officials, the program’s testing simulator, which runs the F-35’s mission systems software and provides test scenarios that cannot be replicated in a real-world environment, will not be complete until at least November 2019. Completion of the testing simulator was originally scheduled for the end of 2017. Any additional delays in operational testing could affect another upcoming program decision: DOD’s decision to begin full-rate production in December 2019. This decision is typically made after operational testing is completed. The F-35 Program Is Still Not Meeting All Reliability and Maintainability Targets The F-35 program has made slow, sustained progress in improving the F- 35’s R&M. R&M determines the likelihood that the aircraft will be in maintenance rather than available for operations. Each F-35 aircraft variant is measured against eight R&M metrics, four of which are in part of the contract. All F-35 variants are generally performing near or above targets for half of the R&M metrics while the other four are still falling short, which is the same as last year. While the program is on track to meet the targets for half of the metrics, the program has not taken adequate steps to ensure the targets for the others will be met. While DOD has an action plan to improve R&M, its guidance does not define specific, measurable objectives for what the desired goals for the F-35’s R&M performance should be. Furthermore, the program office has not prioritized funding for projects that will improve the R&M metrics that are not meeting their targets. The F-35 Program Is Meeting, or Close to Meeting, Half of Its Targets All F-35 variants are measured against eight R&M metrics’ targeted performance levels, and all variants are generally performing near or above targets for four of the eight R&M metrics. This represents little change from their overall performance last year. All eight R&M metrics are described in the program’s Operational Requirements Document (ORD)—the document that outlines the requirements DOD and the military services agreed the F-35 should meet. However, in December 2018, DOT&E reported that, although performance for the four under- performing metrics has shown slow growth over the years, none of these metrics were meeting interim goals needed to reach requirements at each variant’s maturity. Each F-35 variants’ R&M performance against these metrics’ targets is shown in table 2. Since the program began tracking R&M performance in 2009, the program has seen small, annual improvements. Over the past year, all variants showed a slight improvement in targeted performance levels for one metric, the mean flight hours between failures (design controlled), but saw little or no discernable improvement for the four metrics not meeting targets. However, based on current performance, the program does not expect to meet those targets by full aircraft maturity. According to F-35 program officials, the ORD R&M metrics should be re-evaluated to determine more realistic R&M performance metrics, but they have not yet taken actions to do so. Until it does so, the program office remains accountable for ensuring those ORD R&M metrics are achieved. In June 2018, we recommended that the F-35 program identify what steps it needs to take to ensure the F-35 aircraft meet R&M requirements before each variant reaches maturity and update its R&M Improvement Program (RMIP)—DOD’s action plan for prioritizing and funding R&M improvement projects—with these steps. DOD concurred with our recommendation but has yet to take substantive actions to address it. It did, however, complete 16 improvement projects since we last reported on this. Despite completing these projects, there were not significant gains in the R&M metrics not meeting targets. Program officials advised, however, that measurable improvements in R&M can take time to manifest. To speed this process, the program is accelerating planned upgrades to older aircraft where appropriate, which officials stated should translate to an overall improvement in the program’s R&M performance. The F-35 Program Office’s Improvement Plan Does Not Address Under- Performing Metrics The F-35 program office has estimated that implementing all of the identified improvement projects currently contained in its RMIP could result in potential life cycle cost savings of over $9.2 billion by improving the F-35’s R&M. As of December 2018, the guidance the F-35 program office has used to implement the RMIP does not define specific, measurable objectives for what the desired goals for the F-35’s R&M performance should be or align improvement projects with R&M goals. Furthermore, the RMIP has not been a funding priority. Federal internal control standards state that programs should define objectives when implementing programs such as the RMIP. Although the F-35 program RMIP’s guidance has a general goal of improving R&M, it does not identify achieving the targets for the eight R&M metrics the program tracks as an objective. Program officials acknowledged that the RMIP’s guidance does not include such an objective. Instead, officials are using the RMIP to prioritize and fund projects that will improve aircraft availability and mission capability—neither of which are included in the eight R&M metrics, but are necessary and important initiatives. Officials stated that by prioritizing these projects, they will eventually improve performance under all R&M metrics, including the four that are not meeting targets. The RMIP’s guidance, however, does not discuss these priorities or align improvement projects with the eight R&M metrics. In our prior work on weapon system acquisitions, we have identified a number of best practices for improving program outcomes, such as clearly establishing well-defined requirements and securing stable funding that matches resources to requirements. The F-35 program office has not prioritized or dedicated funding in its budget to improve R&M in part because program officials explained that they have been focused on initiatives intended to lower the cost of the aircraft. Further, any current funding for R&M improvement projects comes from the program’s operation and maintenance funds, which are only available for one fiscal year. Officials further explained that, if such funding runs out or is used by the program for other efforts, then R&M projects will go unfunded or be suspended until new funding is available. In fiscal year 2018, for example, while some R&M improvement projects were completed, several other projects were suspended when that year’s funding ran out. According to officials, these projects may not be started back up until fiscal year 2019. In addition, most of the R&M improvement projects that were approved in fiscal year 2018 were not funded. For example, as of December 2018, according to a contractor representative, all of the identified improvement projects currently unfunded in the program’s RMIP would cost about $30 million to implement, but are on hold and waiting to be funded. Program officials stated that they are in the process of revising the RMIP and have considered including more specific objectives, such as a focus on improving aircraft availability and mission capability and a focus on improving R&M performance where the ORD R&M metrics’ targets are not being met. Additionally, in its 2019 annual lifecycle sustainment plan, the program office noted that a dedicated annual budget for R&M improvement projects would benefit the program. According to the program, any revisions to the RMIP and changes to how it will be funded, however, will not be complete until April 2019 or later. Without defining measurable objectives in its RMIP guidance for meeting all eight R&M metrics and aligning which improvement projects will ensure those metrics are met, the program is at risk of not fully meeting its R&M goals. Further, without prioritizing funding for improving R&M, projects may continue to be either prematurely suspended or never get underway. As a result, the warfighter may accept aircraft that (1) are less reliable than originally described in the program’s ORD, and (2) have operation and sustainment costs that may raise affordability questions. The F-35 Program Will Start Block 4 Modernization without a Complete Business Case With development of the baseline program complete, the program is transitioning to early development and testing for modernization efforts known as Block 4, which are expected to cost about $10.5 billion. The F- 35 program plans to award Block 4 development contracts starting in May 2019, before completing a business case—a baseline cost and schedule estimate to track the program’s performance going forward. In doing so, the program will commit resources without adequate knowledge of Block 4’s full cost, schedule, and level of technology maturity, putting Block 4 at risk of experiencing cost and schedule overruns similar to those experienced by the baseline program during its development. The F-35 Program Is Transitioning to Early Block 4 Development and Testing The National Defense Authorization Act for Fiscal Year 2017 required DOD to submit a report containing certain elements of an acquisition program baseline—in essence, a full program business case—to include the cost, schedule, and performance information for Block 4. In 2018, we found that DOD’s report to Congress was incomplete but included information on some elements of the Block 4 acquisition program baseline. In its report, DOD stated that the acquisition program baseline would continue to be refined over the next year. As a result, we presented a matter for congressional consideration to restrict Block 4 funding until the program established a complete business case. DOD’s report to Congress also outlined the F-35 program office’s new development approach to deliver Block 4 capabilities—new requirements beyond the baseline aircraft capabilities to address evolving threats. As we reported in June 2018, this new approach, meant to deliver capabilities to the warfighter faster, is referred to as Continuous Capability Development and Delivery (C2D2). This approach consists of 6-month development cycles in which small groups of capabilities will be developed, tested, and delivered as they are matured. In January 2018, the F-35 program started using this C2D2 approach to develop and test software updates to address deficiencies identified during testing. According to the contractor, the first two software updates also established a foundation for new Block 4 capabilities to be fully developed later. According to program officials, as of December 2018, the program has executed contract actions valued over $1.4 billion to establish testing facilities and support early Block 4 development of capabilities the program plans to deliver through 2024. According to DOD’s January report, results from this work will help the program inform its Block 4 business case. The F-35 Program Will Start Block 4 Development without a Full Business Case The F-35 program plans to award Block 4 development contracts without knowledge of the effort’s full cost or the maturity of critical technologies. Over the past year, the program has been working to complete its business case for Block 4, including incorporating Block 4 activities into its acquisition strategy—which was approved in October 2018. However, three key Block 4 business case documents will not be ready before the program’s planned May 2019 contract awards for development efforts. Independent technology readiness assessment: Although the contracts for Block 4 development efforts are planned to be awarded in May 2019, the program will not conduct an independent technology readiness assessment by that time. A technology readiness assessment is a systematic, evidence-based process that evaluates the maturity of hardware and software technologies critical to the performance of a larger system or the fulfillment of the key objectives of an acquisition program. According to a program official, the program will conduct its own assessments on a rolling basis as initial capabilities are developed. The official stated that technologies will not be integrated into the aircraft until they are adequately mature. The program office plans to conduct a partial assessment of initial capabilities sometime between October and December 2019 with additional assessments to follow. However, without an independent technology readiness assessment, the program has not identified potential critical technology elements and as a result, may be at risk of delaying the delivery of new capabilities. Test and evaluation master plan: Although the F-35 program has begun testing Block 4 capabilities, it does not have an approved test and evaluation master plan. The test and evaluation master plan documents the overall structure, strategy, and objectives of the test program as well as the associated resources needed for execution. It provides a framework for the program office to provide detailed test plans and subsequently determine the resources needed. Test officials have expressed concerns about the lack of an approved test plan, uncertain funding, the number of test aircraft available, and the draft test schedule, among other things. Officials were also concerned as to whether the Block 4 test aircraft would be in the same configuration as fielded aircraft, which are in earlier configurations than the test fleet. Further, DOT&E stated in its annual report that it considers the current Block 4 schedule to be high risk due to the large amount of planned capabilities that will be developed and tested in 6- month development cycles. An approved, properly resourced test plan is essential for planning and preparing for adequate testing of the Block 4 capabilities. Without an approved test and evaluation master plan, the F-35 program is providing the test authorities with capabilities to be tested without giving them the necessary direction on how to adequately prepare to conduct the tests. Specifically, test officials stated the F-35 program office has not provided details on which capabilities are planned for each testing development cycle making it difficult to execute testing. While this is still a concern, F-35 program officials explained that over the past 3 months they have been providing the test authorities with the direction needed to conduct testing. Independent cost estimate: The Block 4 independent cost estimate, which details the program’s total estimated life cycle cost, is not complete. In August 2017, we reported that DOD estimated the development funding needed for the first phase of modernization for Block 4 to be over $3.9 billion through 2022. Since then, the program incorporated more scope and fidelity into the Block 4 cost estimate, which has increased to $10.5 billion for Block 4 capabilities planned through 2024. The program office has provided its Block 4 cost estimate to the Cost Assessment and Program Evaluation office (CAPE) for an independent cost estimate. According to CAPE officials, they will provide the independent cost estimate between October and December 2019 to support the program’s full-rate production decision, but this would occur several months after the program plans to award the Block 4 development contracts. Without an independent cost estimate, Congress does not have insight into the full potential cost of the Block 4 effort. The expected completion dates for these documents are between October and December 2019, at the earliest. Figure 4 shows key Block 4 dates, the planned development contract awards, and planned completion dates for the remaining business case documents. Major defense acquisition programs generally follow DOD acquisition policy, which states that prior to the release of a development contract request for proposal, program officials should have confidence that program requirements are firm. Program officials should also clearly state that the risk of committing to development has been reduced or will be adequately reduced prior to contract award. According to best practices identified by GAO, without several of the business case documents completed, program officials cannot have a high level of confidence that the requirements are firm and that the risk to committing an estimated $10.5 billion in funding to Block 4 has been adequately reduced. According to program officials, business case documents have not been completed because they took a step back to re-examine their approach and the cost estimate for Block 4 that DOD established in 2017. Counter to acquisition best practices, the program plans to initiate additional development work before they acquire the requisite knowledge of the necessary levels of technology maturity and funding. Program officials have reported the planned modernization contracting efforts shown in table 3. If program officials move ahead with awarding Block 4 contracts without gaining the knowledge that a full business case could would provide, Block 4 modernization efforts will be at risk of experiencing the same kind of cost and schedule growth the baseline development program experienced. The F-35’s Unit Cost Has Decreased and Its Production Rate Has Increased With a few exceptions, the negotiated prices for all F-35 variants have generally been decreasing with each production lot, and more aircraft are being procured in each lot. In particular, the F-35A’s price has decreased in each subsequent production lot, with the most recent price per aircraft at $89 million in lot 11, as shown in figure 5 below. In 2018, we reported that while the F-35 program faces affordability challenges, it was investing in several projects to reduce production and sustainment costs. According to DOD, to improve production affordability, the F-35 program office is continuing to make investments to lower the price of an F-35A to below $80 million by lot 13. To realize this goal, the F-35 program office and the prime contractor are increasing the production rate and investing in various initiatives to lower production costs. For example: According to the program office, it has invested a total of $320.3 million in efforts to improve manufacturing processes that it estimates could result in up to $7.9 billion in savings over the life of the program. In addition, the prime contractor has invested $90 million and plans to invest an additional $25 million to lower its production costs. DOD issued a contract announcement for economic order quantity purchases for use in production lots 13-14. This approach involves making large purchases of components that will be used across multiple procurement lots of aircraft to reduce production costs by buying components in bulk and achieving economies of scale. The program had expected $1.2 billion in cost savings from this effort, but according to estimates from the CAPE, cost savings will more likely be $595 million. In addition, according to program officials, once the program achieves full-rate production, it plans to utilize a multi-year procurement strategy, beginning in fiscal year 2021. This strategy is intended to have similar benefits as the economic order quantity purchases by providing industry with a stable, long-term demand. According to Pratt & Whitney, the cost of the engine is also declining. For example, the price of the F-35A and C engine dropped by $100,000 per engine over the past year. The most recent negotiated price is $11.9 million per engine. The F-35 airframe and engine contractors saw a significant increase in their production rates in 2018, but faced some production challenges as well. The airframe contractor—Lockheed Martin—increased its production rate by 50 percent and delivered a total of 91 aircraft in 2018, with a total of 267 aircraft on its production floor or in contract negotiations as of December 2018, as shown in figure 6. In addition, Lockheed Martin delivered more aircraft on time. In 2012, none of the planned aircraft deliveries were on time whereas in 2018, 58 percent were on time. To incentivize the contractor to improve on-time deliveries, the program office has added a performance incentive fee to the lot 11 production contract. Table 4 shows some improvements in Lockheed Martin’s production metrics since 2012 and over the past 2 years. Between 2012 and 2017, Lockheed Martin saw some improvement for all variants’ production metrics, with the F-35A showing improvements through 2018. However, over the past year, several metrics for the F- 35Bs and F-35Cs saw a decline. According to Lockheed Martin, it faced several challenges with the increased production rate which led to these declines. For example, since January 2018, the contractor hired around 900 new personnel, nearly 30 percent of its workforce, all of whom needed training. According to officials, this influx of new personnel led to an increase in the average labor hours for the F-35C and the number of hours required for scrap, rework, and repair of the F-35B and F-35C. According to the contractor, as the newly hired personnel gain more experience in the production processes, the average labor hours it takes to build an F-35C should start decreasing again. The contractor faced several production quality issues and parts delays, which it worked to address over the past year. For instance, we reported last year that due to a fault in the production process, Lockheed Martin halted deliveries after the Air Force identified corrosion between the aircraft’s surface panels and the airframe because Lockheed Martin did not apply primer when the panels were attached. The program office stated that Lockheed Martin and the F- 35 Program Executive Officer reached a mutual agreement on the cost to resolve this issue, the details of which have not been disclosed publicly. With the production rate increase, the supply chain was strained to deliver parts on time, which led to increases in material shortages for key components, such as the radar. Pratt & Whitney has also increased production over the past year and has shown similar manufacturing performance for the F-35 engine as in past years; however, it had fewer on-time deliveries in 2018 due to the challenges it faced, including an increase in the average number of quality issues per engine. Pratt & Whitney’s production rate increased by 10 percent over the past year, with 81 engines delivered in 2018. Table 5 shows the trends in Pratt & Whitney production metrics’ performance. According to Pratt & Whitney, its late engine deliveries increased in 2018 partially due to a subcontractor that did not have all of the needed tooling in place to produce more F-35B engines. To address this and other issues causing the late deliveries, Pratt & Whitney is taking lessons learned from its other production facilities and applying them to the F-35’s engine production. Conclusions The F-35 program has overcome significant hurdles in its 18 years of development of the baseline aircraft, which was completed last year. One recent hurdle that it overcame was resolving many critical deficiencies found during developmental testing, which allowed the program to begin operational testing this past December. Other hurdles remain, including with the F-35’s reliability and maintainability (R&M). Four of the eight R&M metrics continue to fall short of meeting performance targets. Program officials stated that the Operational Requirements Document (ORD) R&M targets need to be re-evaluated to determine more realistic R&M performance metrics but have not yet taken actions to do so. Until the program re-evaluates the targets, it is accountable for achieving those requirements. Furthermore, funding improvement efforts have not been a priority for the program. As a result, over the past year, some projects were started, several were halted while underway, and others are on hold, waiting for funding. As long as targets under all of the R&M metrics continue to fall short, the U.S. military services and the taxpayer will have to settle for aircraft that are less reliable and more costly to maintain than originally planned. Also, with continuing concerns about the program’s long-term affordability, the program is missing a prime opportunity to infuse affordability into the aircraft’s future with better R&M performance. As the program is considering revisions to its R&M Improvement Program (RMIP), it is in a good position to clearly define and communicate its R&M objectives for the aircraft to meet the targets under all of its eight R&M metrics. Until it does so, the program office will not know whether the steps it is taking now are sufficient to ensure each F-35 variant achieves its R&M requirements in the future. As we have reported in the past, the F-35 program started its development before it was ready. It is now at risk of doing the same thing with the Block 4 modernization effort. Since we last reported in June 2018, the program has still not established a solid business case to commit funding and other resources to developing new capabilities for the aircraft. This could result in the program delivering technologies late and over cost estimates. Finally, the program has committed a significant amount of funding to support Block 4, but it has not completed an independent cost estimate of the life-cycle cost. Consequently, Block 4 may follow in the footsteps of the F-35’s baseline program which saw significant cost and schedule growth during its development. This approach leaves the F-35 program, DOD, Congress, and the U.S. military services without key information to make decisions regarding Block 4. Recommendations for Executive Action We are making the following five recommendations to the Department of Defense: The Secretary of Defense should ensure that the F-35 program office assesses whether the ORD R&M targets are still feasible and revise the ORD accordingly. (Recommendation 1) The Secretary of Defense should ensure that the F-35 program office, as it revises its RMIP, identifies specific and measurable R&M objectives in its RMIP guidance. (Recommendation 2) The Secretary of Defense should ensure that the F-35 program office, as it revises its RMIP, identifies and documents which RMIP projects will achieve the identified objectives of the RMIP guidance. (Recommendation 3) The Secretary of Defense should ensure that the F-35 program office prioritizes funding for the RMIP. (Recommendation 4) The Secretary of Defense should ensure that the F-35 program office completes its business case, at least for the initial Block 4 capabilities under development, before initiating additional development work, to include: an independent cost estimate; an approved test and evaluation master plan which addresses resources, aircraft shortfalls, and funding; and an independent technology readiness assessment. (Recommendation 5) Agency Comments and Our Evaluation We provided a draft of this report to DOD for review and comment. Our initial draft report contained only recommendations 2 through 5 above. During the comment period, DOD officials provided additional information about the program’s R&M performance concerning whether the ORD targets continue to be feasible and should be re-examined. As a result, we added our first recommendation above—that the F-35 program office assess whether the ORD R&M targets are still feasible and revise the ORD accordingly. DOD provided written comments on our report, which are reprinted in appendix IV. DOD concurred with our four recommendations on R&M but did not concur with our last recommendation on the Block 4 modernization. DOD also provided technical comments, which were incorporated as appropriate. In concurring with our four R&M recommendations, DOD stated that it would review its R&M requirements and possibly revise them, update its RMIP guidance, and plan for R&M funding going forward. DOD officials did not concur with our recommendation that the F-35 program office complete its business case before initiating additional development work. DOD stated that the F-35 program office has adequate cost, schedule, and technical maturity knowledge to begin the development of initial Block 4 capabilities. DOD also outlined when some of the remaining Block 4 business case documents would be complete. As we stated in our report, these documents will not be complete until after the contracts to initiate additional Block 4 development work will be awarded. We maintain that completing its business case before initiating additional development work would put DOD and the program in a better position to effectively and successfully develop Block 4 capabilities. We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, the Under Secretary of Defense for Acquisition and Sustainment, the Secretary of the Air Force, the Secretary of the Navy, and the Commandant of the Marine Corps. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V. Appendix I: Prior GAO Reports and DOD Actions Key program event Start of system development and demonstration approved. Primary GAO conclusions/recommendations Critical technologies needed for key aircraft performance elements are not mature. We recommended that the program should delay start of system development until critical technologies are matured to acceptable levels. DOD response and actions DOD did not concur with our recommendation. DOD did not delay the start of system development and demonstration stating technologies were at acceptable maturity levels and that it will manage risks in development. Program sets in motion plan to enter production in 2007 shortly after first flight of the non-production representative aircraft. The program was entering production with less than 1 percent of testing complete. We recommended that the program delay investing in production until flight testing shows that the Joint Strike Fighter performs as expected. DOD partially concurred but did not delay start of production because it believed the risk level was appropriate. The program was restructured to reflect findings from a recent independent cost team and independent manufacturing review team. As a result, development funds increased, test aircraft were added, the schedule was extended, and the early production rate decreased. Costs and schedule delays inhibited the program’s ability to meet needs on time. We recommended that the program complete a comprehensive cost estimate and assess warfighter and initial operational capability requirements. We suggested that Congress require DOD to tie annual procurement requests to demonstrated progress. DOD continued restructuring, increasing test resources, and lowering the production rate. Independent review teams evaluated aircraft and engine manufacturing processes. Cost increases later resulted in a Nunn- McCurdy breach. Military services completed the review of capability requirements, as we recommended. The program incorporated positive and more realistic restructuring actions taken since 2010, including more time and funding for development and deferred procurement of more than 400 aircraft to future years. The program was moving in the right direction but needed to fully validate design and operational performance and at the same time make the system affordable. We did not make recommendations to DOD in this report. DOD agreed with GAO’s observations. Year, GAO report 2014 GAO-14-322 Key program event The services established initial operational capabilities dates in 2013. The Marine Corps and Air Force planned to field initial operational capabilities in 2015 and 2016, respectively, and the Navy planned to field its initial capability in 2018. Primary GAO conclusions/recommendations Delays in developmental flight testing of the F-35’s critical software may hinder delivery of the warfighting capabilities to the military services. We recommended that DOD conduct an assessment of the specific capabilities that can be delivered and those that will not likely be delivered to each of the services by their established initial operational capability dates. DOD response and actions DOD concurred with our recommendation. On June 22, 2015, the Under Secretary of Defense for Acquisition, Technology, and Logistics issued a Joint Strike Fighter software development report, which met the intent of GAO’s recommendation. DOD planned to begin what it refers to as a block buy contracting approach that was anticipated to provide cost savings. In addition, DOD planned to manage the follow-on modernization program under the current F-35 program baseline and not as its own separate major defense acquisition program. The terms and conditions of the planned block buy and managing follow-on modernization under the current baseline could present oversight challenges for Congress. We recommended that the Secretary of Defense hold a milestone B review and manage follow-on modernization as a separate major defense acquisition program. DOD did not concur with our recommendation. DOD viewed modernization as a continuation of the existing program and the existing oversight mechanisms, including regularly scheduled high-level acquisition reviews, would be used to manage the effort. The DOD F-35 program office was considering contracts for economic order quantity of 2 years’ worth of aircraft parts followed by a separate annual contract for procurement of lot-12 aircraft with annual options for lot-13 and lot-14 aircraft. However, as of January 2017, contractors stated they were still negotiating the terms of this contract; therefore, the specific costs and benefits remained uncertain. Program officials projected that the program would only need $576.2 million in fiscal year 2018 to complete baseline development. At the same time, program officials expected that more than $1.2 billion could be needed to commit to Block 4 and economic order quantity in fiscal year 2018. GAO recommended DOD use historical data to reassess the cost of completing development of Block 3F, complete Block 3F testing before soliciting contractor proposals for Block 4 development, and identify for Congress the cost and benefits associated with procuring economic order quantities of parts. DOD did not concur with the first two recommendations and partially concurred with the third while stating that it had finalized the details of DOD and contractor investments associated with an economic order quantity purchase and would brief Congress on the details, including costs and benefits of the finalized economic order quantity approach. Year, GAO report 2018 GAO-18-321 Key program event The program office determined that it could not resolve all open deficiencies found in developmental testing within the development program, and they would need to be resolved through post-development contract actions. DOD provided a report to Congress outlining preliminary plans to modernize the F-35. It stated it planned to develop a full acquisition program baseline for the modernization effort in 2018 and provide a report to Congress by March 2019. Primary GAO conclusions/recommendations The program office plans to resolve a number of critical deficiencies after full-rate production. We recommended that the F-35 program office resolve all critical deficiencies before making a full- rate production decision, and identify steps needed to ensure the F-35 meets reliability and maintainability requirements before each variant reaches maturity. We also suggested that Congress consider providing in future appropriations that no funds shall be available for obligation for F-35 Block 4 until DOD provides a report setting forth its complete acquisition program baseline for the Block 4 effort to the congressional defense committees. DOD response and actions DOD concurred with both recommendations and identified actions that it would take in response. The National Defense Authorization Act for fiscal year 2019 included a provision limiting DOD from obligating or expending more than 75 percent of the appropriations authorized under the Act for the F-35 continuous capability development and delivery program until 15 days after the Secretary of Defense submits to the congressional defense committees a detailed cost estimate and baseline schedule. Appendix II: Objectives, Scope, and Methodology The National Defense Authorization Act for fiscal year 2015 included a provision for GAO to review the F-35 acquisition program annually until the program reaches full-rate production. This is the fourth report under that provision. In this report, we (1) provide information on the program’s progress toward completing testing of the baseline aircraft; (2) assess the aircraft’s current reliability and maintainability (R&M) status; (3) assess the program’s modernization efforts (to add new aircraft capabilities), known as Block 4; and (4) provide information on the program’s production costs and efficiency initiatives. To provide information on progress in the F-35’s development, we reviewed the program’s costs, schedule, and performance plans and compared the actual progress in each area with the goals established in its 2012 baseline to identify any significant trends. We also reviewed the F-35’s selected acquisition report and its fiscal year 2019 budget request. To assess progress in testing, we reviewed test results and associated reports, program briefings, and internal DOD program analyses. We interviewed officials from the program office, military test authorities, and contractors—Lockheed Martin (airframe) and Pratt & Whitney (engine)— on key aspects of F-35 development progress, including flight testing, future test plans, and recent findings from test events. We also interviewed the Director, Operational Test and Evaluation office and F-35 program developmental and operational test pilots. To assess the program’s progress in achieving its R&M targets, we obtained and analyzed its monthly reports on R&M performance from January 2018 through December 2018. We compared these to the program’s R&M targets documented in the F-35 Operational Requirements Document and the Joint Contract Specification. We examined program data for the metrics’ performance across 12 months to identify any trends. We assessed the reliability of this data by reviewing supporting documentation and interviewing program office officials who track reliability metrics and other knowledgeable DOD officials. We also reviewed the program’s Reliability and Maintainability Improvement Program’s guidance to determine if it contained specific and measurable objectives and the projects needed to meet those objectives. We determined that the R&M metric data were sufficiently reliable for our purposes of determining whether the program will meet its targets. To assess the program’s Block 4 modernization plans, we reviewed documents that GAO best practices identify should be completed prior to awarding a development contract. We interviewed DOD and program office officials, and contractor representatives regarding the program’s Block 4 activities to date and future plans. We compared the program’s accomplishments over the past year and its future plans to the product development best practices identified by GAO. We reviewed the fiscal year 2019 budget request to identify costs associated with the Block 4 effort. We obtained contract documents for Block 4 activities between March 2014 and December 2018 to determine the total amount of funding that has been obligated to Block 4 and the scope of work that has been contracted. To provide information on ongoing manufacturing performance and the program’s plans to achieve full rate production, we obtained and analyzed the prime contractor’s production metrics and its aircraft delivery rates and from 2012 through 2018. We compared this performance to the program’s procurement plans from its selected acquisition reports since 2003. We reviewed metrics and briefings provided by the program office, Lockheed Martin, Pratt & Whitney, and the Defense Contract Management Agency to identify progress in improving manufacturing processes. We analyzed changes in delivery dates for lot 10 aircraft delivered in 2018. We discussed reasons for any delivery delays and plans for improvement with officials from Lockheed Martin and Pratt & Whitney. We obtained cost investment and savings estimates and discussed cost and manufacturing efficiency initiatives, such as the economic order quantity purchases, with the contractors and program office officials to understand potential cost savings and plans. We also obtained and analyzed metrics on parts and aircraft quality through December 2018 and discussed steps taken to improve quality and deliveries with Lockheed Martin and Pratt & Whitney officials. We determined that the contractor’s production metrics and delivery dates were sufficiently reliable for our purposes of determining production efficiency and deliveries. We conducted this performance audit from June 2018 to April 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix III: Status of Selected F-35 Technical Risks The F-35 program continues to address technical risks discovered in testing. Since our 2018 report, the program identified new risks with the canopy, fuel tubes, and cockpit, described below. The program has also incorporated design changes that have mitigated technical risks that we previously highlighted. The status of the Department of Defense’s (DOD) efforts to address these issues follows. Newly Identified Technical Risks Canopy Coating De-laminations and Corrosions: The F-35 fleet has experienced approximately 20 incidents of the canopy transparencies delaminating after less than 100 flight hours. The contractor is currently testing numerous solutions for the de-laminations, with intentions of completing testing by January 2019. F-35 aircraft are also experiencing canopy corrosion resulting from moisture intrusion due to the aircraft’s adhesive cracking under pressure and insufficient tape adhesion. The program has identified the need to modify over 173 canopies over 4 years. The contractor has begun to incorporate alternative material and tape into production, and released standardized repair procedures to mitigate this issue. Engine Fuel Tubes: In September 2018, a manufacturing fault in an engine fuel tube caused an in-flight failure, which resulted in an F-35B crash. The investigation identified several other life-limited fuel tubes in each F-35 variant. The fleet was grounded while all aircraft were inspected, and any fuel tubes identified were replaced or will be replaced by June 2019. Cockpit Display: In November 2018, operational test pilots experienced the cockpit display freezing and blanking, and identified the problem as a category 1 deficiency. The display issues occurred after a software update. The start of operational testing was delayed until the contractor could provide a software update to correct the problem, which was accomplished with a work-around in December 2018. Technical Risks Identified In Our Previous Reports Helmet Mounted Display: During low-light flights, the Helmet Mounted Display’s technology cannot display pure black, causing a green glow on the screen which makes it difficult to see the full resolution of the night vision video feed. The contractor is developing a new system to avoid this effect, and the contractor delivered this system to the test fleet in September 2018 with final flight testing planned through January 2019. Figure 7 is a photograph of the Helmet Mounted Display. Aerial refueling probes: The F-35B and F-35C variants use a “hose and drogue” system in which an aerial refueling tanker aircraft extends a long, flexible refueling hose and a parachute-like metal basket that provides stability, the receiving aircraft then connects to the drogue basket with its extendable refueling probe, as shown in figure 8. The refueling probe tips are meant to break in the event there is a stress occurring during refueling. However, the breaking is occurring more often than expected. Since April 2014, more than 20 incidents have occurred where the F-35’s aerial refueling probes broke off while conducting aerial refueling, leading to a restriction of aerial refueling operations. Tire service life: We reported in June 2018, the average service life of tires on the F-35B is below 10 landings. Lockheed Martin is currently working with three tire manufacturers to develop a new design with the goal of 20 landings. Testing of the new tires will occur throughout 2019. Figure 9 shows an F-35B during a landing. Life support system: The program has identified over 35 pilot physiological events, of which nearly 30 occurred in-flight. An action team made of government officials, contractors, and doctors completed their work by May 2018. A root cause investigation did not identify any F-35 system deficiencies, but reported it was difficult to fully determine the problem due to a lack of real-time data. Contracting officials stated that this is partially because the technology has not yet been developed to monitor pilot’s health in flight, in real time. The prime contractor continues to try to develop a means to monitor pilot health. Appendix IV: Comments from the Department of Defense Appendix V: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, the following staff members made key contributions to this report: Justin Jaynes (Assistant Director), Jennifer Baker, Emily Bond, Brandon Booth, Erin Butkowski, Matthew T. Crosby, Desirée E. Cunningham, R. Eli DeVan, Laura Jezewski, Jennifer Leotta, Meghan Perez, Hai Tran, Abby Volk, Mary Weiland, Alyssa Weir, and Robin M. Wilson. Related GAO Products F-35 Joint Strike Fighter: Development is Nearly Complete, but Deficiencies Found in Testing Need to Be Resolved. GAO-18-321. Washington, D.C.: June 5, 2015. Warfighter Support: DOD Needs to Share F-35 Operational Lessons Across the Military Services. GAO-18-464R. Washington, D.C.: April 25, 2018. F-35 Aircraft Sustainment: DOD Needs to Address Challenges Affecting Readiness and Cost Transparency. GAO-18-75. Washington, D.C.: October 26, 2017. F-35 Joint Strike Fighter: DOD’s Proposed Follow-on Modernization Acquisition Strategy Reflects an Incremental Approach Although Plans Are Not Yet Finalized. GAO-17-690R. Washington, D.C.: August 8, 2017. F-35 Joint Strike Fighter: DOD Needs to Complete Developmental Testing Before Making Significant New Investments. GAO-17-351. Washington, D.C.: April 24, 2017. F-35 Joint Strike Fighter: Continued Oversight Needed as Program Plans to Begin Development of New Capabilities. GAO-16-390. Washington, D.C.: April 14, 2016. F-35 Sustainment: DOD Needs a Plan to Address Risks Related to Its Central Logistics System. GAO-16-439. Washington, D.C.: April 14, 2016. F-35 Joint Strike Fighter: Preliminary Observations on Program Progress. GAO-16-489T. Washington, D.C.: March 23, 2016. F-35 Joint Strike Fighter: Assessment Needed to Address Affordability Challenges. GAO-15-364. Washington, D.C.: April 14, 2015. F-35 Sustainment: Need for Affordable Strategy, Greater Attention to Risks, and Improved Cost Estimates. GAO-14-778. Washington, D.C.: September 23, 2014. F-35 Joint Strike Fighter: Slower Than Expected Progress in Software Testing May Limit Initial Warfighting Capabilities. GAO-14-468T. Washington, D.C.: March 26, 2014. F-35 Joint Strike Fighter: Problems Completing Software Testing May Hinder Delivery of Expected Warfighting Capabilities. GAO-14-322. Washington, D.C.: March 24, 2014. F-35 Joint Strike Fighter: Restructuring Has Improved the Program, but Affordability Challenges and Other Risks Remain. GAO-13-690T. Washington, D.C.: June 19, 2013. F-35 Joint Strike Fighter: Current Outlook Is Improved, but Long-Term Affordability Is a Major Concern. GAO-13-309. Washington, D.C.: March 11, 2013. Fighter Aircraft: Better Cost Estimates Needed for Extending the Service Life of Selected F-16s and F/A-18s. GAO-13-51. Washington, D.C.: November 15, 2012. Joint Strike Fighter: DOD Actions Needed to Further Enhance Restructuring and Address Affordability Risks. GAO-12-437. Washington, D.C.: June 14, 2012. Joint Strike Fighter: Restructuring Added Resources and Reduced Risk, but Concurrency Is Still a Major Concern. GAO-12-525T. Washington, D.C.: March 20, 2012.


Background DOD has long recognized that the contracts for items like weapon systems are capital intensive in nature and take a long time to produce. Contract financing assists the defense contractor in managing expenses, such as material, labor, and overhead. In such cases, DOD can agree to help finance these expenses as the work progresses through various types of contract financing payments, including progress payments and performance-based payments. Progress payments based on cost are determined as a percentage of the costs incurred by the contractor as work progresses. Currently, DOD pays 80 percent of incurred costs of large business concerns and 90 percent of incurred costs of small business concerns. To receive progress payments on the basis of cost, contractors are required to have an accounting system and controls that are adequate for proper administration of progress payments. The cognizant contract administration office is to maintain surveillance of the contractor’s accounting system as appropriate, and the Defense Contract Audit Agency is to audit the accounting system. DOD provides contract financing on fixed-price type contracts for non- commercial items. Performance-based payments enable the contractor to be paid for achieving certain contractual milestones, such as delivery of a major subcontracted component. DOD can pay up to 90 percent of either the contract’s price or the price of a deliverable item using performance-based payments. DOD’s performance-based payments guide states that these payments should not be structured such that they amount to advance payments, which in general terms are payments made before work is complete on a contract regardless of what performance milestones are met. Unlike progress payments, however, performance-based payments do not require that the contractor have an adequate accounting system. Lastly, contract financing can also be used when the terms and conditions of a contract are not yet “definitized,” a term that generally means finalized. These actions, which are termed undefinitized contract actions (UCAs) at DOD, are to be used only when the negotiation of a definitive contract action is not possible in sufficient time to meet DOD’s requirements and the department’s interest demands that the contractor be given a binding commitment so that contract performance can begin immediately. The government may incur unnecessary costs if requirements change before the contract is definitized. Defense acquisition regulations generally require UCAs to be definitized within 180 days of the UCA date or before more than 50 percent of the estimated contract price is obligated, whichever occurs first. During this period, progress payments are limited to 80 percent of work accomplished. DOD’s 2014 performance-based payments guide recommends that a UCA be awarded using progress payments first; performance-based payments should then be considered during the definitization process. Table 1 summarizes the conditions and rates applicable to progress payments based on costs and performance-based payments. Several offices and agencies within DOD have a role in managing contract financing. The office of DPC, within DOD’s Office of the Under Secretary of Defense for Acquisition and Sustainment, is responsible for all pricing, contracting, and procurement policy matters. This office formulates and oversees DOD-wide pricing policies and strategies supporting the procurement of major defense programs, including programs that use progress and performance-based payments. DCMA and other contract administration offices monitor contractors’ performance and management systems to ensure that cost, product, and performance are in compliance with the contract terms. DCMA generally maintains contract financing payment data for DOD progress and performance-based payments for contracts DCMA administers. Within DCMA, the Cost and Pricing Center supports DOD-wide analysis of contract data to support DOD-wide decision making, among other things. Relationship of Contract Financing to Contract Profit and Contractor Profitability Contract financing has an impact on the price of negotiated contracts and, more generally, on the health and profitability of the defense contractor. On negotiated contracts, DOD requires contracting officers to use weighted guidelines, a structured approach used to develop profit objectives for individual defense contracts. DOD implements its profit policy through the weighted guidelines. As part of their efforts to determine the government’s negotiating position, including how much profit the contractor should receive under the contract, contracting officers are to consider various factors, including the degree to which the government is providing contract financing. Assuming other factors are held constant, the weighted guidelines suggest that the negotiated profit rate of a fixed-price defense contract might be 1 to 2 percentage points lower when the government provides contract financing. The contracting officer may vary the amount to consider other risk elements when establishing the government’s negotiating position. DOD-provided contract financing can also provide contractors higher rates of return on the amount of corporate funds contractors invest on that same contract. One measure of this benefit is “internal rate of return” (IRR), a tool that can be used to assess the impact of contract financing on overall contractor profitability. DOD’s 2001 Incentives Guidebook notes that IRR is one of the basic tools used by industry to determine where to invest its funds and assess the risks and potential rewards involved in contracting with the government or commercial entities. IRR is a measure that integrates both the contractor’s investment to produce the product and the profit earned on that product. In contrast to contracts in which the contractors must either self-finance or borrow from commercial lenders, when contractors receive financing on a contract from the government the contractor’s IRR can be significantly higher. Figure 1 provides a hypothetical example of how changes in the progress payment rate on a 40-month, fixed-priced contract affects the expected contract profit rate and the contractor’s IRR. As illustrated above, providing contract financing (in this case, progress payments) has a significant impact on the contractor’s IRR and a lesser impact on the actual profit that DOD expects the contractor to make. For example, if DOD provided no contract financing, the weighted guidelines would suggest a profit rate on this hypothetical contract of 13.8 percent, which would provide an internal rate of return to the contractor of 7.5 percent. If DOD provided progress payments at the customary rate of 80 percent, the weighted guidelines would suggest a profit rate on this hypothetical contract of 10.4 percent, or 3.4 percentage points lower than if no financing was provided. However, even though the contractor’s expected profit is lower, the IRR for the contract would increase to 30.9 percent, or a little more than four times what would be realized if the contractor had to finance the effort on its own. Prior Studies of Contract Financing and Contract Profitability Several studies conducted by DOD, nonprofit organizations and GAO have assessed the impact of contract financing on contract profit or contractor profitability. These studies have generally found that, depending on the measure used, the defense industry generates high returns on investment. For example, In 1976, DOD’s Profit ‘76 study examined earnings’ relationship to capital investment and increased productivity. The Profit ‘76 study group concluded that government contractors were able to maintain higher profits by keeping investment low partly because DOD did not have profit policies in place to encourage investment in items such as facilities. As a result of the Profit ‘76 study, DOD made a number of changes to its profit policy to encourage corporate investment in facilities, among other things. In 1991, GAO suggested that using return on assets to measure profitability of defense contractors is beneficial because it recognizes how government financing can affect contractors’ levels of profitability. In 2008, the Institute for Defense Analyses reported that defense contractors generated high returns with low operating margins, in part because government-provided contract financing helped fund the contractors’ long, asset-intensive product cycles. According to DPC officials, however, the most comprehensive study of contract financing and profit policies was conducted by the DFAIR commission in 1985. We discuss this study in more detail below. Changes to the Legislative and Regulatory Framework Governing Contract Financing since 1985 Since the DFAIR commission issued its report in 1985, Congress and DOD have made a number of changes to the statutory and regulatory framework intended to (1) reduce the administrative burden associated with contract financing and (2) encourage the use of performance-based payments (see figure 2). Our review found that DOD paid less in performance-based payments after making some changes to contract financing policies, but started increasing these payments again in 2016. DFAIR Commission’s 1985 Study Considered to Be the Last Comprehensive Study of DOD’s Contract Financing and Profit Policies According to DPC officials, the most comprehensive study of contract financing and profit policies was conducted by the DFAIR commission in 1985. The DFAIR commission assessed, among other issues, whether DOD contract financing policies were equitable in maintaining the defense industrial base and cost-effective for DOD, the effectiveness of DOD contract financing policies as a means of encouraging contractor cost efficiencies, the profitability of defense work and its reasonableness in comparison with the profitability of the non-defense sector, and the interrelationship of DOD’s contract finance and profit policies. In evaluating contractor financing costs, DFAIR developed a model of a typical contract to use in calculating contractors’ contract financing costs, the amount of interest a contractor would have to pay if it were required to bear all those costs, and the effect of payment delays on contractor financing costs. The DFAIR commission reached a number of conclusions about DOD’s contract financing and profit policy in effect at that time. The study concluded that: The progress payment rate was appropriate for the time period studied but should be revised based on changes in short-term interest rates. DOD’s profit policy as reflected in the weighted guidelines at the time of the study did not explicitly take into account the cost of working capital (the difference between a contractor’s assets and liabilities). The profitability of individual defense contracts the commission reviewed had been consistently lower than the profit levels reported to have been negotiated by government contracting officers. DOD’s profit policy needed to be simplified and better integrated with contract financing policy. The study also concluded that there was a need to make DOD contract financing more responsive to economic conditions and that profit policy, contract financing, and contractor investment are related. We agreed with the conclusion that profit policy, contract financing, and contractor investment are related. We also highlighted the need for recurring DOD contract profitability studies using a generally accepted methodology in our 1986 report. DOD Efforts to Reduce the Administrative Costs Our work found that since the DFAIR study was issued, DOD made several changes to reduce the administrative burden associated with contract financing requirements. These changes included Elimination of flexible progress payments (1999) – DOD introduced flexible progress payments in 1981 as a new approach to contract financing. Under flexible progress payments, DOD contracting officers were to use the DOD Cash Flow Computer Model to develop an applicable progress payment rate for that contract. Under this approach, DOD specified the minimum percentage the contractor was required to invest and DOD would provide the remainder. The amount of contractor investment required by DOD varied from 5 to 25 percent, depending upon the year. Flexible progress payments were not allowed on contracts issued after November 11, 1993; the references were eliminated completely from the DFARS in 1999. Elimination of “paid cost rule” (2000) – The paid cost rule required large businesses to pay subcontractors before billing the government for payment. After DOD eliminated this rule in March 2000, large businesses were generally able to include subcontract costs incurred but not yet actually paid on progress payment requests to the government. Elimination of “financial need requirement” (2016) – Since 2000, one of the ways contractors could receive progress or performance- based payments under the FAR was on the basis of financial need or the unavailability of private financing. In that regard, an April 2013 DOD Inspector General report found that contracting personnel did not properly negotiate and verify contractors’ need for contract financing before authorizing performance-based payments. The Inspector General recommended that contracting personnel determine whether private financing is available to a contractor before authorizing performance-based payments. While DOD concurred with the recommendation, it subsequently amended the DFARS in 2016 to eliminate the requirement for DOD personnel to justify the use of contract financing for certain fixed-price contracts. In doing so, DOD stated it was in DOD’s best interests. Efforts to Encourage the Use of Performance- Based Payments Congress enacted the Federal Acquisition Streamlining Act (FASA) in 1994 to provide the executive branch with requirements to improve the process for acquiring goods and services. FASA, among other things, established performance-based payments “wherever practicable” as a form of contract financing. In 1995, the FAR Council amended the FAR to enable the use of performance-based payments up to a maximum amount of 90 percent of the contract’s price. In 2000, DOD issued a rule amending the DFARS to emphasize that performance-based payments were the preferred method of financing. The rule required contracting officers to consider and deem performance-based payments impracticable before deciding to provide progress payments. This rule was part of a larger effort by DOD to make contract financing procedures easier to understand and to simplify related provisions. DOD subsequently issued a user’s guide in 2001 to help its contracting personnel and contractors in using performance-based payments. Despite the provisions to encourage the use of performance-based payments when appropriate, DOD subsequently initiated department- specific actions that, according to industry officials, decreased the frequency with which they received performance-based payments on defense contracts. For example, the Under Secretary of Defense for Acquisition, Technology and Logistics’ September 2010 Better Buying Power memorandum instructed contracting officers to use progress payments as the basis for price negotiations. After the contractor and DOD contracting officer agreed on price using progress payments, contractors could propose using an alternate financing arrangement, including performance-based payments. The memorandum indicated that the rationale for this change was to provide increased incentives for contractor performance. In April 2011, the Director of Defense Procurement and Acquisition Policy (now known as DPC) issued a memorandum that focused on the “practicality” of performance-based payments, stating they “are not practical for use on all fixed-price contracts and require considerable effort between the contractor and Government.” The memorandum noted that if contractors wanted to use performance-based payments, then the contractor should submit a proposed schedule to include all performance-based payment events, completion criteria, and event values, along with the contractor’s expected expenditure profile. To implement its April 2011 performance-based payment policy, DOD issued a proposed rule to amend the DFARS in January 2012. This rule was finalized in March 2014. The 2014 version of the DOD performance-based payments user’s guide noted that performance-based payments are the preferred method only when they are deemed practical by the contracting officer. However, industry officials told us that they frequently cannot reach agreement with DOD regarding performance milestones, and therefore agree to the use of progress payments instead. The impact of DOD’s changes on the relative use of progress versus performance-based payments is uncertain. Between fiscal years 2010 and 2018, DCMA data indicates that DOD provided between $36 billion and $49 billion a year in contract financing on contracts DCMA administered. We found that nearly 98 percent of those contract financing payments were paid to medium and large defense contractors. We also found that the amount DOD paid out in performance-based payments on those contracts fell between 2010 and 2016 before increasing in 2017. In December 2016, Congress enacted Section 831 of the Fiscal Year 2017 NDAA to establish performance-based payments as the preferred type of contract financing for DOD in statute. Section 831 also directed the Secretary of Defense to ensure that nontraditional defense contractors and other private sector companies are eligible for performance-based payments, in line with best commercial practices. Figure 3 shows the differences in DOD’s progress and performance-based payments between fiscal years 2010 and 2018 for contracts administered by DCMA. In August 2018, DOD introduced a proposed rule that was intended to use contract financing rates to help incentivize contractor performance and to implement Section 831. The proposed rule would have set a base progress payment rate for large businesses (specifically, for other than small businesses) at 50 percent and small businesses at 90 percent. At the same time, however, the proposed rule provided opportunities to increase the rate if the contractor achieved certain enterprise-wide priorities such as meeting contract delivery dates. The proposed rule also eliminated some of the administrative requirements associated with performance-based payments to encourage their use. According to DPC officials, the rates would be subject to an annual adjustment based on the performance criteria provided in the rule. Table 2 summarizes key aspects of the proposed rule. DOD officials acknowledged that if implemented, contractors would initially receive a lower level of contract financing, but believe with improvements in their overall performance contractors would eventually receive much higher levels of financing than currently provided. Industry officials voiced a number of concerns about the proposed rule at the January and February 2019 public meetings held after the rule was proposed, as well as in our interviews with them. For example, these officials noted that the proposed rule would change the intent of contract financing from a means of assisting contractors to help meet short-term expenses to a mechanism for ensuring compliance with contract terms and conditions on an enterprise-wide basis. Industry officials said they believe compliance with contract terms and conditions should be addressed on a contract-by-contract basis. Further, industry officials stated that the changes suggested in the proposed rule could negatively impact the health, competitiveness, and resiliency of the defense industrial base and introduce significant uncertainty as to how much contract financing DOD would provide. Additionally, industry officials noted that the rule did not contain specific implementation details in such areas as whether the incentives would be applied on an enterprise-wide basis and how to ensure the data were reliable. DOD withdrew this rule in October 2018, citing the need to conduct additional outreach with industry regarding contract financing methods. Subsequently, DPC held three public meetings in January and February 2019 to obtain public comments on revising policies and procedures for contract financing, performance incentives, and associated regulations prior to proposing a new rule. DPC provided no timeframes for doing so. DOD officials issued the proposed rule in April 2019 to implement Section 831’s statutory preference for performance-based payments for public comment. The proposed rule notes that performance-based payments are the preferred method of contract financing at DOD whenever practicable. The period for public comments ends on July 1, 2019. DOD officials indicated that they hope to issue a final rule in early 2020. DOD Has Not Comprehensively Assessed the Impact of Its Contract Financing and Profit Policy on the Defense Industry Since 1985 Defense Industry and Market Conditions Have Changed since 1985 DOD has not conducted a comprehensive assessment of the impact of its contract financing and profit policies on the defense industry since the DFAIR study was completed in 1985. In the intervening time, there have been significant changes in the composition of the defense industry, business practices, and economic conditions. In December 2018, DPC officials acknowledged the need to assess contract financing policies against market and economic conditions on an ongoing basis and determine the effect these policies have on the defense industry, but did not provide a timeframe for doing so. DOD officials acknowledged that the department has not done a comprehensive assessment of how its contract financing policies affect the defense industry since the DFAIR study was issued in 1985. DOD had previously stated its intent to do such an assessment on a regular basis. Specifically, in 1991 DOD noted that it would issue progress payment rates each February. DOD also noted that it would use the methodology from the DFAIR study to determine the progress payment rate based on short-term commercial interest rates. However, DOD removed the DFARS provision related to flexible progress payments in 1999. Overall, we found that DOD has adjusted the progress payment rate five times since the DFAIR study was completed, but only adjusted the progress payment rate twice since 1991 when DOD indicated its intent to assess the rate annually. DOD last changed the progress payment rate in 2001 (see table 3). Since the DFAIR study was conducted and DOD last assessed progress payment rates, DOD and industry officials noted that the composition of the defense industry has changed, as we have noted in our prior work. For example, in 1997, we reported that the end of the Cold War and the subsequent declines in DOD budgets resulted in, among other changes, a reduction in the number of defense contractors through various mergers and acquisitions. In our current work, DPC officials pointed to a changing proportion of subcontractors relative to prime contractors. Industry officials also identified the emergence of contractors who do not typically work with DOD and technology companies into the defense sector as an issue that should be considered when looking at contract financing and profit policies. According to industry officials, the industrial base has moved away from heavy industrial manufacturing toward technology and more sophisticated industry partners, including contractors who do not typically work with DOD. These officials noted that these contractors may not be eligible for contract financing because they may not have an approved cost accounting system needed to receive progress payments. In that regard, in July 2017, we reported that one company conducted a study that determined it would take at least 15 to18 months and millions to establish a government-unique cost accounting system. Industry officials also noted that the emergence of high-technology companies may pose a challenge to traditional defense contractors in terms of attracting financing and investment from commercial and private investors at competitive rates. Industry officials also identified changing business practices, including the increased use of UCAs, which affect their ability to use performance- based payments. Industry officials stated that it is more difficult to negotiate performance-based payments on UCAs, noting that DOD’s guidance suggests that performance-based payments should not be provided for UCAs until definitization occurs. Our review of DOD’s semi- annual reports to Congress on the use of UCAs found that the number of UCAs and unpriced change orders reported by DOD has varied between March 2014 and September 2018 (see figure 4). DOD reported that the total not-to-exceed dollar value of all UCAs and unpriced change orders was approximately $76 billion as of September 2018. Finally, market and economic conditions have changed since the DFAIR study. For example, at the time of the DFAIR study, short-term interest rates were around 8 percent, whereas the short-term interest rate in 2018 was 2 percent. Figure 5 shows the changes in short-term interest rates and inflation since 1980. Industry officials noted, however, that a comprehensive economic assessment of defense industry returns and the cost of contract financing policies should be conducted. For example, they noted that a reduction to progress payment rates in times of higher interest rates would increase their cost of working on complex contracts. Industry officials acknowledged that while interest rates have been low, they anticipate rates increasing in the near future. DOD’s August 2018 Proposed Rule Did Not Consider Impact on the Defense Industry DPC officials acknowledged that DOD’s August 2018 proposed rule did not assess the proposed rule’s impact on the health and profitability of the defense industry. DPC officials noted that since the proposed rule was focused on incentivizing contractor performance, DOD’s supporting analysis did not include an assessment of how the proposed rule would impact the overall profitability of defense contractors (such as assessing the impacts to a contractor’s internal rate of return) or of the profitability of defense work relative to non-defense industry opportunities. Rather, DOD’s analysis estimated the total financial impact the rule would have on large and small contractors primarily based on interest costs. Further, DOD stated in its supplementary material that it did not consider the extent to which the contract profit policy (in the form of weighted guidelines) would need to be adjusted given the proposed rule changes. DPC officials explained that changes to the weighted guidelines would need to consider how such changes would support the intent of providing higher rates of contract financing for higher levels of contractor performance. If DOD were to only propose a change to the progress payment rate, DPC officials acknowledged that such an assessment should consider what changes, if any, would need to be made to the weighted guidelines. DPC officials said they conducted an informal analysis that assessed contractor profitability, but this analysis was not made publicly available. In December 2018, DPC officials acknowledged the need to assess contract financing policies against market conditions on an ongoing basis and determine the effect these policies have on the defense industry. GAO’s Standards for Internal Control in the Federal Government call for monitoring the effectiveness of systems and policies throughout an organization on a recurring basis. Until DOD conducts a comprehensive assessment and updates that assessment on a recurring basis, it will not be in a position to understand whether current or future contract financing policies are achieving their intended objectives. Conclusions DOD and industry officials have acknowledged that the defense industry, economic and market conditions, legislative and regulatory requirements, and business practices have all changed since the issuance of the DFAIR study in 1985. Despite this recognition, DOD has not conducted a comprehensive assessment of how its contract financing policies affect the defense industry in more than 30 years. Without assessing the collective impact of these changes, DOD may be assuming too much financial risk or providing contractors with levels of working capital that are not commensurate with what is needed to help finance long-term projects, and affecting its ability to attract new entrants into the defense market. That assessment, however, should not be a one-time effort. A prior DOD study, our work, and the department have acknowledged the need to do so on a regular and recurring basis. Without a comprehensive and systemic assessment, conducted on a recurring basis, of DOD’s contract financing policy’s effect on the defense industry, DOD will not be in a position to understand whether current or future policies are achieving their intended objectives. Recommendation for Executive Action We recommend that the Acting Secretary of Defense direct the Under Secretary for Acquisition and Sustainment to ensure it conducts a comprehensive assessment of the effect that its contract financing and profit policies have on the defense industry and update that assessment on a recurring basis. (Recommendation 1) Agency Comments and Our Evaluation We provided a draft of this report to DOD for review and comment. DOD provided written comments, which are reprinted in appendix I, and concurred with our recommendation. In concurring with our recommendation, DOD stated it would seek fiscal year 2020 funds to contract a study on DOD contract financing policies and their effect on the defense industry. DOD also provided technical comments, which we incorporated as appropriate. We are sending copies of the report to the Acting Secretary of Defense; the Principal Acting Director, Defense Pricing and Contracting; the Director, Defense Contract Management and Agency; the Director, Office of Management and Budget; the Administrator for Federal Procurement Policy, and appropriate congressional committees. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-3665 or dinapolit@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. Appendix I: Comments from the Department of Defense Appendix II: Legal Chronology of Select Contract Financing Changes Appendix III: GAO Contacts and Staff Acknowledgments GAO Contact Staff Acknowledgements In addition to the contact named above, Bruce H. Thomas, Assistant Director; Lorraine Ettaro, Elizabeth Field, Gina Flacco, Stephanie Gustafson, Kristen Kociolek, John Lopez, Beth Reed Fritts, Miranda Riemer, Anne Stevens, Megan Stewart, Anne Louise Taylor, Alyssa Weir, Robin Wilson, and Alex Winograd made key contributions to this report.