Background Navy shipbuilding is a costly and complex endeavor that requires billions of dollars to develop, design, and construct ships. However, the acquisition phase of a ship’s life cycle only accounts for approximately 30 percent of a ship program’s total life-cycle cost. Notionally, the remaining 70 percent of the life-cycle cost of a ship program is incurred after the Navy delivers new ships to the fleet during the phase known as O&S. DOD guidance states that these long-term sustainment costs are determined in large part by decisions made early in the acquisition process. Approximately 80 percent of a program’s O&S costs fixed at the time the shipbuilding program’s requirements are set and the ship is designed. Additionally, we have found that once these decisions are made, it can be very difficult and costly to make changes if sustainment improvements are needed. According to DOD, operational support is a function of several related factors—reliability, availability, maintainability, and cost—that are determined in large part by decisions made before the start of construction. Reliability is the probability that an item, such as a system, can perform a required function under stated conditions for a specified period of time. Availability is a measure of the degree to which an item is in an operable state and can be called upon to work at the start of a mission and at an unknown (random) point in time. In other words, the degree to which a system is operable and available for mission tasking when needed. Maintainability is the ability of an item, such as a system, to be retained in or restored to a specified condition when maintenance is performed by skilled personnel using prescribed procedures and resources, at each prescribed level of maintenance and repair. Cost refers to the O&S costs associated with sustaining the ship. When planning for and executing ship sustainment, DOD guidance states that the program manager’s goal is generally to find a solution that maximizes reliability, availability, and maintainability within cost constraints. As the Navy acquires its ships, it makes a series of decisions that have implications for how a ship class can be affordably sustained, including decisions about engineering, ship design, equipment selection, and planned maintenance approaches. As such, DOD guidance advises acquisition programs, including Navy shipbuilding programs, to plan for and design reliability, availability, and maintainability into the weapon system early in the acquisition effort. For the purposes of this review, we define early in the acquisition process as the time period between the beginning of the program and the start of construction on the lead ship. Giving attention to these sustainment issues early in the acquisition process is intended to help programs ensure that their ships will be sustainable and affordable over their entire life cycle. Conversely, if reliability, availability, and maintainability are not adequately designed into the ship, there is a risk the ship will cost more to own and operate than expected and will not be available for use when needed by the fleet. Since Navy ships are comprised of numerous systems that need to work together, planning for sustainment and designing reliability, availability, and maintainability into a ship is a complicated task. Most Navy ships can accomplish several different missions, and accomplishing these missions usually requires a set of mechanical, electrical, combat, information technology, and other systems to work together. Each of these systems individually needs to be reliable, available, and maintainable in order for the ship as a whole to be sustainable. As such, addressing sustainment during the acquisition process is an effort that requires coordination and input from a variety of officials associated with the program, including the program manager, requirements officials, ship design managers, engineers, PSMs, and others. DOD and Navy Policies for Acquisition Programs DOD acquires new weapons, including ships, for its warfighters through a process described in two key acquisition policies: Department of Defense Directive 5000.01, which establishes the overarching framework for the Defense Acquisition System, and Department of Defense Instruction 5000.02, which implements the Defense Acquisition System. These policies provide management principles and detailed procedures, respectively, for the execution and management of defense acquisition programs. Specifically, these policies establish the phases of the acquisition process, key milestone decision points, required acquisition documentation, and roles and responsibilities for acquisition officials, among other things. Under this framework, shipbuilding programs move through several acquisition phases, including requirements setting, material solution analysis, technology development, ship design, ship construction, deployment, and sustainment. In order to proceed through the acquisition process, shipbuilding programs must be reviewed periodically at key decision points, called milestones, at which a Milestone Decision Authority assesses the program’s progress in achieving its goals. These milestones typically coincide with significant increases in the resources devoted to the program. To ensure senior leadership is well- informed at these decision points, shipbuilding programs are generally required to create or update key acquisition documents for milestone reviews that contain information on the program’s requirements, costs, and schedule, among other things. The Navy has also established its own acquisition policy and processes to supplement the DOD-wide acquisition policies and to oversee acquisitions managed internally to the Navy. The Navy’s acquisition policy, Secretary of the Navy Instruction 5000.2, provides instructions for implementing the Defense Acquisition System within the Navy, as well as additional Navy-specific acquisition procedures. In particular, Navy acquisition policy establishes a series of seven Navy decision points throughout the acquisition process, called Gate reviews, which complement the DOD milestones. These Gate reviews are split into two phases that the Navy calls passes: the first is led by the CNO and focuses on requirements setting and the second is led by the Assistant Secretary of the Navy for Research, Development, and Acquisition (ASN (RD&A)) and focuses on acquisition. As programs move through the acquisition process, Navy leadership—comprised of officials from the acquisition, requirements, resources, and warfighting communities— convenes Gate reviews to conduct oversight and ensure programs are on track to achieve their acquisition and sustainment goals. Each Gate review has a different objective and list of topics that need to be included in the Gate briefing. Lastly, DOD and Navy policy both allow for the Milestone Decision Authority to tailor the acquisition process outlined in these policies. Figure 1 depicts the acquisition process for Navy shipbuilding programs, as established by DOD and Navy acquisition policies. The Navy’s acquisition policy also details the acquisition responsibilities of key Navy officials, including the ASN (RD&A), the CNO, and program managers. The CNO and the ASN (RD&A) are key Navy leaders who chair the Gate review process and approve acquisition documents. For most shipbuilding programs, the ASN (RD&A) also serves as the decision authority to approve the advancement of these programs through the acquisition process at the appropriate milestones. Further, the policy enclosures delineate various elements of acquisition programs, such as systems engineering, testing, and sustainment planning. DOD and Navy acquisition policies both include requirements for shipbuilding programs to consider sustainment throughout the acquisition process. For instance, prior to Milestone A, DOD policy states that sustainment planning and considerations should inform the development of program requirements and early ship design decisions. As programs move into the design and construction phases, programs are to develop a comprehensive product support package and evaluate it through engineering reviews and other tests to ensure it is sufficient to meet sustainment requirements and affordability targets. The planning documents that comprise these support packages, such as life-cycle sustainment plans (LCSPs), are intended to set the foundation for how the fleet will sustain a class of ships. Statutory Changes That Have Increased Attention on Sustainment during the Acquisition Process In addition to the requirements set in DOD and Navy policies, Congress has passed laws related to increasing DOD and Navy attention on sustainment throughout the acquisition process. Chief among these is the creation of the role of the PSM. A PSM should develop and implement a comprehensive product support strategy for weapon systems, among other things. More recently, Congress has directed organizational changes related to DOD and Navy acquisition leaderships’ attention to sustainment. In response, the Navy has added a sustainment function to the ASN (RD&A)’s portfolio. The Navy implemented this direction in fiscal year 2020 with the appointment of a Deputy Assistant Secretary for Sustainment that reports to the ASN (RD&A). Congress has also established several requirements related to DOD and Navy management of acquisition programs’ O&S costs, sustainment planning, and sustainment reporting. For example, statute requires weapon system programs to consider, where appropriate, sustainment in key acquisition documents, such as acquisition strategies, designs, contracting, and cost estimates. Additionally, statute requires DOD to provide Congress with annual Selected Acquisition Reports that have sustainment and life-cycle cost information. Key Documents That Support Sustainment Planning during the Acquisition Process Shipbuilding programs are required to develop a suite of acquisition documents that provide information about the goals of the program and how the program office is developing and executing to these goals, pursuant to DOD and Navy acquisition policies. Many of these key acquisition documents contain information about the program’s sustainment requirements and plans, as discussed below. The Capability Development Document should define the program’s operational requirements, including the program’s key performance parameters. Key performance parameters are the most critical requirements a system must demonstrate to deliver an effective military capability. In 2007, DOD updated its requirements setting policy, called the Joint Capabilities Integration and Development System, to require all programs to establish key performance parameters for sustainment in response to concerns that acquisition programs were not adequately planning for sustainment. This requirement helps ensure that acquisition programs provide a weapon system to the warfighter with optimal availability and reliability at an affordable price. The sustainment key performance parameter is comprised of two measures—operational availability and materiel availability—which addresses the availability of the ship while in operations and under maintenance, respectively: Operational availability measures the probability that a system will be ready for use when expected. This requirement helps programs determine how reliable, maintainable, and supportable a system needs to be. Operational availability is also understood as the percentage of time a ship can perform its primary mission. Materiel availability measures the percentage of the total inventory of a system that is operationally capable based on materiel condition, which for ship platforms, is the percentage of a ship class available for deployment. This metric helps programs determine how many ships to buy in order to meet planned deployment schedules. This requirement should inform decisions that could increase or decrease planned maintenance time for a shipbuilding program. According to DOD guidance, the operational and materiel availability requirements should be considered in tandem to produce ships that work as expected and are available when needed, as shown in figure 2 below. During the acquisition process, the operational availability requirement should inform decisions about how to best increase reliability for systems needed to meet the key performance parameter. To do this, engineers can, among other things: (1) design systems that require less frequent maintenance, (2) add redundancy to key systems, or (3) ensure that systems can be fixed quickly and cheaply. At the same time, the materiel availability requirement should inform how many ships are purchased based on the quantity needed to accomplish missions at any one time. It also informs acquisition decisions that could affect the length of maintenance availabilities, such as maintenance time needed to repair or replace key components. The Life-Cycle Cost Estimate should provide information on the total estimated cost to develop, build, deploy, sustain, and dispose of a ship class over its life cycle, regardless of funding source. The life- cycle cost estimate is based on program objectives and operational requirements for the ship class. It should reflect a realistic appraisal of the program’s risks and the level of cost most likely to be realized. The life-cycle cost estimate includes O&S costs, which provide information on the estimated costs for crewing, operations, maintenance, sustaining support, continuing system improvements, and indirect support. The Acquisition Program Baseline (APB) is an overarching acquisition document that describes the shipbuilding program and presents the program’s approved cost, schedule, and performance goals. The APB is a formal, binding agreement between the Milestone Decision Authority, Program Manager, and their acquisition chain of command to be used for tracking and reporting on the program. The Life-Cycle Sustainment Plan (LCSP) should document the program’s product support strategy and governs planning for sustainment during the acquisition process, as well as the execution of sustainment activities after ships are delivered to the fleet. The LCSP describes the efforts necessary to develop and integrate sustainment requirements into the ship’s development, design, procurement, testing, fielding, and operation. It also lists the activities necessary for the shipbuilding program to develop, implement, and deliver a product support package that maintains affordable operational effectiveness over the ship’s life cycle. For example, the LCSP should contain information on sustainment engineering, O&S cost estimates and affordability constraints, reliability analysis, and sustainment contracts, among other things. The Independent Logistics Assessment (ILA) should be an impartial analysis of a program’s sustainment planning and execution to determine its ability to meet established performance and sustainment requirements. The ILA is intended to assess the adequacy of the program’s product support strategy, product support risks that are likely to drive future O&S costs, changes to system design that could reduce O&S costs, and effective strategies for managing O&S costs. According to DOD guidance, programs should use the results of the ILA to improve sustainment outcomes. Acquisition and Sustainment Stakeholders for Shipbuilding Programs There are a large number of Navy stakeholders involved in the effort to design, build, and support a ship class over its life cycle. In general, the acquisition community is led by the ASN (RD&A), while the operations and sustainment community is led by the CNO. Naval Sea Systems Command (NAVSEA) provides support to both the acquisition and sustainment communities and is comprised of experts across multiple disciplines. Figure 3 provides more information on the various acquisition and sustainment stakeholders that support the Navy’s ship classes. The ASN (RD&A) acts as the Navy Service Acquisition Executive and oversees the Navy’s shipbuilding program offices. Program Executive Offices are responsible for the life cycle management of their assigned programs. The program executive office is led by a program executive officer who, according to DOD’s updated acquition policy, should balance the risk, cost, schedule, performance, interoperability, sustainability, and affordability of a portfolio of acquisition programs and deliver an integrated suite of mission capability to users. For ships, there is a shipbuilding program office that is responsible for acquiring ships and an in-service program office that supports ships in sustainment. In some cases, these program offices are located within the same program executive office while, in other cases, these offices are split between different Navy organizations (typically the program executive office and NAVSEA). As such, the Navy’s shipbuilding programs and some program executive offices do not have responsibility for ship programs throughout their life cycle. The shipbuilding program offices manage their assigned shipbuilding programs through program initiation, technology development, ship design, construction, testing, and delivery. Acquisition program managers lead shipbuilding program offices and are responsible for the management of a program. Acquisition policies delineate a number of sustainment-related responsibilities for acquisition program managers, such as: developing and implementing an LCSP to inform acquisition and sustainment phases of the program; developing strategies for managing intellectual property; using systems engineering to identify tradeoffs between life-cycle costs and performance requirements during design and construction; implementing a comprehensive reliability and maintainability engineering program; developing an obsolescence management plan; monitoring the program’s performance against its sustainment requirements and developing strategies to improve operational availability, O&S affordability, maintainability, and reliability, as necessary; and working with a PSM, among other things. Product Support Managers (PSMs) generally work with the acquisition program manager and are tasked with developing and implementing a comprehensive product support strategy for their assigned programs. PSMs are supposed to ensure that a comprehensive product support strategy is developed and implemented. The CNO is the senior military officer of the Department of the Navy, overseeing the Navy’s fleet and NAVSEA, among other organizations. The CNO also has acquisition responsibilities, such as approving a shipbuilding program’s requirements and determining whether to accept delivery of ships from the shipbuilders. The Office of the Chief of Naval Operations (OPNAV) is a collection of offices under the purview of the CNO responsible for various functions necessary for the operation of the Navy. For example, there are divisions within OPNAV that manage the Navy’s budget, logistics, and requirements setting process, among other things. The operational fleet forces (fleet) of the Navy, including operational units and type commands, assume full financial responsibility for operating and maintaining ships. Naval Supply Systems Command provides supply and services support to the Navy by managing supply chains and inventory for Navy aircraft, surface ships, submarines, associated weapons systems, and non-nuclear ordinance stockpiles. NAVSEA is responsible for providing expertise in designing, engineering, building, buying, and maintaining ships, submarines, and combat systems to meet the fleet’s operational requirements. NAVSEA is comprised of directorates and warfare centers that specialize in these areas of expertise. NAVSEA reports to the CNO, but also supports the shipbuilding program offices, and is organized by the following specialties, among others: Naval Sea Systems Command Engineering Directorate (NAVSEA 05) is an engineering command that is comprised of cost estimators, ship designers, systems engineers, and other technical experts. Among other things, this office is responsible for the development of life-cycle cost estimates and systems engineering for ships. Naval Sea Systems Command Acquisition and Commonality Directorate (NAVSEA 06) is a command that brings together personnel dedicated to bridging communication gaps between government and industry, in order to enable cost reductions and commonality throughout the acquisition life cycle. Among other things, this office leads the Navy’s ILA process. Naval Warfare Centers are a group of centers that offer services on a fee-for-service basis, including: obsolescence mitigation, in-service engineering, and data analysis, among many other tasks. Navy Spends Billions to Fix New Ships That Are More Difficult and Costly to Sustain than Shipbuilding Programs Initially Planned Shipbuilding program officials did not identify or mitigate sustainment risks during the acquisition process that subsequently resulted in significant and costly problems for the fleet. During the course of our review, the fleet identified 150 problems that affected multiple ships in a class. These problems resulted in more effort and cost for the fleet in sustainment than expected. In particular, we estimated that the Navy’s fleet has spent or is planning to spend at least $4.2 billion to mitigate and correct approximately 30 percent of these problems beyond what was planned for during the acquisition process. We could not quantify the cost impact of the remaining 70 percent of problems because the Navy was unable to provide data on the cost to correct them. Examples from the SSN 774, LPD 17 Flight I, and LHD 8/LHA 6 ship classes illustrate how shipbuilding program officials did not identify and mitigate sustainment risk during the acquisition process, which resulted in significant and costly maintenance paid for by the fleet once realized. The Fleet Identified over a Hundred Problems with New Ships That Required More Maintenance Effort than Planned for During Acquisition The fleet identified 150 sustainment problems affecting multiple ships in a class that required more sustainment effort than planned for during acquisition, which we verified through Navy data and documentation. Officials in the fleet, such as operators, maintainers, and engineers, reported these problems to us as major class-wide problems requiring more sustainment effort than planned. These problems manifested after ships were delivered and most of these problems have yet to be resolved. Where data were available on the cost to correct the problems, we estimated that the fleet paid at least $4.2 billion and had to perform more onerous maintenance than planned. These problems stemmed from shipbuilding program officials not identifying or mitigating sustainment risks in sustainment planning during the acquisition process, before ships were delivered to the fleet. Figure 4 summarizes the number of problems among multiple ships in the same class that required more sustainment effort than the shipbuilding programs’ had planned. It also reflects the costs associated with fixing these problems for the 30 percent of the problems where we could identify these costs based on available data. According to fleet leadership, these problems contribute to the fleet’s inability to maintain ships at planned cost and schedule, which we have previously found is a significant Navy-wide issue. In part to accommodate this extra effort, the Navy has experienced maintenance delays and has had to defer planned maintenance for ships in operations that the fleet determined was not as urgent as other maintenance needs. For instance these problems have contributed to: nearly 5,300 total days of delays to planned maintenance availabilities since 2012 on ships built during the last 10 years, new ships deferring planned maintenance, and insufficient funding to meet maintenance needs. To generate the list of 150 problems, we interviewed operators and maintainers for the shipbuilding programs in our review and asked them to discuss problems that occurred across multiple ships in the same class. We then verified these problems with available Navy data on system reliability and equipment failures. The list of problems only includes those that stemmed from risks that were not identified, evaluated, or mitigated by the shipbuilding program offices in their sustainment planning during the acquisition process. The list does not include problems that can be attributed to normal wear and tear or problems caused by sailor error. The estimate of $4.2 billion in additional costs to address these problems includes the fleet’s cost to correct or mitigate problems, but excludes costs associated with day-to-day maintenance that the fleet must perform. If the Navy’s fleet chooses to correct a problem, it typically requires the Navy to replace systems on ships that have already been delivered to the fleet or are under contract, which can be a costly undertaking. According to fleet maintenance officials, if a permanent correction is not implemented, the Navy’s operators and maintainers typically have to incorporate a more onerous maintenance approach than expected. The effects of more onerous day-to-day maintenance costs are hard to quantify using available Navy data. For example, the Navy used a brand new toilet and sewage system on the CVN 77 and 78, similar to what is on a commercial aircraft, but increased in scale for a crew of over 4,000 people. To address unexpected and frequent clogging of the system, the Navy has determined that it needs to acid flush the CVN 77 and 78’s sewage system on a regular basis, which is an unplanned maintenance action for the entire service life of the ship. According to fleet maintenance officials, while each acid flush costs about $400,000, the Navy has yet to determine how often and for how many ships this action will need to be repeated, making the full cost impact difficult to quantify. We generally did not include these types of ongoing costs in our calculation. In our cost calculation, we also excluded costs associated with adding sailors to ships to address maintenance gaps because sailors have been added for many reasons, making it difficult to isolate the money spent on sailors to address equipment problems. For instance, we omitted the $225 million that the fleet plans to spend to add sailors to LCS class ships, even though the Navy is taking this action in part, to ensure that the ship’s crew can perform necessary maintenance. The Fleet Experienced Problems as a Result of Risks That Were Not Identified, Evaluated, or Mitigated in Shipbuilding Programs’ Sustainment Planning We determined that the 150 problems identified by the fleet generally fall into three categories: (1) problems maintaining commercial equipment on ships, (2) ship design that did not effectively consider maintainability, and (3) untested sustainment assumptions that turned out to be incorrect after ships were delivered to the fleet. We found that nearly all Navy shipbuilding programs we reviewed experienced problems in each of these three categories, as shown in figure 5. The following examples illustrate each of the three categories of problems: Problems maintaining commercial equipment on ships. Dozens of primarily commercial systems on multiple SSN 774 class submarines are experiencing unexpected failures. During the acquisition process, the Navy based sustainment planning decisions on the assumption that these parts would last for the life of the submarine without the need for any maintenance. According to officials, the Navy did not verify these assumptions and now at least 16 of these systems require scheduled maintenance and several more systems need periodic updates that were not previously planned. As a result, as we have previously found, operators and maintainers have had difficulty obtaining the spare parts, accomplishing this planned maintenance within resource constraints. Ship designs that did not effectively consider maintainability. The Navy used a new design for CVN 77’s stores elevators, which are used to move provisions between decks. However, among other issues, the elevators are too small to fit a standard sized pallet jack. Thus, provisions cannot be loaded or unloaded with a pallet jack or a forklift and must be manually unpacked and stacked by hand on to the elevator. Unloading is further complicated, according to the ship’s crew, because the elevator doors are so small that the average sailor cannot stand up as they enter and exit the elevator. The fleet has mitigated a few of these problems, but a redesign of the elevator would be necessary to fit standard pallets and fully resolve the other problems. Untested sustainment assumptions that turned out to be incorrect after ships were delivered to the fleet. The Navy had originally planned to use a contractor to conduct the majority of LCS maintenance. However, the fleet determined that a heavy reliance on contracted support is inefficient for maintaining and sustaining the LCS and is in the process of establishing maintenance teams comprised of Navy personnel. Since it planned to use contractor support, the LCS shipbuilding program officials stated that they did not purchase the technical documentation necessary to maintain the commercial equipment used on the ship. As a result, fleet engineers told us that they are now attempting to buy and develop the necessary maintenance data, which adds cost and complexity to the maintenance process. The following section highlights four of the 150 problems identified by the fleet. Other examples from among these 150 issues are discussed throughout the report when appropriate to illustrate how the acquisition process contributed to sustainment problems. A full listing of the 150 problems is in a version of this report that is for official use only. Example One—LPD 17 Flight 1 Titanium Piping In an effort to improve sustainment of the LPD 17 class ships, the Navy decided to install titanium piping to carry seawater for firefighting and to cool machinery instead of copper-nickel piping because of its lighter weight and increased durability. However, instead of saving effort in sustainment, these pipes required more maintenance effort than planned and, in many cases, eventually had to be replaced. Early in the acquisition process, the Navy studied this decision and discovered that unlike copper-nickel piping, titanium piping carrying seawater is susceptible to “biofouling”—meaning sea life such as shellfish grow inside the pipes—as shown in figure 6. To prevent biofouling, Navy engineers determined that a chlorination system—which adds chlorine to seawater entering the ship in order to kill biological material in the water—and a dechlorination system—which removes the chlorine before the seawater is dumped from the ship— would be needed and included specifications for the shipbuilder to install these systems. Then, according to shipbuilding program officials, after the ship was on contract, the shipbuilder reported to the Navy that it could not find suitable chlorination and dechlorination systems. The program office decided to proceed with ship construction absent these systems and evaluate the extent of the biofouling problem after ship delivery. We reviewed the LPD 17 program’s sustainment planning documents and found that a discussion of this sustainment risk was not included in any of the maintenance planning documents, and, according to the fleet, this risk was not communicated to the Navy’s maintenance organizations. In July 2009, about one year after the lead ship was provided to the fleet, Navy operators and maintainers began to notice biofouling in the piping system. Biofouling degraded the functionality of a number of other systems on the ship that depend on the water delivered by the piping system, resulting in overheating of main and ship service engines and loss of electric power generation, among other problems. To address these and related issues across the LPD 17 class, the Navy’s fleet spent at least $250 million to: (1) buy and install new copper-nickel piping that is now costlier, heavier, and not as durable as titanium; (2) install chlorination systems that were later found to be unreliable, requiring significant maintenance; and (3) conduct unplanned maintenance and replace systems that broke due to shellfish contamination, among other interventions. Example Two—SSN 774 Special Hull Treatment The Navy’s attack submarines utilize a special covering on the hull. However, as shown in figure 7, portions of the hull-covering have de- bonded from the hull resulting in additional maintenance requirements during scheduled availabilities. Shipbuilding program officials told us that, during the acquisition process, they did not analyze how long the special hull treatment would last even though it is a critical technology. According to the program office, they now have identified the root cause and have continuously conducted engineering analysis to monitor and improve the material and construction processes. Due to the 5 to 6 year process of building a submarine, the time from identification to proven success can be 8 to 10 years, which is a long time to wait to know if a potential solution works in operations. However, in the meantime, the shipbuilding program has continued to deliver submarines to the fleet without knowing how long the special hull treatment will adhere to the vessel. As a result, maintainers cannot effectively plan for special hull treatment replacements in advance and, instead, are replacing material as needed. Performing timely and necessary maintenance is further complicated because it takes up to two years to receive this material after the Navy orders it from the manufacturer. Currently, Navy maintainers are budgeting $735 million to address the missing hull treatment on 11 of the 14 submarines constructed prior to implementing the potential solution. Example Three—LHD 8 and LHA 6 Machinery Control System To enable reduced crew sizes and sustainment costs, the Navy chose to use an automated machinery control system on LHD 8 and LHA 6. Sailors describe the machinery control system as a vital software-based system that controls the operation of 92 percent of shipboard systems. The Navy initially sought to purchase a highly-automated commercial system that would perform tasks previously completed by the ship’s crew. However, according to the shipbuilding program, during the acquisition process, it verified reliability testing conducted by the manufacturer of the system. At the end of the shipbuilding process, the Navy discovered that this system required more maintenance and sustainment effort than planned. Specifically, the Navy’s Board of Inspection and Survey—the organization that inspects ships prior to delivery—discovered problems with this system on LHD 8 in March 2009. The Board of Inspection and Survey identified false alarms and a lack of technical documentation as a serious defect. Specifically, the test report found that the system’s spurious and numerous alarms created an environment whereby the ship’s sailors would be conditioned to ignore alarms and that more sailors would be needed to monitor the ship’s systems. Nevertheless, the CNO decided to take delivery of the ship and the shipbuilding program did not correct these problems prior to providing the ship to the fleet. Additional problems emerged on LHD 8’s first deployment, such as overheating that led to failure of the electrical distribution system resulting in loss of power on multiple occasions. However, the technical data provided by the manufacturer, according to Navy engineers, was insufficient for the sailors to operate, troubleshoot, and repair the system. Further, according to ship engineers and the shipbuilding program, 9 of 28 critical components within the machinery control system hardware were obsolete when LHA 6 was delivered to the Navy. As a result, fleet officials told us that it has been difficult to obtain replacement parts. The Navy has spent over $90 million to repair the software and replace key components of the system on LHD 8, LHA 6, and LHA 7. Example Four—LPD 17 Flight I Knuckleboom Crane The LPD 17 Flight I knuckleboom crane carries boats and cargo (such as ammunition) from the ship to the water and back again, and is pictured below in figure 8. However, according to Navy reliability data, this system only works 30 percent of the time it is supposed to and has been difficult for sailors to use and maintain since the lead ship was delivered in 2005, nearly 15 years ago. There are a number of challenges in sustaining this crane that the Navy did not identify or sufficiently mitigate during the acquisition process. For example, the fleet does not have the necessary technical data to operate and fix the system, spare parts can be difficult to find or take many months to obtain, and pieces of the system are obsolete. According to fleet officials who use the data, the shipbuilding program office did not acquire sufficient technical data nor conduct sustainment planning for this large and complicated crane primarily because they planned to contract for the maintenance of the entire ship, including this system. The Navy subsequently discovered that contracting for the maintenance of the whole ship was cost prohibitive and maintenance responsibility was transferred back to the Navy. However, because there had not been adequate sustainment planning, the fleet did not have necessary resources, such as technical data, to effectively maintain the system. Additionally, as the fleet has been developing the capacity to maintain this crane, the shipbuilding program office continues to accept cranes with unmitigated risks leading to unplanned fleet effort. For example, across the eleven LPD 17 Flight I ships that have been delivered, there are four different versions of the crane, which further complicates maintainability because it increases the types of spare parts needed and the knowledge required of the sailors to fix the system. Specifically, officials stated that sailors who learned to maintain a crane on one ship cannot transfer all of their knowledge to other ships in the class. Due to the numerous sustainment challenges the fleet has experienced with this crane on LPD 17 Flight I, LPD 17 program officials told us that the Navy has since revised its new construction crane requirements for LPD 17 Flight II. According to the shipbuilding program office, these requirements allow the shipbuilder to use a more standard crane, which will be easier to sustain. While we could not calculate the added costs of maintaining this crane, we found that the Navy has spent over $10 million on the following actions: (1) contracting with the original equipment manufacturer for repairs, (2) replacing key components of the system, and (3) making changes to improve the system. DOD Policy for Shipbuilding Sustainment Requirements Results in Inadequate Information for Acquisition Decisions and Reporting That Is Misleading DOD policy that the Navy uses to set sustainment requirements does not capture factors that affect whether ships are reliable and maintainable. This results in shipbuilding programs having ineffective sustainment requirements that do not support sound acquisition decisions. When sustainment requirements are used to inform acquisition decisions, they can help ensure that shipbuilding programs design and build reliable ships that can be effectively sustained within planned costs. The effectiveness of a shipbuilding program’s sustainment requirements depends on how the requirements are set, used, and reported. Setting the sustainment requirements. We found that weaknesses with specific portions of DOD’s requirements policy resulted in the Navy setting sustainment requirements that are poorly defined and not representative of the availability of the ship during operations and sustainment. Using the sustainment requirements. To achieve the requirements, shipbuilding programs need to incorporate the requirements into decisions made throughout the acquisition process, such as developing the ship design. Due to problems setting the requirements, shipbuilding programs cannot incorporate the sustainment requirements into acquisition decisions. Reporting on the sustainment requirements. Statute requires that programs report on the status of these requirements on a regular basis. However, the Navy’s reporting on these requirements is misleading because it is based on the Navy’s deficient sustainment requirements and it does not reflect the fleet’s experience. Navy Ship Sustainment Requirements Reflect Weaknesses with How DOD Policy Defines Requirements for Ships The Navy sets sustainment requirements based on definitions for ships established by DOD policy, called the Joint Capabilities Integration and Development System, but the shipbuilding programs’ requirements are not robust even when they follow DOD policy. This is because the definitions for ship sustainment requirements in DOD requirements setting policy do not capture all factors that reduce the ability of ships to achieve their missions. For example, the definitions of operational and materiel availability in this policy exclude key factors and failures that reduce ship availability, such as catastrophic failures of mission-critical systems and unplanned maintenance. DOD policy states that the purpose of sustainment requirements is to ensure ships work when expected and are available when needed. But because the definitions of these requirements for ships do not capture all factors that can influence operational or materiel availability, the specific definitions for setting sustainment requirements for ships do not support the achievement of this goal. DOD’s requirements setting policy has designated these metrics to be key performance requirements since 2007, which means that they are one of a small number of mandated critical requirements that a weapon system must demonstrate. Without a definition for ship sustainment requirements in DOD policy that accounts for all factors that make Navy ships unavailable for operations, Navy shipbuilding programs cannot reasonably ensure that they are setting sustainment requirements that will result in reliable, maintainable, and available ships. In 2015, DOD added guidance to its policy that instructed shipbuilding programs to establish operational and materiel availability requirements based on the extent to which ships are expected to experience major failures, referred to as category 4 casualty reports. The fleet writes casualty reports when there are significant equipment failures that contribute to the ship’s inability to perform its missions. There are three categories of casualty reports (2, 3, and 4), with category 4 being the most severe. According to Navy guidance, category 3 and 4 casualty reports indicate degradation to critical mission capability that needs immediate repair, while category 2 reports contain failures that are important to the fleet but do not affect the ship’s core missions. In particular, DOD policy was updated to define operational and materiel availability for ships as follows: Operational availability (work when expected) is the percentage of time an operationally deployed ship is not in a category 4 casualty report state over a given operating period. The Navy typically sets this requirement at approximately 80 percent for shipbuilding programs. Materiel availability (ready when needed) is the portion of a ship class available for tasking. Ships are typically not available for tasking when in a planned maintenance availability or have an open category 4 casualty report. The Navy followed DOD requirements setting policy by establishing these key performance parameters for the four shipbuilding programs we reviewed that established requirements since fiscal year 2015—SSBN 826, FFG(X), DDG 51 Flight III, and LPD 17 Flight II. Prior to 2015, there were no ship-specific definitions in DOD requirements setting policy. Shipbuilding programs that set requirements prior to 2015 have generally adapted the definitions in JCIDS for calculating and reporting operational and materiel availability, which is why we include examples from these programs as appropriate The following two sections discuss shortfalls with DOD’s policy for setting sustainment requirements for Navy shipbuilding programs. Setting Operational Availability Requirements for Shipbuilding Programs DOD’s definition of operational availability for ships in its policy is problematic because it defines operational availability: (1) using category 4 casualty reports and (2) for the entire ship with a single metric. As a result, the operational availability requirement does not capture all critical failures that reduce a ship’s ability to perform mission-critical tasks. Category 4 casualty reports. DOD’s operational availability definition for ships counts only the most severe casualty reports—category 4. The definition excludes category 3 casualty reports, which also represent a severe degradation to the Navy’s primary missions. According to several fleet officials, category 4 casualty reports are typically used only in rare instances when the entire ship is out of commission. Fleet officials added that category 3 casualty reports can also represent severe mission-critical casualties that affect the ability of the ship to perform primary missions. In addition, the Navy’s categorization of casualty reports tends to be subjective or based on other factors than the severity of the defect, such as, according to maintenance officials, communicating a maintenance priority. In other words, there are additional deficiencies that could be mission-critical that may not be captured by category 3 or 4 casualty reports. Of the 11 ship classes in our review, six have delivered ships and have casualty report data available. We reviewed Navy casualty report data for 18 ships from these six ship classes and found that all of these ships had near-perfect operational availability when using only category 4 casualty reports. However, when we calculated operational availability using category 3 casualty reports, we found that 14 of these 18 ships fell short of their operational availability targets. Table 2 summarizes the category 3 and 4 casualty reports during two LCS missions as an example of how major failures are captured as category 3, and not category 4, equipment casualties. Therefore, by using category 4 casualty reports to define operational availability, the Navy is developing a requirement that does not accurately account for all ship failures that affect whether or not a ship works as expected. Setting operational availability at a whole ship level. DOD requirements setting policy specifies that shipbuilding programs should establish a single metric for the entire ship. However, when set at the ship level, the operational availability requirement is not effective at capturing the probability of whether or not a ship and its systems will work as expected. This is because ships are comprised of hundreds of systems that are of varying importance to achieving missions. For example, a ship may have an air-defense mission that requires a select group of systems—such as an air-search radar and a missile system—to work together to achieve the mission. However, a ship-level requirement is set using a single metric for the entire ship, which does not account for the fact that some systems are critical to achieving a ship’s primary missions while some systems are not as critical. Further, a ship level requirement is difficult to calculate. According to a Naval Sea Systems Command operational availability manual, it is improbable that the operational availability of hundreds of complex systems within a ship can be accurately calculated and represented in a ship level requirement. Figure 9 below illustrates how setting requirements pursuant to DOD requirements setting policy resulted in an operational availability requirement for the FFG(X) program that the fleet considers unacceptable. According to Navy handbooks and manuals on using operational availability during ship design, the operational availability requirement is a more effective input for acquisition decisions when it is set at the mission level. Since ships have multiple missions, this would result in multiple operational availability requirements instead of a single ship-level requirement. The Navy’s operational availability handbooks and manuals endorse this approach because a mission-level requirement is focused on a smaller group of systems that support the mission and, therefore, allows the Navy to prioritize availability for these key systems. Setting operational availability requirements by mission area would provide shipbuilding programs with information about how to identify and prioritize key systems for additional reliability analysis or sustainment planning to ensure that they will be sufficiently available to meet mission needs. Also, even though this would likely result in several operational availability requirements for each ship class, it would simplify the calculation of these requirements, which could make them more helpful inputs for acquisition decisions. Setting Materiel Availability Requirements for Shipbuilding Programs We found that DOD’s definition of materiel availability for Navy ship classes in its requirements setting policy does not ensure that ships will be ready when needed—the purpose of the materiel availability requirement. This is because DOD requirements setting policy for ships does not specifically account for other factors that affect materiel availability—such as unplanned maintenance, unplanned losses, and training—during which times ships may not be available for operations. Unplanned maintenance. Unplanned maintenance can occur when planned ship maintenance lasts longer than expected or a mission- critical failure occurs during deployment that needs immediate attention. As our prior work has found, Navy ships experience significant levels of unplanned maintenance. For example, from fiscal year 2012 through fiscal year 2018, the Navy has reported over 3,900 days of unplanned maintenance across the ships we reviewed. Unplanned losses. Unplanned losses are instances when a ship is out of commission for an extended length of time due to severe damage or when a vessel was not prioritized for maintenance. For example, we have previously reported that due to heavy shipyard workload, some submarines are waiting significantly longer than planned—in some cases several months or years—to enter maintenance periods. Training. The Navy also conducts several training periods, and the DOD requirements setting policy does not address whether or not a ship is considered available or unavailable during these training periods. Six of the 11 shipbuilding programs we reviewed developed their program requirements since DOD made sustainment requirements mandatory in 2007. One of these six programs—LHA 6—did not established a materiel availability requirement as required by DOD requirements setting policy. LHA program officials told us that materiel availability does not apply to ships, which is not reflected in DOD requirements setting policy. Four shipbuilding programs—DDG 51 Flight III, LPD 17 Flight II, FFG(X), and LCS—developed materiel availability requirements that generally align with DOD’s requirements setting policy and, as such, do not specifically account for unplanned maintenance, unplanned losses, and training. The remaining shipbuilding program—SSBN 826—went above and beyond DOD requirements setting policy by incorporating these additional areas that could affect materiel availability. Program officials stated that sustainment requirements are more critical to achieving the SSBN 826’s missions than other shipbuilding programs. However, DOD and Navy guidance clearly state that materiel availability is a mandatory critical requirement for all programs. Since DOD’s definition for materiel availability does not include all factors that could result in a ship being unavailable for operations, shipbuilding programs cannot ensure that ships will be ready when needed. Sustainment Requirements Are Inadequate to Support Well-Informed Decisions during the Acquisition Process Because of how DOD policy defines sustainment requirements for ships, these requirements do not provide the information needed to support acquisition decisions. In particular, the Navy’s sustainment requirements developed according to DOD policy rarely provide adequate information about how reliable, available, and maintainable ships need to be, which is necessary to support well-informed decisions pertaining to ship concept development, design, and construction. For example, during the acquisition process, shipbuilding program offices make decisions that transform top-level requirements—like operational and materiel availability—into detailed, low-level requirements that can be achieved with available resources. We found that ongoing and new shipbuilding programs continue to make acquisition decisions that influence sustainment without the information that could be provided by better- defined sustainment requirements. Since shipbuilding programs cannot use these requirements to inform acquisition decisions, they cannot ensure that ships will be sufficiently reliable and available. The following two sections discuss the Navy’s issues with using sustainment requirements when making acquisition decisions for its shipbuilding programs. Using Operational Availability Requirements in Acquisition Decisions The Navy’s operational availability requirements for ships—which follow the DOD policy discussed above—do not provide adequate information to support acquisition decisions that affect whether or not ships are reliable enough to meet their missions. For example, in January 2020, we found that engineers can use a variety of activities when designing weapon systems to increase reliability to meet requirements, such as conducting failure analysis and adding redundant systems. In order for these engineering decisions to be successful, the requirements that inform the process must be firm, well-defined, feasible, and affordable. However, when the operational availability requirements do not adequately describe the needed reliability and maintainability for key systems—as is the case for most of the shipbuilding programs we reviewed—Navy engineers cannot ensure that the ship’s design supports the program’s top-level operational availability requirement. Further, they cannot identify aspects of the design that could put the requirement at risk. Instead of using the operational availability requirement to inform decisions across all key ship systems, Navy ship engineers told us that they interpret the requirement to only apply to catastrophic failures that put the entire ship out of commission. Therefore, in practice, shipbuilding program officials told us that they only apply this requirement to systems that the ship needs to get underway, such as the main engines and propellers. As such, shipbuilding programs are making engineering decisions during the acquisition process for many mission-critical systems, such as radars, weapons, and systems necessary for launching and recovering aircraft, without understanding how often these systems need to work to achieve key missions. This means the operational availability requirement only applies to the bare minimum of ship systems needed to get underway rather than the full complement of systems needed to meet the ship’s missions. For instance, LPD 17 Flight I ships can often sail away and are considered operationally available even as key systems—such as the knuckleboom crane, davit, air conditioning, and potable water systems among others—work less than 75 percent of the time the ship is at sea, according to fleet databases that track system failures. By interpreting the requirement to only focus on systems needed to move the ship and not accounting for other mission critical systems, shipbuilding programs cannot ensure that all critical systems needed to meet missions will work as expected. In addition, since shipbuilding programs have a ship-level operational availability requirement and interpret this requirement to focus on systems needed to get ships underway, they have not consistently leveraged available data on various ship systems when making engineering and ship design decisions. Navy sustainment experts told us that shipbuilding programs rarely use data on the actual availability of ship systems. If the requirement was set at the mission-level and focused on key systems, the data could show whether or not planned systems, already operating in the fleet, are available enough to meet requirements. Then, if the data shows that these systems are not sufficiently available, shipbuilding programs could make investments in improving the availability, such as improving supply support, making the system more reliable, or adding redundancy. Since shipbuilding programs cannot use operational availability requirements to make informed acquisition decisions, they are at risk of continuing to deliver ships to the fleet that are not as reliable and sustainable as needed. Using Materiel Availability Requirements in Acquisition Decisions Of the five shipbuilding programs we reviewed that had established materiel availability requirements, we found that only one program has a requirement that provides adequate information for acquisition decisions. In particular, the SSBN 826 program’s materiel availability requirement has been a key input in establishing the submarine class’ planned maintenance schedules and procedures. Shipbuilding program officials told us they are using the maintenance period length determined by the materiel availability requirement to inform acquisition decisions—such as adjusting the submarine’s design to facilitate timely maintenance. For instance, the SSBN 826 shipbuilding program assessed the potential effect of new technology on the amount of maintenance that the submarine is planned to undergo. In doing so, the shipbuilding program officials believe that, if the new technology works as planned, the SSBN 826 class will meet the same presence requirement as its antecedent class with two fewer submarines. While this concept is a good example of how materiel availability can be used during the acquisition process, it is too early to know if the Navy’s plan will work for this class of submarines and fleet officials told us that they have doubts that the Navy can achieve this goal as planned. Officials from other shipbuilding program offices told us that they are not using the materiel availability requirement to inform maintenance decisions. Further, according to these shipbuilding program offices, the materiel availability requirements do not connect with the ship class’ planned maintenance schedules and, therefore, they do not make decisions to ensure that planned maintenance can be achieved within specific time frames. Program officials from several of these programs stated that the materiel availability requirement is not critical to performance goals, and, as such, it is not a priority to achieve this requirement. Without improving how the Navy defines and uses materiel availability requirements, shipbuilding programs are missing opportunities to make informed acquisition decisions about how ships are maintained and, therefore, cannot ensure that ships are available for operations when needed. Reported Sustainment Requirements Do Not Reflect the Fleet’s Experience The Navy’s reports to Congress are misleading because they do not reflect all of the failures and factors that reduce ship operational and materiel availability once ships are in the fleet. Shipbuilding programs report all key requirements in their Selected Acquisition Reports to Congress, including operational and materiel availability. According to DOD guidance for executing Selected Acquisition Reports, DOD program offices should provide accurate information to Congress to aid in determining if the program is meeting its key requirements. We reviewed the December 2018 Selected Acquisition Reports for the five shipbuilding programs that reported one or both of these sustainment requirements to Congress. We found that the Navy reported that these shipbuilding programs were meeting or surpassing their sustainment requirements. However, based on our analysis of data on mission-critical failures after ships were delivered, we found failures that would prevent these ships from conducting critical missions. Hence, the Navy’s reports to Congress do not reflect the actual availability of ships in the fleet. As a result, Congress does not have full insight into whether shipbuilding programs are on track to meet their operational and materiel availability requirements. The following two sections further discuss the Navy’s issues with reporting sustainment requirements for its shipbuilding programs. Reporting on Operational Availability Requirements to Congress We found three out of seven shipbuilding programs report on operational availability in their Selected Acquisition Reports. These three programs all stated that they were meeting or exceeding their requirements, but these reports often did not match the fleet’s experience. For example: For one vessel class, the Navy reported that it was exceeding its operational availability goal by over 10 percent. At the same time, however, several mission critical systems are unreliable. Officials from the fleet stated that critical ship equipment is consistently failing. The Navy is reporting that another ship class—that has yet to finish construction —is exceeding its operational availability target by 5 percent. This ship class has already experienced several catastrophic failures that limit its ability to conduct primary missions during its limited at-sea periods. These examples demonstrate how reporting based on a ship-level operational availability requirement does not provide insight into reliability and maintainability problems that the fleet is experiencing and that prevent ships from meeting missions. Consequently, Congress is not receiving accurate information on the results of its investments and the sustainment problems the fleet is experiencing. Reporting on Materiel Availability Requirements to Congress We found that two of the Navy’s shipbuilding programs we reviewed currently report materiel availability in Selected Acquisition Reports to Congress. One other shipbuilding program that has materiel availability as a key requirement in its approved baseline does not report this requirement, contrary to DOD guidance. For example, the LCS shipbuilding program indicates that it is meeting the requirement despite evidence of issues with materiel availability. The Navy’s Selected Acquisition Report for the LCS states that the program is meeting its materiel availability requirement even though internal DOD reports state that the LCS’ materiel availability is significantly below its requirement. Further, fleet officials stated they are worried the maintenance workload required for the LCS class ships may result in additional unplanned maintenance delays that further reduce materiel availability. The Navy has also chosen to take steps that will reduce the materiel availability of the ship class throughout the ship class’ service life, such as assigning the first four ships as test ships, making one of every four LCS a training ship on a rotating basis, and increasing planned maintenance days, among other things. Since several of the Navy’s shipbuilding programs do not report information to Congress on this critical requirement, Congress does not have insight into whether or not ships are as available as intended. Shipbuilding Programs Do Not Consistently Identify and Evaluate Sustainment Costs and Risks in Acquisition Documents The shipbuilding programs included in our review did not consistently conduct effective sustainment planning when developing three key acquisition documents: life-cycle cost estimates, life-cycle sustainment plans (LCSPs), and independent logistics assessments (ILAs). According to DOD and Navy acquisition policy, these documents, along with other documents, help programs ensure the ships they are acquiring can be sustained affordably and adequately over their life cycle. However, for the shipbuilding programs in our review, we found that these documents did not provide a thorough assessment of the sustainment implications and risks for many of the programs’ acquisition decisions. Specifically, we found that: 1) O&S costs in shipbuilding programs’ life-cycle cost estimates did not account for major sustainment risks and grew significantly; 2) LCSPs rarely included information needed to demonstrate ships could reliably meet sustainment requirements at an affordable cost; and 3) ILAs did not consistently identify major sustainment risks that were subsequently realized by the fleet. Because shipbuilding programs are not effectively using these acquisition documents to plan for sustainment, they are passing unmitigated sustainment risks on to the fleet. O&S Cost Estimates Have Significantly Increased, Largely Because Life- Cycle Cost Estimates Did Not Account for Sustainment Risks We found that shipbuilding programs’ current estimates of O&S cost are significantly higher than initial estimates. This is largely because the Navy cost estimators based their initial estimates for the shipbuilding programs in our review on unproven sustainment assumptions without assessing the potential cost risk of the assumptions. According to shipbuilding program officials, O&S cost estimates grew after shipbuilding programs revised their sustainment assumptions, such as by increasing the number of crew required to operate and maintain the ships or by changing the level of maintenance needed for various ship systems. We compared programs’ initial life-cycle cost estimates for the six shipbuilding programs in our scope that had available estimates to current cost estimates that were updated after programs delivered ships to the fleet. As shown in table 3, we found that the shipbuilding programs’ estimates of O&S costs increased by over $130 billion from the initial estimate to the most recent estimate. Navy cost estimators stated that up to 20 percent, $26 billion, of the cost estimate growth could be accounted for by process changes that resulted in including more indirect costs, such as health and child care for sailors, into O&S estimates. Further, we adjusted our analysis to account for any program quantity changes over time. Even accounting for these changes, the Navy still experienced over $100 billion in O&S cost growth. The O&S cost growth for these six shipbuilding programs is likely higher than the $130 billion that we calculated in table 3. This is because the Navy has not updated these estimates to reflect actual O&S costs for several of the ship classes. For example, the LCS program, in its initial O&S cost estimate, projected $7.1 million (in fiscal year 2019 dollars) per year per hull for maintenance. However, thus far, the average LCS seaframe currently costs $21 million (in fiscal year 2019 dollars) per hull per year to maintain—an increase of over $13 billion if these higher than planned maintenance costs continue over the life of the ship class. We found that the shipbuilding programs we reviewed underestimated initial O&S costs, largely because cost estimators used unproven O&S assumptions without assessing the sensitivity of those assumptions on potential cost growth, as discussed below. Unproven O&S Assumptions The O&S costs estimates we reviewed had grown primarily because initial unproven assumptions turned out to be optimistic. O&S cost estimates for four of the six shipbuilding programs we reviewed were based on a Navy-wide effort that began in the early 2000s to reduce crew sizes on Navy ships and lower O&S costs by, among other things, replacing some sailors with automated systems. We found that cost estimators used the shipbuilding program offices’ unverified assumptions regarding crew size to develop the initial O&S estimate for four of these six programs. Over time, the Navy found that the automated systems were not as reliable as planned and, therefore, reduced crewing levels were not realistic. To address this and other issues, the Navy added sailors back on to ships—resulting in increases in O&S cost estimates. For example, cost estimators for the CVN 78 class initially estimated a 15 to 23 percent decrease in crewing levels compared to the previous class of carriers in order to create O&S savings. However, the Navy is now in the process of adding crew back on to the ship, even before its initial deployment, thereby contributing to increased O&S cost estimates, as shown in Table 4. Similarly, DDG 1000, LCS, and LPD 17 program officials also reported that increasing crew sizes was a major contributor to higher sustainment costs for these programs. Further, the shipbuilding programs we reviewed made assumptions based on unproven initiatives, in conjunction with reducing crew sizes that ended up having a greater effect on the cost of maintaining ships than initially estimated. For example, for four ship classes—SSN 774, DDG 1000, LPD 17 Flight I, and LCS—the Navy originally planned to use a maintenance initiative called performance-based logistics, which called for the use of contractors to conduct maintenance instead of sailors on board the ships. In 2001, DOD policy recommended that all weapon systems use performance-based logistics and Navy shipbuilding programs subsequently anticipated that this strategy would reduce maintenance costs. Based on our review of shipbuilding program cost estimates, we found that Navy cost estimators included cost savings from these new and unproven approaches—assuming that they would work as expected. Shipbuilding program officials stated that the Navy has now largely abandoned this approach after attempting to contract for performance-based logistics and discovering that it was much more costly than planned. Another initiative that began in the early 2000s involved the Navy using more shipbuilder-provided commercially-bought systems on ships rather than systems the Navy developed and provided to the ship. However, maintaining commercial systems has been more expensive than anticipated for a variety of reasons, such as systems becoming obsolete and challenges acquiring manufacturer support. For example, the SSN 774 shipbuilding program made an effort to use commercial equipment that it assumed would never need repair or replacement—meaning that these parts would last the life of the submarine—without evaluating whether these parts actually had no repair needs. Further, SSN 774 program officials told us that the program office did not plan for the Navy to support many of the submarine’s commercial components because they initially planned to contract for logistics support. In all, the SSN 774 program asserted that over 4,000 parts on the submarine class would not need maintenance for the duration of the submarine’s life. However, since the submarines have been operating, many of these parts are failing, which has created unanticipated expenses. For example, Navy maintenance officials stated that they are planning to pay $360 million over the next 12 years to maintain a part of the propulsion system that it wrongly assumed would not need any maintenance at the time O&S costs were established. No Risk and Sensitivity Analyses for Key Assumptions A key reason that shipbuilding programs underestimated O&S costs is that the Navy’s cost estimators did not test the sensitivity of key O&S cost assumptions to quantify risks. According to DOD and Navy guidance and GAO-identified cost estimating best practices, cost estimates should include risk and sensitivity analyses to understand how changing program assumptions can affect cost—including O&S costs. However, for the six cost estimates that we reviewed, the Navy did not conduct risk and sensitivity analysis on key sustainment assumptions, such as unproven crewing and maintenance assumptions. The Navy’s cost estimators told us that they typically only conduct sensitivity analysis on the acquisition portion of a life-cycle cost estimate and not the O&S portion of the estimate. Instead, cost estimators told us that they use shipbuilding program office assumptions about the crew and how the ship class will operate as defined requirements that will not change. However, as discussed throughout this report, we found numerous instances in which incorrect maintenance assumptions resulted in billions of dollars of O&S cost growth. As a result, Navy’s cost estimators had reduced estimated O&S costs to reflect the programs’ presumed sustainment efficiencies without accounting for and quantifying the corresponding risk inherent in these assumptions. As such, in several cases, shipbuilding programs had optimistic estimates of O&S cost that later grew when unproven assumptions did not pan out as anticipated. According to shipbuilding program officials, their programs experienced significant O&S cost growth because the initial cost estimate did not sufficiently account for the risk of major changes to the program, such as revisions to the shipbuilding program’s assumptions about sustainment, that were realized once ships were provided to the fleet. For example, on the shipbuilding programs that adopted reduced crewing initiatives, Navy cost estimators reduced O&S costs due to fewer planned sailors on board, but did not determine how the O&S costs would be affected if automation did not achieve its intended efficiencies and the Navy had to add additional sailors to the crew. If the Navy’s cost estimators had conducted risk and sensitivity analyses of the O&S costs early in the acquisition process, shipbuilding programs could have had better insight into how much their O&S costs might increase if the key sustainment assumptions were not correct. Such insight into the potential sustainment cost impact could help shipbuilding programs identify the assumptions most likely to drive O&S cost growth. In turn, this information could help shipbuilding programs justify allocating additional resources during the acquisition process to ensure these sustainment assumptions are achieved, such as investing in additional testing to ensure the reliability of automated systems needed to reduce crewing levels. See figure 10 for an example of how unproven assumptions that were not evaluated using risk and sensitivity analyses led to optimistic O&S cost estimates for the DDG 1000 program. Navy officials told us that they are considering several pilot programs to improve cost estimators’ ability to conduct sensitivity analyses of maintenance costs, but have yet to provide details on these programs or the time frame for implementing them. While it is not possible for shipbuilding programs to predict future O&S costs with complete certainty, risk and sensitivity analyses could help shipbuilding programs’ better identify potential drivers of cost growth. In the absence of this cost analysis, shipbuilding programs will lack a clear assessment of the range of O&S costs their ships may require after they are delivered to the fleet. Additionally, without this O&S cost information, shipbuilding programs cannot provide Navy leadership with full insight into the range of resources that will potentially be required to sustain new ship classes over their lifetime or support recommendations for additional resources during acquisition to achieve sustainment assumptions. Shipbuilding Programs’ LCSPs Rarely Include Information Described in Policy and Guidance and Are Not Used to Inform Acquisition Decisions Five of the eleven shipbuilding programs we reviewed do not have LCSPs, and we found that the six programs that have LCPS do not use them to inform acquisition decisions that could help ensure ships are sustainable at an affordable cost. As of a September 2011 policy memorandum, DOD guidance requires every acquisition program we reviewed to have a LCSP. Shipbuilding programs, according to DOD acquisition policy, should develop and maintain LCSPs beginning at Milestone A, which is early in the acquisition process. According to DOD guidance, these plans should be the basis for all of the programs’ sustainment efforts. In particular, shipbuilding programs’ LCSPs should include information that demonstrates how a ship class can be affordably operated and maintained while meeting its sustainment requirements. To do so, DOD guidance describes that shipbuilding programs should use LCSPs to establish connections between life-cycle costs, reliability requirements, and crew size estimates, and identify and address sustainment issues, among other things. With nearly half of its shipbuilding programs not having completed LCSPs, the Navy is making acquisitions decisions without the context of a comprehensive sustainment planning document to help identify and mitigate the sustainment effect of its decisions. Figure 11 provides an example of a sustainment issue with the CVN 78 advanced arresting gear, which was identified during testing but not addressed in a LCSP. Officials from two of the five shipbuilding programs that do not have LCSPs stated that they had drafts of the plan, in some cases for several years, which leadership has yet to approve. In another case, shipbuilding program officials stated that they were not required to complete an LCSP even though DOD’s 2011 guidance directed them to create these plans immediately. For the six shipbuilding programs that had LCSPs, we found several challenges with how the programs develop and use these documents. Specifically, we found that the LCSPs: (1) rarely included a business case analysis, as required, that analyzed the relationship between life-cycle costs, reliability requirements, and crew size estimates; and (2) rarely identified and addressed sustainment issues in line with guidance. No Business Case Analyses We found that none of the six LCSPs we reviewed contained business case analyses as required by DOD acquisition policy and guidance. According to DOD’s acquisition policy, an acquisition program’s LCSP should include a business case analysis annex, which should contain relevant assumptions, constraints, and analyses used to develop the product support strategy to the LCSP. According to DOD’s guidance for PSMs, who are responsible for developing and maintaining LCSPs, acquisition programs should use a product support business case analysis to help establish a product support package that balances sustainment costs against required sustainment outcomes. As such, the LCSP’s business case analysis is a tool to help programs assess the costs, benefits, and risks of key acquisition decisions from a sustainment perspective. Additionally, the LCSP should contain information on the activities needed to achieve the sustainment key performance parameters and a discussion of how much funding is required for those efforts. For example, Navy leadership approved the LCSP for FFG(X) in March 2019 even though the plan lacked the required sustainment business case analysis. Instead, the FFG(X) LCSP contains ship-level sustainment requirements and O&S cost information from the program’s life-cycle cost estimate, but no accompanying business case analysis demonstrating how the desired sustainment requirements (operational and materiel availability) can be achieved within these costs. As another example, several ship classes were designed with highly automated systems to enable reduced crew sizes and lower O&S costs, such as the LHD 8/LHA 6 machinery control system discussed earlier in this report. However, the LCSPs for these programs did not analyze the extent to which meeting O&S estimates and sustainment requirements were reliant on the reliability of these automated systems and the risks associated with using automation. Without connecting life-cycle costs to key sustainment factors such as reliability and crew size estimates, the Navy will not know if its sustainment planning is achievable within cost constraints until ships are provided to the fleet and have been operated for a significant period of time. We have previously found that it is often too expensive or time- consuming to make meaningful changes to the ship at this point in the shipbuilding process. Limited Identification and Mitigation of Sustainment Issues LCSPs we reviewed rarely identified and proposed a plan to address programs’ sustainment issues, as described by guidance. According to DOD’s LCSP guidance, acquisition programs should assess their progress, challenges, and corrective actions when developing a plan to sustain a ship class. Two shipbuilding programs identified some sustainment risks and only one of the six LCSPs included plans for mitigating or correcting these risks. In the absence of proactively identifying and mitigating sustainment risks in the LCSP during the acquisition process, as described by guidance, we found that the Navy discovered and mitigated many of its sustainment challenges only after ships were delivered to the fleet. Without creating LCSPs that identify sustainment risks and proposing a plan to mitigate these risks, the Navy cannot ensure that it is making acquisition decisions that support ship sustainment. Two examples of significant sustainment risks that were experienced by nearly all of the programs we reviewed, but not identified or mitigated in LCSPs are: (1) insufficient technical data and (2) the use of performance- based logistics. Technical data. The LCSPs we reviewed that included an intellectual property strategy, as required by DOD acquisition policy during the operations and support phase, did not consistently address the full spectrum of potential intellectual property related issues, such as attaining intellectual property needed to repair and replace ship systems. According to DOD’s acquisition policy, shipbuilding programs should document the intellectual property strategy initially in the acquisition strategy and later in the LCSP to assess technical data needs and determine what intellectual property deliverables and license rights the program needs to acquire from contractors. Nearly all of the LCSPs we reviewed stated, in general terms, that the Navy would obtain the technical data to which it had rights. However, in these LCSPs, the Navy did not address how this strategy met the Navy’s needs for competitive and affordable acquisition and sustainment over the life cycle of a ship class, such as to ensure maintenance could be carried out as planned by a ship’s crew. Without ship programs fully planning for acquiring needed intellectual property to maintain ship systems in the LCSP, we found that the fleet was often not aware that certain ship systems were considered proprietary and only discovered what intellectual property was unavailable after ship systems were broken and Navy maintainers could not repair them. At this point, fleet maintainers stated that it is often too late to implement proactive strategies, such as working on an agreement with the manufacturer. Instead, after ships are delivered, fleet maintainers told us that they have several options, all of which are expensive and time- consuming. Fleet maintainers can (1) purchase these data on an expensive sole-source basis from the original equipment manufacturer; (2) spend significant time and effort reverse-engineering the system to be able to repair it; or (3) pay the manufacturer to conduct maintenance. Performance-based logistics. For three shipbuilding programs that planned to use performance-based logistics, the shipbuilding programs assumed it would work as expected and did not identify the risks associated with this maintenance approach or develop any mitigation plans. For example, as stated earlier in figure 10, the DDG 1000 program adopted a performance-based logistics approach during the acquisition process in an attempt to reduce sustainment costs. As such, the program’s LCSP stated that a contractor would be responsible for maintaining the ships in the class, including a number of new and unique systems installed on the ships. However, the LCSP also noted that the DDG 1000 program had not been able to determine how much the performance-based logistics approach was likely to cost or what sustainment outcomes the Navy could expect from this approach, in large part due to the number of new systems installed on the ships. After the shipbuilding program delivered the first ship in the class from the shipyard, DDG 1000 program officials determined that the fleet and other Navy maintenance organizations would instead be responsible for the maintenance that the shipbuilding program previously planned to execute by hiring a contractor. According to fleet officials, since taking over maintenance responsibility, the Navy has also determined that these systems are difficult to sustain, citing lack of commonality, missing technical data, and other challenges. In some cases, the fleet is now replacing DDG 1000’s unique systems after delivery with systems common to other Navy ships in an effort to mitigate sustainment cost growth and readiness effects. Despite these critical changes in the sustainment approach, the DDG 1000 program has not updated its LCSP since 2009. ILAs Do Not Consistently Evaluate Shipbuilding Programs’ Sustainment Planning While the Navy has conducted an ILA on nearly every shipbuilding program we reviewed, we found that many of these assessments did not identify key sustainment issues or make recommendations to mitigate them. ILAs are conducted by assessment teams comprised of officials from across the Navy. The Navy ILA teams often validated program office sustainment assumptions contained in the LCSPs and other sustainment planning documents without evaluating those assumptions and identifying key areas of risk—even when programs introduced new sustainment concepts. DOD acquisition policy establishes that ILAs should provide an independent assessment of the shipbuilding program’s sustainment planning, including the identification and evaluation of issues that are likely to drive future O&S costs, design changes that could reduce O&S costs, and the adequacy of the product support strategy, among other things. ILAs are also supposed to make recommendations for mitigating the issues identified in the report, according to DOD and Navy guidance. Statutory requirements similarly emphasize the role of ILAs in identifying and mitigating sustainment risks that could increase O&S costs, and require DOD to establish guidance that requires the Navy to conduct ILAs prior to key acquisition decision points, including milestone decision events. ILAs for the shipbuilding programs included in our review did not sufficiently identify and evaluate the program offices’ sustainment assumptions and risks during the acquisition process. This was the case even when Navy testers had identified sustainment risks in early assessments conducted prior to the development of the LCSPs and ILAs. The following examples discuss instances in which Navy testers or maintainers identified sustainment risk before the ILA was conducted that have since caused sustainment challenges for the fleet, but the ILA team did not identify or make recommendations to address these problems. The SSN 774 shipbuilding program. As early as 2014, supply officials identified delays in over 1,000 supply orders for spare parts—many of these orders were in excess of 5 months old. However, in 2016, the ILA team rated this area as low risk and found that the supply support planning and execution was “outstanding.” Since supply support was rated as “low risk,” the ILA team did not make any recommendations to improve this planning. Subsequently, the SSN 774 class has experienced significant supply support issues. For example, the Navy’s maintainers routinely cannibalize hundreds of parts in 2017 and 2018 from SSN 774 class submarines to prepare other submarines for deployment, at an estimated rework cost of $2-3 million per year. The CVN 78 shipbuilding program. In 2013, testers stated that the number of berthing spaces on CVN 78 class carriers may not be sufficient to accommodate the planned crew size, particularly for the life of the carrier. When conducting its ILA in 2016, the ILA team rated crewing as low risk and the assessment noted extensive analysis had been conducted to validate the platform crewing profile. However, the ILA team did not document validation of the assumptions underpinning this analysis, such as whether or not automated systems needed to reduce crew levels would work as intended. The crewing concerns identified in 2013, but for which the ILA team did not make recommendations, are now a problem for the Navy’s fleet. For example, the Navy has already increased the size of the planned crew to the maximum allowed by the ship’s design. Nonetheless, additional crewing concerns persist for key systems—including the weapons elevators, advanced arresting gears, the machinery control system, among others—that are not yet well understood and may require additional sailor support to operate and maintain. The LCS shipbuilding program. In 2005 and 2006, Navy testers expressed significant concerns about the validity of the assumptions necessary to execute the program’s logistics support plan, specifically that the design of the new logistics system failed to include needed features to enable this logistics approach. In 2012, the Navy ILA team rated this area as low risk, specifically noting that the LCS program had developed a wide-range of well-written, informative, and comprehensive logistics planning documents. However, in part, since the ILA team did not recognize that the underlying issues previously identified by the testers had not been mitigated, the program provided ships to the fleet that had logistics issues. Specifically, the CNO conducted a study in 2016 that found the shipbuilding program’s logistics approach to be unstable and overly complex. As a result, the Navy is undertaking an overhaul of the LCS logistics approach, by taking actions such as creating Navy-led maintenance teams. The DDG 1000 shipbuilding program. The Navy requires significant volumes of technical data to manage the systems on the DDG 1000. In 2005, Navy testers noted that there were many details absent from the technical data management plan, including multiple sections that were left blank. In 2011, the Navy’s ILA team found that technical data management was low risk and stated that the requirements for technical data were well-written and clearly identified. According to fleet engineers and maintainers, as of September 2019—more than 3 years after lead ship delivery—all of the manuals remain in draft and are accurate enough for the sailors to acquaint themselves with systems, but not sufficient for supporting these systems. For example, fleet maintenance officials stated that several key documents for operating and maintaining critical ship systems, which were identified in the ILA as sufficiently complete, are not suitable for crew use. Several Navy officials across NAVSEA and shipbuilding program offices told us that ILAs are largely a document compliance check and vary significantly depending on the competency of the lead assessor. Therefore, in practice, according to Navy officials responsible for conducting these assessments, ILAs are not a thorough assessment of a ship classes’ logistics planning. This falls short of the purpose of ILAs, stated in Navy guidance, which is to provide acquisition programs with an effective measure of the program’s product support planning and execution. Officials from the NAVSEA organization responsible for ILA guidance also told us that they are in the process of improving how the Navy conducts ILAs for ships, such as by developing a new handbook and refocusing ILAs to better assess the quality of the sustainment planning. Specifically, these officials discussed the following five improvements: 1) starting ILAs as early as preliminary design; 2) tying the ILAs more closely to programs’ systems engineering efforts; 3) increasing focus on analytics, modeling, and simulation; 4) giving the Navy’s fleet and maintainers approval authority over the assessment; and 5) making investments to ensure that assessments are always led by officials with appropriate skills and expertise. If the Navy makes changes such as these or others, it would be a positive step toward making ILAs a more thorough and effective assessment of shipbuilding programs’ sustainment planning early in the acquisition process. However, these officials also stated that there is pushback from Navy program offices regarding these improvements because a more robust ILA requires more time and money from shipbuilding programs. Navy officials also noted that implementing the planned improvements is predicated on finding evaluators to conduct ILAs with appropriate skill sets, which has been a challenge. Until the Navy evaluates and implements proposed changes or other changes to improve the ILA process, the Navy will continue to be at risk of not identifying and resolving shipbuilding programs’ sustainment challenges during the acquisition process, before ships are provided to the fleet. Navy Leadership Does Not Ensure Shipbuilding Programs Effectively Consider Sustainment and Congress Does Not Have Full Insight into Sustainment Cost Growth We found that the senior leaders responsible for shipbuilding program oversight—the ASN (RD&A) and the CNO—have generally prioritized acquisition outcomes during Gate reviews, without considering how acquisition decisions affect sustainment outcomes. Navy acquisition policy states, however, that programs should be managed from a life- cycle perspective, with attention to both acquisition and sustainment outcomes. In an effort to increase senior leaders’ and shipbuilding programs’ attention on sustainment outcomes and to be responsive to Congressional efforts to improve weapon system sustainment, the Navy recently began pursuing two new initiatives—a Gate 7 for sustainment and the sustainment program baseline. These are promising steps that could help increase leadership insight into shipbuilding programs’ sustainment outcomes once ships are delivered to the fleet. However, we found that some of these efforts will likely not address the underlying need for Navy leadership to improve its consideration of shipbuilding programs’ sustainment goals early in the acquisition process as programs are making the decisions that have a long-term effect on ship sustainment. In addition, Congressional decision makers do not have full insight into sustainment cost growth. Navy Leadership Has Not Consistently Considered Sustainment in Gate Reviews, and Some Recent Changes Will Not Address Existing Shortfalls Navy leadership has not consistently reviewed shipbuilding programs’ sustainment planning at acquisition Gate reviews. According to senior Navy policy officials, in an effort to increase leadership attention on program sustainment, the Navy recently updated its acquisition policy to add a Gate for sustainment, called Gate 7. However, this recent change will not address the need for leadership to more consistently assess sustainment during earlier Gates. In addition, the Navy established a Deputy Assistant Secretary for Sustainment within the ASN (RD&A)’s office who will be responsible for managing the Navy’s sustainment funding and life-cycle management policies. However, it is too soon to assess the role that this official may have in the acquisition process. The Navy’s acquisition policy states that participants in Gate reviews should review program health and discuss and resolve areas of concern. Additionally, shipbuilding programs should be overseen and executed from a life-cycle perspective—in other words, with attention paid to balancing near-term acquisition outcomes and long-term sustainability. In support of this goal, the policy establishes required sustainment-related briefing content or actions for each Gate. While Gate 7 will function as the dedicated Gate for sustainment, all of the earlier Gates have sustainment- related requirements as well, as shown in Table 5 below. These Gate reviews offer Navy leadership opportunities to conduct oversight of shipbuilding programs’ sustainment planning during early phases of the acquisition process when key program decisions about requirements, design, and contracts are being made. Navy Leadership Has Not Consistently Used the Gate Process to Review Shipbuilding Programs’ Sustainment Planning and Outcomes Navy acquisition policy establishes that leadership should be briefed on a number of sustainment factors at Gate reviews, with a program’s life- cycle sustainment strategy/plan and O&S cost drivers being the minimum amount of sustainment information required for nearly all Gate reviews, as presented in table 5. We analyzed briefings and meeting minutes prepared for the 22 Gate reviews held for the shipbuilding program in our review between fiscal year 2014 and 2018. We found that Navy leadership had not assessed shipbuilding programs’ life-cycle sustainment strategies/plans in approximately 86 percent of Gate reviews and had not assessed O&S cost drivers in approximately 64 percent of Gate reviews, as shown in figure 12. According to Navy acquisition policy, this sustainment information should have been evaluated during all 22 of the Gate reviews held between fiscal year 2014 and 2018 for the shipbuilding programs included in our review. Instead, we found that the Gate reviews most often discussed acquisition updates. While a focus on acquisition updates during Gate reviews is appropriate, by infrequently devoting attention to how acquisition decisions affect sustainment, Navy leadership is missing an opportunity to assess the comprehensiveness and validity of shipbuilding programs’ sustainment plans and cost estimates, among other sustainment factors. As we previously discussed, shipbuilding programs’ LCSPs and O&S cost estimates were incomplete or insufficient, and, therefore, did not provide a thorough assessment of the programs’ sustainment risks. Additionally, Navy leadership is not consistently using Gate reviews to communicate to shipbuilding programs that achieving sustainment goals is a high priority. For pre-construction Gate reviews (Gates 1-5), Navy leadership evaluated three of the programs included in our report—SSBN 826, FFG(X), and DDG 51—in the 5-year period between fiscal year 2014 and 2018. These Gate review briefings included some discussion of program sustainment but did not meet all of the objectives and goals described by Navy acquisition policy for sustainment briefing content, as presented in table 5. As such, the Gate reviews did not provide a complete assessment of whether the programs’ acquisition decisions about sustainment would support the delivery of ships that could meet sustainment requirements at an affordable cost. Officials from the majority of programs included in our review told us that these early phases of the program are critical because it is at this point in the program where decisions are made that can have a long-term effect on ship sustainment, and it is difficult to make significant changes to sustainment outcomes after these key decisions are made. For example, when Navy leadership reviewed the SSBN 826 program at a Gate 4 review in November 2015 and a Gate 5 review in September 2016, the briefing discussed the SSBN 826 program’s sustainment costs in detail, including O&S cost goals, cost drivers, and contract incentives for O&S affordability. However, among other things, the Gate 4 briefing did not include a review of the program’s life-cycle sustainment strategy, and the Gate 5 briefing did not verify that all critical technical data and intellectual property issues had been addressed, which fleet and engineering officials stated are known sustainment issues for the Virginia class of submarines. Officials from the SSBN 826 program stated that some sustainment information that was not discussed in the Gate reviews was addressed in other forums. For example, leadership approved the program’s LCSP in August 2016, between the Gate 4 and Gate 5 reviews. In another example, when Navy leadership reviewed Flight III of the DDG 51 program at a combined Gates 4 and 5 review in March 2014, none of the required sustainment topics were included in the briefing. By not thoroughly assessing and resolving the sustainment effect of early acquisition decisions during its Gate reviews, Navy leadership is missing opportunities to ensure that shipbuilding programs are adequately considering sustainment goals and is at risk of allowing programs to proceed through the acquisition process without verifying that there is adequate planning for sustainment. For Gate 6 reviews held between fiscal year 2014 and 2018, we similarly found that Navy leadership did not consistently discuss sustainment, even as programs began ship construction and delivering ships to the fleet. Our analysis of Gate 6 documentation showed that the primary focus of most Gate 6 briefings and meeting minutes was acquisition outcomes, such as construction progress or follow-on ship contract awards. In particular, we found that 16 of the 18 Gate 6 reviews we assessed for eight shipbuilding programs did not include information about both the program’s life-cycle sustainment plan and O&S cost drivers, which are part of the required briefing content for every Gate 6 review. Officials from most of the programs in our review confirmed that leadership placed greater emphasis on acquisition updates than sustainment during Gate 6 reviews. For example, the SSN 774 program is pursuing a reduction in total ownership costs initiative for its Block IV submarines, but the program’s recent Gate 6 briefings included only limited details on design changes that the program was pursuing to improve sustainment and no information on the anticipated O&S cost savings from the effort. Officials from this program confirmed that leadership has historically focused only on acquisition issues during the Gate 6 reviews. Additionally, we found that Navy leadership issued sustainment-related action items to only three of the eight programs in the Gate 6 reviews we assessed, even though all of these programs had ongoing sustainment challenges, as discussed earlier in this report. Although nearly 90 percent of the Gate 6 reviews we assessed did not include briefing content on the program’s life-cycle sustainment plan and O&S costs, as required, nearly all of the Gate 6 reviews included a discussion of at least one ongoing sustainment challenge affecting the ship class. In these cases, the discussion centered on mitigating realized sustainment issues already being experienced by the fleet after ship delivery. For example, all of the LPD 17 Gate 6 reviews over the past 5 years included updates on the activities of the LPD 17 Strike Team and its progress in resolving class-wide design and construction issues that negatively affected the ships’ operational availability and reliability after they began fleet operations. While Gate 6 can be used as a venue to discuss sustainment issues that are already being experienced by the fleet, until Navy leadership more consistently reviews programs’ sustainment planning and expected outcomes during earlier Gates, programs will continue to be at risk of delivering ships to the fleet that have unmitigated sustainment risks or are unaffordable. Recent Gate Process Changes Enhance Sustainment Focus, but Do Not Address the Need to Consider Sustainment Issues Earlier in the Acquisition Process The Navy recently updated its acquisition policy to expand the scope of its Gate process and add a new Gate 7 for sustainment. Effective March 2019, the Gate 7 reviews will begin 5 years after shipbuilding programs achieve initial operational capability and recur every 5 years thereafter. As such, Navy officials told us that the scope of the Gate 7 review will be oversight of programs that are well into production and delivering ships to the fleet. According to the Navy’s acquisition policy, Gate 7 will evaluate the effectiveness of a program’s product support strategy, compare actual sustainment costs to estimates, discuss fleet-identified sustainment issues, and assess sustainment risks and mitigation measures, among other things. Senior officials told us that the Navy developed a Gate 7 for sustainment for two reasons. First, similar to our findings, officials stated that the Navy recognized sustainment was generally not being discussed during existing Gate reviews, particularly during Gate 6 reviews as ships were starting to be delivered to the fleet, even though this was required briefing content for Gate 6 in the Navy’s acquisition policy. Second, in the National Defense Authorization Act for Fiscal Year 2017, Congress directed the military services to conduct sustainment reviews on major weapons systems—such as the shipbuilding programs included in our review—within 5 years of the weapon system achieving initial operational capability and then periodically throughout their life cycles. Such sustainment reviews are to assess the weapon system’s product support strategy, performance, and O&S costs. Based on our analysis of the Navy’s revised acquisition policy, the new Gate 7 appears responsive to the Congressional requirement for sustainment reviews and, if implemented as planned, will provide an oversight forum for addressing realized sustainment challenges. However, we found that adding a new Gate to the end of the acquisition process is too late to drive meaningful improvements to sustainment outcomes and is not sufficient to address current shortfalls in how the Navy’s acquisition process addresses sustainment concerns. Senior Navy officials we spoke to who had knowledge of this change expressed doubt that a Gate 7 for sustainment would be an effective means of holding programs accountable for addressing acquisition-related sustainment issues, since it occurs late in the acquisition process. Whereas the Gate 7 for sustainment will occur at the end of the acquisition process, the decisions that influence sustainment outcomes, such as decisions about ship design and the planned sustainment strategy, are made much earlier in the process, normally between Gates 1 and 5. Thus, while Gate 7 will provide leadership with insight into the execution of ship sustainment and any challenges being experienced by the fleet, it does not address the need for Navy leadership to evaluate shipbuilding programs’ efforts to design and plan for sustainable ships during earlier Gates, when key long-term decisions are being made. According to a senior Naval Sea Systems Command official, Gate 7 is timed well for being able to “sit back and admire the problem” as opposed to preventing the issue. Until Navy leadership brings attention to sustainment during earlier Gate reviews, it will continue to miss opportunities to proactively ensure shipbuilding programs are acquiring sustainable ships before they are provided to the fleet. Acquisition Program Baselines Currently Include Few Sustainment Goals and Ongoing Improvements Lack an Accountability Mechanism We found that acquisition program baselines (APB)—which are intended to be binding agreements between leadership and the program manager and document the program’s goals—currently include limited information about sustainment. While the Navy is developing a new initiative to create a dedicated baseline for sustainment, it does not have a mechanism for holding shipbuilding programs accountable for sustainment goals during the acquisition process. Like all major weapon systems, shipbuilding programs have APBs that summarize the programs’ cost, schedule, and performance goals and set the baseline from which programs must, as appropriate, obtain approval from agency leadership to deviate and must report certain changes to Congressional defense committees. Statute requires that baselines will contain information on the program’s cost estimate, schedule, performance, and supportability, among other factors. In practice, for shipbuilding programs in our review, we found that the program goals established in the APB are largely focused on acquisition cost, acquisition schedule, and performance requirements, with limited information provided on sustainment. In particular, the sustainment information provided is generally limited to a high-level O&S cost estimate and the sustainment key performance parameters, if the program has them. A Congressionally established panel, called the Section 809 panel, charged with making recommendations to improve the efficiency and effectiveness of DOD’s acquisition process, among other things, recently studied challenges with the sustainment of major weapon systems. It similarly found that the APB does not provide sufficient governance of the sustainment phase of an acquisition program since it is focused on acquisition cost, schedule, and performance goals. The panel further noted that program success has been measured against the achievement of the APB’s acquisition goals, so program managers have generally prioritized the achievement of acquisition outcomes and deemphasized sustainment. As a result, the panel recommended the creation of an additional program baseline, called the sustainment program baseline (SPB), to help ensure programs are held accountable for sustainment- related outcomes and establish balance between acquisition and sustainment priorities. In March 2019, the Navy initiated an effort to begin developing an SPB framework. Senior officials stated that the Navy intends to pilot the SPB with a few aviation programs in fiscal year 2020 before expanding the initiative to ship classes that are already in sustainment, and then finally to programs that are still in the acquisition process. According to Navy officials involved with this initiative, the SPB is intended to complement the APB, and Navy leadership will use the two program baselines to review and approve the acquisition and sustainment aspects of a program throughout the acquisition process. The shipbuilding program should draft the initial SPB early in the acquisition process to support Milestone A and update it as the program matures. Officials in the office of the ASN (RD&A) told us that the Navy plans for the SPB to be grounded in a program’s sustainment key performance parameters for operational and materiel availability and include targets for various other sustainment metrics, such as sparing, equipment failure rates, mission capable time, and logistics time, among others. The SPB should also provide detailed information about all of the costs and funding sources that will support sustainment. Navy officials identified a number of potential improvements the SPB could offer for how shipbuilding programs consider sustainment, such as devoting additional time and resources to the development of sustainment metrics early in a shipbuilding program, assessing the sustainment effect of acquisition decisions, creating a common understanding of a program’s sustainment goals across disparate stakeholders, and providing a more accurate accounting of sustainment funding. If the Navy implements the SPB as described, it will likely be a positive step toward ensuring shipbuilding programs are increasing their focus on sustainment planning during the acquisition process. While the SPB could potentially provide increased attention on program sustainment, we found that developing this new baseline may not fully address the underlying challenge of shipbuilding programs managing to the APB’s acquisition goals and the lack of consideration of sustainment in acquisition decision-making. This is because, according to current proposals, programs will continue to be measured against the APB during the acquisition process, with the SPB not serving as the governing baseline until later in the program life cycle during the sustainment phase. Instead, during the acquisition process, the Navy’s efforts related to the SPB will be limited to initially developing the SPB and updating it as the program matures. While updates to the SPB during the acquisition process could provide more transparency into the sustainment effect of various acquisition decisions within the program and to leadership, this approach primarily documents the sustainment effect of a decision. Because the APB will remain the governing baseline during the acquisition process and the program will not be measured against the SPB until the sustainment phase, shipbuilding programs will continue to have an incentive to prioritize acquisition outcomes over sustainment when making acquisition decisions. Congress Does Not Have Insight into Shipbuilding Programs’ O&S Cost Growth during the Acquisition Process DOD does not provide Congress with detailed information on the extent and causes of shipbuilding programs’ O&S cost growth during the acquisition process. For example, a mechanism for Congressional oversight of major defense acquisition programs’ unit cost growth, called the Nunn-McCurdy statute, is focused on acquisition costs and not sustainment cost estimates. A Nunn-McCurdy breach is triggered by increases in a program’s unit cost estimates against the acquisition unit cost goals established in the program’s APB. The Nunn-McCurdy statute provides Congress greater visibility into major defense acquisition programs’ estimated acquisition cost growth and encourages DOD to manage costs by requiring programs in a breach to include acquisition cost estimates in Selected Acquisition Reports and notify Congress of a breach. While the APB also includes O&S cost estimates, the Nunn- McCurdy statute does not require reporting of O&S cost growth to Congress. The Nunn-McCurdy statute also requires DOD to take a series of actions whenever a program experiences critical acquisition cost growth, which is growth in the program acquisition unit cost estimate of at least 25 percent over the current baseline estimate documented in the APB or of at least 50 percent over the initial baseline estimate. Among other things, these actions include (1) conducting a root cause analysis of the cost growth, (2) reassessing program costs, and (3) terminating the program or taking other steps that include restructuring the program. If DOD decides not to terminate a program that has critical cost growth, the Secretary of Defense must restructure the program in a manner that addresses the root cause of the cost growth, rescind the program’s most recent Milestone decision, and review the program regularly, among other tasks. As stated earlier, we found that leadership oversight during Gate reviews and program execution is primarily focused on acquisition outcomes. Additionally, as the Section 809 panel noted, the Nunn-McCurdy breach provided a strong incentive for major defense acquisition programs to control acquisition cost, but that there was not an equivalent incentive for controlling sustainment costs. As such, the shipbuilding programs’ acquisition decisions and Congress’ oversight mechanisms have focused on acquisition cost outcomes, without a comparable focus on sustainment cost outcomes during the acquisition process. For example, when the DDG 1000 program experienced a critical acquisition cost growth breach, the Nunn-McCurdy statute required DOD to reassess and certify to Congress the need for the program at the increased cost levels. DOD was also required to identify and address the cause of the acquisition cost growth when reassessing the program and conduct additional program oversight, among other things. According to DDG 1000 program officials, DOD and the Navy recognized that the acquisition decisions leading up to and following the breach would have a sustainment effect. For example, the decision to reduce the number of ships in the class to manage acquisition cost growth has contributed to higher per ship O&S costs, as the investment needed to sustain this new class is now spread across fewer ships than initially planned. However, the focus of their restructuring efforts was on addressing the acquisition cost growth. By contrast, there was not a similar effort to manage growth in the program’s O&S cost estimates, which have increased by more than 50 percent on a per ship per year basis. For the six shipbuilding programs with O&S cost estimates we were able to assess, we found that four experienced cost growth greater than 50 percent for their average annual O&S per hull cost, as compared to the programs’ original estimates. Table 6 shows the extent of these shipbuilding programs’ O&S cost estimate growth over time. This level of cost growth for acquisition costs would have constituted a Nunn-McCurdy breach. While the Selected Acquisition Reports for these programs include some information on shipbuilding programs’ O&S costs, this reporting does not provide Congress with detailed information about the causes of the cost growth and potential program changes to address it and, therefore, does not facilitate the same level of oversight as is given to acquisition unit cost growth. In particular, DOD was not required to notify Congress that the programs had experienced high levels of O&S cost growth above a certain threshold, and DOD was not required to identify the root cause of the O&S cost growth and restructure the programs to address the cost growth. As a case in point, for the programs we reviewed, Navy leadership only directed one of the programs—LPD 17—to identify opportunities to reduce O&S costs following a Gate 6 review. For other programs that had extensive O&S cost growth, the programs were not required to take additional steps during the shipbuilding process to manage these costs and mitigate the long-term sustainment effect of their acquisition decisions. The LCS program, for example, has seen the highest rate of per ship O&S cost growth among the shipbuilding programs included in our O&S cost analysis, but Congress and agency leadership have not required the shipbuilding program to take action to address these issues. Instead, the shipbuilding program continues to deliver ships to the fleet that are significantly more expensive to maintain than initially planned and which have significant maintenance and logistics challenges, according to sustainment officials. The fleet is now undertaking its own efforts to improve sustainment outcomes for LCS, such as changing its crewing and maintenance approaches, which are further adding to the O&S cost growth for the program. According to DOD and Navy acquisition policy, program managers should be the single point of accountability for the full life cycle of ship programs. However, without a mechanism to provide Congress with more detailed information about shipbuilding programs’ O&S cost growth and the drivers of such cost growth, Congress cannot know if shipbuilding programs are accounting for the full life-cycle implications of their acquisition decisions. In particular, without such a mechanism, Congress will continue to lack full insight into the extent to which shipbuilding programs’ O&S cost estimates have grown over time and what steps DOD and the Navy could take to better control O&S cost growth during the acquisition process. Product Support Managers Have Limited Influence in the Acquisition Process Congress directed DOD to establish PSMs as key sustainment managers for weapons systems, such as shipbuilding programs. However, we found that PSMs in the shipbuilding program offices have limited influence on decisions made during the acquisition process that affect ship sustainment. In 2009, Congress passed legislation that required DOD to appoint PSMs to support each major weapon system. According to DOD guidance, PSMs are senior sustainment officials in program offices who are tasked with ensuring that DOD weapon systems, including Navy ships, are reliable and can be maintained effectively at an affordable cost. The guidance states that PSMs should be involved in the acquisition decision-making process to ensure the weapon system—in this case a ship—can be supported throughout its life cycle. All but one of the shipbuilding programs included in our review have a dedicated PSM. However, we found that these sustainment experts have generally had limited involvement in key acquisition decisions, such as developing sustainment requirements and estimating O&S costs, because: (1) Navy acquisition policy does not ensure that PSMs are involved early in the acquisition process when key decisions that affect sustainment are made, and (2) their responsibilities to support sustainment outcomes during the acquisition process are often at odds with the program office’s overarching focus on acquisition cost and schedule outcomes. Navy Policy Does Not Require PSMs to Be Involved Early in the Acquisition Process Navy acquisition policy does not ensure that PSMs are appointed early enough to inform key acquisition documentation and initiate sustainment planning early in the acquisition process. Until recently, Navy acquisition policy did not specify when PSMs should be involved in the acquisition process. However, a March 2019 update to the Navy’s acquisition policy established that Navy leadership should assign PSMs by initiation (normally Milestone B). We found that this timing is too late in the acquisition process, as critical acquisition decisions that have significant repercussions for sustainment are made before Milestone B, such as developing the program’s requirements and initial sustainment strategy. For example, according to DOD’s PSM guidance, PSMs need to be involved prior to initiation of the program. Among other things, the PSM should provide a sustainment perspective into key decisions such as developing the acquisition strategy and setting requirements. This guidance also states that the PSM is responsible for authoring or providing input on key program documents, such as the LCSP, which are required by Milestone A. The Navy policy, therefore, does not facilitate the early contributions of PSMs to key documents as described by DOD guidance, and it does not help ensure PSMs are appointed to shipbuilding program offices early enough to influence key decisions about the program’s sustainment. For its two most recent shipbuilding programs, which began after the enactment of the PSM legislation in 2009, Navy acquisition policy has not ensured that PSMs are involved early enough in the acquisition process to influence decisions that affect sustainment. As a result, the programs have appointed PSMs at different points in the acquisition process and their ability to influence key decisions has varied, with the PSM appointed earlier able to affect more decisions related to sustainment. For example, the SSBN 826 program’s PSM was appointed before the program reached Milestone A. This is in line with DOD guidance but before Navy acquisition policy requires the PSM to be appointed. As a result, the SSBN 826 PSM stated that he was involved in the setting of the program’s sustainment requirements and has subsequently used those requirements to ensure sustainment is being considered in the acquisition process, including during the development of the submarine’s design. By contrast, the FFG(X) program, which began in 2017, does not yet have a dedicated PSM as the program approaches the Milestone B review. While this is permitted by the Navy’s acquisition policy, the program has now made critical sustainment decisions, such as developing the sustainment strategy, the maintenance and training schedule, and the sustainment key performance parameters, without a PSM. For the nine shipbuilding programs in our review that started prior to 2009, key acquisition decisions were made without the input of a senior sustainment official who has the responsibility and authority of a PSM. Nearly all of the PSMs for these nine programs stated they that they were not involved in or did not have insight into key acquisition decisions that took place early in the acquisition process, such as ship design. Instead, PSMs told us that their job has been to implement decisions that were already made. For example, one PSM said that “the die has been cast” once major decisions about automation, crew size, and service life are made, and after that all the PSM can do is “try to undo the sustainment harm that has been caused.” Given these results, officials from nearly all of the shipbuilding programs we spoke with stated that shipbuilding programs should assign PSMs at the very beginning of the program when key decisions are being made about how and what to acquire. In particular, program officials stated that the PSM should be appointed at the start of the program to ensure early decisions consider sustainment. Such decisions include establishing the sustainment requirements, developing the acquisition strategy, and designing the ship. We previously found that Navy PSMs considered early appointment of the PSM critical to ensuring they can influence their programs’ sustainment considerations. If shipbuilding programs do not appoint PSMs early in the acquisition process, the programs will continue to make critical decisions that affect sustainment without the input of the programs’ senior sustainment official. Without revising its acquisition policy to establish that PSMs should be appointed to shipbuilding programs at the beginning of the acquisition process, the Navy cannot ensure PSMs are involved early enough to influence key decisions that affect sustainment, such as requirements setting and the drafting of the LCSP. PSM Responsibilities Can Be at Odds with Shipbuilding Program Cost and Schedule Objectives Since PSMs focus on sustainment and the shipbuilding programs focus on managing acquisition outcomes, the PSMs’ roles and responsibilities are at times at odds with the goals and priorities of the program office in which they work. A Navy working group recently found that the effectiveness of PSMs is limited because the PSM’s goals do not always align with the shipbuilding program’s acquisition cost and schedule goals. The Navy issued a strategic plan for fiscal years 2018 to 2023 that was focused on strengthening the life-cycle logistics workforce that supports acquisition programs, including PSMs. The strategic plan established a working group on product support authority, which found that program manager and PSM roles and responsibilities are often in conflict and misaligned, reducing the authority and effectiveness of PSMs. As a result, the working group is assessing possible changes to improve the effectiveness of PSMs, such as revising Navy policy to better reflect the PSMs’ statutory authority or increasing PSMs’ independence by creating an additional reporting chain of command outside of their acquisition program. We similarly found that the ability of PSMs to influence key acquisition decisions may be limited because their focus on improving sustainment outcomes can be at odds with the shipbuilding programs’ emphasis on achieving acquisition goals, such as acquisition cost and schedule. As discussed above, Navy leadership has generally only focused on shipbuilding programs’ acquisition outcomes during the Gate process, without considering how acquisition decisions affect sustainment. In turn, program officials from all of the shipbuilding programs we reviewed reported that Navy leadership had directed them to prioritize the achievement of acquisition outcomes, such as acquisition cost goals, during the execution of their programs, and none had been directed to devote additional attention to sustainment. Additionally, officials in many of the shipbuilding programs we reviewed told us that a key ASN (RD&A) memorandum on managing acquisition costs framed their decision- making, including decisions about program changes to improve sustainment. This focus on managing acquisition costs can run counter to PSMs’ efforts to improve sustainment outcomes, such as increasing system reliability or providing adequate technical documentation, as these efforts frequently require investment of additional shipbuilding funds. Rather than investing acquisition funds to improve sustainment outcomes, we found that shipbuilding programs instead have an incentive to delay sustainment improvements until after ships are delivered to the fleet and funding sources other than those managed by the shipbuilding program can be used for these purposes. According to officials from 16 different acquisition, engineering, and sustainment offices, because shipbuilding programs are only responsible for ships until they are provided to the fleet, the Navy’s shipbuilding programs have an incentive to delay sustainment improvements until after ships are delivered to the fleet, when other parts of the Navy take over responsibility for funding them. As one fleet official explained, shipbuilding programs are not incentivized to address sustainment issues because the shipbuilding programs are held responsible only for the achievement of acquisition cost goals and not for sustainment cost goals. Some Navy officials characterized this dynamic as throwing sustainment concerns “over the fence” once ships are provided to the fleet. Further, we found that Navy leadership made decisions, in some cases, even though PSMs expressed concerns about the feasibility of implementing the decision from a sustainment perspective. Figure 13 provides an example of when LCS sustainment officials in the shipbuilding program expressed concern about the feasibility of the LCS crew size. While it is important for shipbuilding programs to manage acquisition cost and schedule, focusing only on these acquisition outcomes reduces the effectiveness of the PSM and increases the risk that ships will have long- term sustainment challenges. Conclusions The quantity and breadth of issues identified in this report—resulting in billions of dollars in unexpected costs, maintenance delays, and unreliable ships—suggest that existing policies and guidance have not ensured that new ships are reliable and can be sustained as planned. Recently, due to some of these problems, DOD and the Navy have recognized the importance of considering the requirements and costs of sustainment during the acquisition process, and Congress has passed legislation related to sustainment planning. This report, along with other DOD initiatives discussed in this review, demonstrate that the Navy needs to take many steps to infuse its acquisition decision-making with a greater focus on sustainment outcomes. Systemic changes are needed to improve shipbuilding programs’ sustainment outcomes, including: setting clear sustainment requirements that are useful for acquisition decision-making and reporting the results to Congress, improving O&S cost estimates, sustainment planning, and logistics assessments, and involving the PSM early in the acquisition process. However, these changes will only be successful if Navy leadership commits more time, attention, and resources to ensuring that sustainment is thoroughly considered throughout the acquisition process. Until the Navy resolves these issues, its shipbuilding programs will continue to pass costly sustainment risk to the fleet that results in ships and submarines that experience major sustainment problems. Matter for Congressional Consideration Congress should consider developing an oversight mechanism for evaluating shipbuilding programs’ sustainment cost estimate growth during the acquisition process, with requirements for the Navy to: (1) report sustainment cost estimate growth information to Congress and (2) reassess shipbuilding programs that are experiencing a high level of sustainment cost estimate growth. Recommendations for Executive Action We are making the following 11 recommendations to DOD: The Secretary of Defense should change its definition for setting operational availability for ships in its Joint Capabilities Integration and Development System policy by adding information that defines the operational availability requirement by mission area in addition to the ship level and includes all equipment failures that affect the ability of a ship to perform primary missions. (Recommendation 1) The Secretary of Defense should change its definition for setting materiel availability for ships in its Joint Capabilities Integration and Development System requirements policy to include all factors that could result in a ship being unavailable for operations, such as unplanned maintenance, unplanned losses, and training. (Recommendation 2) The Secretary of the Navy should direct the ASN (RD&A) and the CNO, once DOD requirements setting policy is revised, to update existing operational availability requirements for ongoing shipbuilding programs. When revising these requirements, the Navy should set operational availability requirements that: (1) are based on failures that affect the ability of a ship to perform primary missions and (2) are set at the mission level instead of ship level. (Recommendation 3) The Secretary of the Navy should direct the ASN (RD&A) and the CNO, once DOD requirements setting policy is revised, to update the materiel availability requirements for ongoing shipbuilding programs. When developing or revising these requirements, the Navy should set materiel availability requirements that fully capture all factors that could preclude a ship from being ready when needed. (Recommendation 4) The Secretary of the Navy should direct the ASN (RD&A) and the CNO, once the Navy revises its sustainment requirements, to ensure that shipbuilding programs report operational availability and materiel availability requirements in Selected Acquisition Reports, and alternatives to the Selected Acquisition Reports, for Congress. (Recommendation 5) The Secretary of the Navy should direct the Commander of Naval Sea Systems Command to ensure that cost estimators follow current guidance and GAO-identified best practices and conduct sensitivity analyses and other analyses to improve their assessment of cost risk in the O&S costs in shipbuilding programs’ life-cycle cost estimates. (Recommendation 6) The Secretary of the Navy should direct the ASN (RD&A) to ensure all shipbuilding programs develop and update LCSPs, in accordance with DOD policy, that demonstrate how a ship class can be affordably operated and maintained while meeting sustainment requirements, including associated business case analyses and identifying sustainment risk. (Recommendation 7) The Secretary of the Navy should direct the Commander of Naval Sea Systems Command to evaluate and implement changes to the ILA in order to position the ILA to effectively identify key sustainment risks and make recommendations for risk mitigation, which may include existing Navy proposals to change the ILA process. (Recommendation 8) The Secretary of the Navy should direct the ASN (RD&A) and the CNO to ensure sustainment-related briefing topics prescribed by the Navy’s acquisition policy are consistently discussed at Gate reviews. (Recommendation 9) The Secretary of the Navy should direct the ASN (RD&A) and the CNO to implement the sustainment program baseline initiative for shipbuilding programs and, in so doing, develop a mechanism that ensures that sustainment outcomes are a factor in shipbuilding programs’ decision- making during the acquisition process. (Recommendation 10) The Secretary of the Navy should revise SECNAVINST 5000.2 and other associated guidance to ensure PSMs are assigned to shipbuilding program offices in time to inform early acquisition decisions, including development of the program’s sustainment requirements and LCSPs. (Recommendation 11) Agency Comments and Our Evaluation We provided a draft of our report to DOD for comment. DOD’s written comments are reprinted in appendix III of this report. DOD concurred with 8 recommendations and partially concurred with 3 recommendations. However, for at least 5 of the recommendations in which DOD partially concurred and concurred, DOD did not describe the specific actions it is planning to take to address our recommendations. These are discussed below. In response to our first and second recommendations on operational and materiel availability requirements, DOD stated that the Navy and Joint Staff would revisit requirements definitions for shipbuilding programs to better ensure that they are traceable to a ship’s mission and can be used across ship development and fielding. DOD also agreed that it will align the sustainment definitions with how the Navy defines critical failures for ship programs. While these are important steps, they do not fully address our recommendations. Specifically, DOD officials told us that the department plans to continue defining operational availability with a single metric for an entire ship or ship class. While this approach is appropriate for materiel availability, as we state in the report, it is misaligned with Navy guidance for operational availability, which states that such an approach is not mathematically feasible for ships. Until DOD ensures that its sustainment requirements for ships are well-defined and usable during acquisition and sustainment, shipbuilding programs will continue to implement requirements that do not result in reliable and available ships. In response to our third and fourth recommendations, DOD agreed to incorporate changes to its requirements-setting policy into new shipbuilding programs. However, DOD and the Navy may miss key opportunities to improve the Navy’s sustainment requirements for existing programs, including at least four ship classes that have plans for a new flight, block, and/or major modification. This approach also excludes existing programs that have established requirements but have yet to start design or construction. Changing these requirements, in line with our recommendation, would help ensure that more rigorous sustainment requirements inform Navy ship designs. For example, as we discuss in the report, the current FFG(X) operational availability requirement would allow the ships to be out of service for extraordinary lengths of time. Until the FFG(X) requirement and those for other existing ships (such as DDG 51 Flight III) are remedied, the sustainment requirements will continue to be poorly defined and unable to influence design decisions in a manner that results in more reliable ships. In response to our fifth recommendation, DOD concurred with the recommendation because it stated that it already reports the status of both sustainment requirements in its Selected Acquisition Reports. However, as we state in our report, implementing this recommendation is dependent on the Navy changing the definition of its sustainment requirements to improve the accuracy of its reporting to Congress. Since DOD only agreed to modify material availability requirements for existing ship programs as it deemed appropriate, its Selected Acquisition Reports could continue to be misleading for many of its ship programs because they may not reflect all of the failures and factors that reduce operational and materiel availability once ships are in the fleet. In addition to DOD’s response, the Navy’s ASN (RD&A) also submitted a letter stating that he generally agreed with the recommendations and indicated that his office has already started making some changes over the last 10 years to improve consideration of sustainment while acquiring ships. The Navy also sought to add context to some of our report findings. We respond to the ASN RD&A’s letter in appendix III. DOD and the Navy also provided technical comments that we incorporated as appropriate. We are sending copies of this report to the Secretary of Defense, Secretary of the Navy, interested congressional committees, and other interested parties. This report will also be available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions concerning this report, please contact me at (202) 512-4841 or by e-mail at oakleys@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV. Appendix I: Objectives, Scope, and Methodology This review assesses the extent to which: (1) the Navy’s shipbuilding programs deliver ships to the fleet that can be sustained as planned; (2) the Navy develops and uses effective key sustainment requirements during the acquisition process; (3) shipbuilding programs effectively identify and evaluate sustainment costs and risks in key acquisition planning documents; (4) Navy and Congressional leadership have insight into and effectively consider programs’ sustainment planning and outcomes; and (5) the shipbuilding programs leverage Product Support Managers (PSMs) during the acquisition process. The scope of our review included all shipbuilding programs for warships that had ships under construction or in development in the last 10 years, from fiscal years 2009 through 2019. We defined a shipbuilding program as under construction if any ship in the class was under construction in the last 10 years. We defined a shipbuilding program as in development if the Navy had awarded a development or design contract for the class in the last 10 years. We excluded military sealift command vessels and other Navy vessels with logistics missions from this review to help ensure that our resources matched the scope of our review. We assessed LHD 8 and CVN 77 as their own classes for the purposes of our review because the Navy considers them to be transitional designs between antecedent classes. These parameters resulted in 11 ship classes for inclusion in our review. We also selected several ships within these classes to serve as case studies for additional analysis. To select these ship-specific case studies, we reviewed all warships delivered from fiscal years 2007-2018 and selected up to four of the most recent hulls within this time frame from each class as case studies. We selected these ships for additional analysis because they are still relatively new, but the fleet has had experience maintaining them and could discuss sustainment challenges for those ships, if any. All ship classes and case study hulls in scope are listed in table 7. Over the course of this audit, we interviewed officials from over 100 Navy organizations involved in designing, building, inspecting, testing, sustaining, and operating Navy ships to gain an understanding of the extent to which they are involved in the acquisition process and how they consider and manage sustainment risk during the acquisition process. These interviews also provided information on the nature and magnitude of sustainment issues being experienced by the fleet on recently delivered ships. These included approximately 30 interviews with organizations reporting to the ASN (RD&A), 31 interviews with organizations reporting to the Chief of Naval Operations, 29 interviews with organizations within Naval Sea Systems Command (NAVSEA), interviews with shipbuilders that have been awarded multiple Navy shipbuilding contracts, and three interviews with other Department of Defense (DOD) entities. We conducted these interviews in several locations throughout the United States, including Washington, D.C.; San Diego, CA; Norfolk, VA; Philadelphia, PA; and Mechanicsburg, PA. During visits to naval bases, we toured DDG 111, DDG 1000, LHD 8, LPD 22, LCS 3, LCS 4, CVN 77, and CVN 78. To identify the extent to which ships can be sustained as planned, we interviewed shipbuilding program officials, in-service program officials, engineers, and fleet organizations, as well as analyzed ship and system performance data from many Navy organizations. Through this assessment, we identified and analyzed 150 significant class-wide issues across the shipbuilding programs in our scope that required more sustainment resources than planned. Such issues include systems or parts that exhibited poor design, construction, reliability, or planning; systems that were obsolete before or soon after ship delivery; and systems that could not be maintained by the fleet due to vendor or manufacturer proprietary information. We counted only issues that were class-wide, meaning they were related to ship design, equipment used across the class, or construction procedures, rather than hull-specific issues that could be caused by a unique accident or sailor error. We also did not assess issues related to fleet preference. For example, one ship’s crew told us they did not prefer the location in which consoles for operating a certain system were installed, as they are typically installed in a different location on other ship classes. However, because the consoles were installed in the location specified in the design, we eliminated this issue from our analysis. We also eliminated issues if maintenance and other work on the affected system were accounted for during the acquisition process in the program’s initial Operating and Support (O&S) cost estimate, rather than being an unexpected expense. For example, program offices can address expected obsolescence by budgeting for future system modernizations or purchasing quantities of spare parts that will last for the ship’s entire life cycle. To identify the costs associated with fixing problems that are the result of not being able to sustain ships as planned, we reviewed documentation from Navy organizations, budget justifications, and estimates provided by Navy officials. We were able to collect cost information for 30 percent of the problems reported to us by the fleet. To assess the extent to which maintenance schedules are executed as planned, we analyzed Navy data on regularly scheduled, depot-level maintenance periods for surface ships—including those maintained at overseas homeports and in the United States. NAVSEA collects and manages data on these maintenance periods—known as Chief of Naval Operations maintenance availabilities—for surface ships, submarines, and aircraft carriers. We obtained the data on surface ship depot-level maintenance periods used by NAVSEA’s Surface Maintenance Engineering Planning Program and the Commander, Navy Regional Maintenance Center. We reviewed the data we obtained for inconsistency errors and, when possible, obtained multiple documents that discussed the same problem for validation. We then discussed these problems with multiple officials across the Navy, including officials involved in ship maintenance and operation. From these efforts, we determined that these data are sufficiently reliable for the purposes of this report. To assess the extent to which shipbuilding programs develop and use effective sustainment requirements during the acquisition process, we reviewed DOD requirements setting policy and determined the extent to which shipbuilding programs set requirements in accordance with this policy. In doing so, we assessed the extent to which DOD policy aligned with fleet experience and captured all factors that influence ship availability and analyzed any discrepancies. We then assessed the extent to which the Navy set sustainment requirements that contributed to well- informed decision-making throughout the acquisition process and in accordance with DOD policy and Navy guidance. To assess how accurately the Navy measures operational availability and materiel availability outcomes, we reviewed the Navy’s operational availability measurements as reported in Selected Acquisition Reports to Congress, and compared these values to fleet reliability data and casualty reports, as well as information about the ships’ performance obtained in interviews with Chief of Naval Operations and NAVSEA officials. To assess the extent to which shipbuilding programs effectively identify and evaluate sustainment costs and risks in key acquisition planning documents, we evaluated the Navy’s development and use of life-cycle cost estimates, Life-Cycle Sustainment Plans and Independent Logistics Assessments. To evaluate the Navy’s development of O&S cost estimates, we reviewed the life-cycle cost estimates created when programs were in development and compared them to updated estimates of O&S costs reported in Selected Acquisitions Reports and Navy provided data. We adjusted program estimates for quantity to more accurately capture cost growth between initial and current O&S estimates. Further, we adjusted the estimates for inflation to compare the O&S estimates as accurately as possible. For programs that experienced O&S cost growth, we interviewed program officials and Navy cost estimators to determine the process that the Navy’s cost estimators used to build O&S cost estimates for shipbuilding programs and to discuss the reasons for cost growth. We also reviewed DOD cost estimation guidance to determine whether the cost estimators and programs complied with its requirements. While we have previously found issues with the reliability of the Navy’s cost estimates, we believe that the cost estimates we reviewed are sufficiently reliable for the purposes of this report. To evaluate the Navy’s use of key sustainment planning documents, we reviewed LCSPs and ILAs for programs in our scope. We interviewed program, NAVSEA, fleet, and maintenance officials to determine the extent to which the LCSPs and ILAs for those programs were used to plan for sustainment, including whether these documents identified and mitigated sustainment risks. We compared the results of the ILAs to realized ship sustainment problems that we identified through interviews shipbuilding program officials, in-service program officials, engineers, and fleet organizations, as well as to analyses of ship and system performance data from many Navy organizations. To evaluate the extent to which Navy and Congressional leadership has insight into and considers sustainment planning and outcomes, we examined the Navy’s Gate review process and Congress’ Nunn-McCurdy breach process. To assess the Navy’s use of the Gate review process, we reviewed Navy acquisition policy governing the reviews, as well as the briefings and meeting minutes from reviews for programs in our scope from fiscal years 2014 through 2018. We compared the content of the briefings and meeting minutes to the acquisition policy to determine the extent to which required sustainment topics were briefed and discussed at each review and identified other mentions of sustainment issues that were outside the scope of the policy requirements. We also reviewed a recent revision to Navy acquisition policy that creates a Gate 7 review for sustainment and interviewed senior Navy officials to obtain their perspectives on how Gate 7 will affect ship sustainment. To assess Navy leadership’s effectiveness in holding shipbuilding programs’ accountable for achieving sustainment outcomes using Acquisition Program Baselines (APB), we reviewed statute that established the APB as well as the findings of the Section 809 Panel, which recommended the creation of the SPB to supplement the APB. We also interviewed Navy officials involved in developing the SPB framework in accordance with the Panel’s recommendations to obtain information on their work. To determine what information Navy shipbuilding programs are required to provide to Congress about sustainment cost issues during the acquisition process, we reviewed the statutory requirements found in Nunn-McCurdy, a key Congressional oversight tool requiring information about baselines and cost estimate growth. We also assessed how the Nunn-McCurdy breach influenced programs’ management of acquisition and sustainment costs by interviewing Navy officials in the shipbuilding program offices, Office of the Chief of Naval Operations, and ASN (RD&A) offices, among others. Additionally, we reviewed O&S cost growth for programs in our scope and compared the percent increase to the 50 percent cost growth threshold used for Nunn-McCurdy acquisition cost breaches to determine if the sustainment cost growth was of a magnitude the Congress considers critical for acquisition costs. To assess how shipbuilding programs leverage PSMs during the acquisition process, we reviewed DOD and Navy acquisition guidance governing the roles and responsibilities of program offices, program managers, and PSMs. We interviewed officials from shipbuilding programs in our scope about their priorities and responsibilities throughout the life cycle of a ship class. Further, we reviewed legislation creating the PSM role, DOD and Navy acquisition guidance regarding PSMs, prior GAO reporting on PSMs, and interviewed PSMs from programs in our scope. We compared the key acquisition activities that legislation requires PSMs to participate in with the activities the PSMs reported they had participated in. We also compared DOD and Navy guidance on assigning PSMs to a program office to when program officials told us the PSMs needed to be assigned to be effective. We also reviewed findings that NAVSEA logistics officials reached about the authority and effectiveness of PSMs. We conducted this performance audit from April 2018 to March 2020 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: List of Fleet-Identified Ship Class Problems That Required More Sustainment Resources than Planned GAO Response to ASN (RD&A)’s Letter In addition to responding to our recommendations, the Assistant Secretary of the Navy for Research, Development, and Acquisition (ASN (RD&A)) provided observations on a number of issues related to the findings in our report. In his letter, the ASN (RD&A) agreed with our recommendations but sought to add context to our report’s conclusion that the Navy can save billions by improving its consideration of sustainment throughout the acquisition process. Our response to the ASN (RD&A)’s letter is as follows. In his letter, the ASN (RD&A) highlighted a number of changes that the Navy has instituted over the last 10 years to improve sustainment planning, including policies pertaining to life cycle sustainment plans and independent logistics assessments, strengthening the role of the Product Support Managers, and establishing a new Gate 7 review focused on sustainment. We agree that the Navy’s framework for including sustainment planning in the acquisition process offers promise and we discuss these policies and processes in depth in this report. However, we found considerable weaknesses in the Navy’s application of its own policies. Specifically, we found that the Navy did not provid a thorough assessment of the sustainment implications and risks in its LCSPs and ILAs and Product Support Managers aften are not assigned until well into a shipbuilding program thereby limiting their influence on early acquisition decisions. While adding a Gate 7 offers benefits, it is not a substitute for discussions about sustainment concepts during earlier Gates, when key long-term decisions are being made. Our findings and recommendations demonstrate that DOD and the Navy should better use the policies and processes it currently has, including the Gate reviews, as well as Product Support Managers, LCSPs and ILAs, to improve their understanding of how their acquisition decisions will affect sustainment. In his letter, the ASN (RD&A) stated that many of the Navy’s ship programs were designed with sustainment initiatives early in the acquisition process and, further, acknowledged that these initiatives did not achieve efficiencies as initially planned. We agree with both of these points, as we discuss in depth in this report. Whereas the ASN (RD&A) indicated in his letter that leadership, philosophical, and technology changes can lead to outcomes that were not originally envisioned, we found that these initiatives largely failed because, early in the acquisition process, the Navy did not sufficiently assess the costs or evaluate the risks associated with pursuing these initiatives. Absent such analysis, the Navy did not mitigate the risks that threatened their success. The ASN (RD&A) highlighted several examples of sustainment initiatives considered early in the acquisition process for several ships. We believe that these examples (many of which we discuss in our report) serve to further highlight our findings. For example: The ASN (RD&A) discussed the use of a “full service contractor,” meaning performance-based logistics for LPD 17 class ships. According to the ASN (RD&A), while this approach had been successfully used for aircraft, the Navy had never applied it to ships. As we state in our report, in attempting to use performance-based logistics for several shipbuilding programs including LPD 17 class ships, the Navy did not consider the challenges in implementing this radical departure from traditional ship maintenance and did not consult the fleet on this change until after ships were delivered. The Navy’s life-cycle sustainment plans and cost estimates for several shipbuilding programs did not articulate how much the performance-based logistics approach was likely to cost or what sustainment outcomes the Navy expected. For instance, for three out of the four programs that pursued performance-based logistics, the Navy learned that this approach was cost-prohibitive once it began seeking contractors to sustain its ships. The ASN (RD&A) stated that the Navy’s focus on Ford class sustainment has reduced sustainment costs and labor by an estimated $4 billion across the Ford class carriers compared to the previous class of carriers. However, it is too early to tell how much the Navy will save compared to the cost of its previous class of carriers because the Navy’s fleet has yet to operate the new carrier. Further, while the O&S estimated for the Ford class may currently be lower than the previous carrier class, our report notes that the O&S costs for the Ford class carrier program are nearly $46 billion more than initially estimated. Finally, in his letter, the ASN (RD&A) stated that the Navy plans to correct the vast majority of CVN 78 sustainment problems (including those we identified in this report) with ship construction funding—and these cost will not be passed on to the fleet. The $4.2 billion to address the 150 problems that we identified in this report already excludes all ship construction funding and also excludes corrections on CVN 78. Our calculation of $4.2 billion only includes the costs to correct the problems that are not funded using ship construction funding. We agree with the ASN (RD&A)’s assertion that external factors can take place over the lengthy time needed to design and build a new ship that can lead to changes that were not initially envisioned. While the Navy cannot prepare for all of the unknowns, it can critically evaluate sustainment assumptions that form the basis of its shipbuilding programs early in the acquisition process. Such analysis could significantly improve the Navy’s ability to response to changes over time and increase the likelihood of success. Further, critical analysis could also help decision makers determine when an initiative is too risky before implementing it on an entire shipbuilding program. In its letter, the ASN (RD&A) also states that a careful reading of the early program documentation demonstrates that sustainment stakeholders were involved in the acquisition process. We reviewed available acquisition documents for 11 shipbuilding programs in the last 20 years and found that sustainment leadership, specifically the CNO and other in OPNAV, attended meetings and approved sustainment planning documents. However, we found that sustainment was rarely discussed during early acquisition meetings—even when the planned shipbuilding programs sought new sustainment initiatives. Further, we reviewed thousands of Navy documents and met with over 100 Navy organizations and found that sustainment organizations across the Navy that are responsible for ship sustainment have a limited role in the acquisition process, even when having such a role could have likely prevented many of the problems we discuss in the report. As we state in our report, the quantity and breadth of the 150 problems we found— resulting in billions of dollars in unexpected costs, maintenance delays, and unreliable ships—suggest that existing policies and guidance have not ensured that new ships are reliable and can be sustained as planned. We are concerned that the ASN (RD&A)’s letter is an indication that the Navy’s shipbuilding program offices will not take the necessary action to improve sustainment planning during the acquisition process. The ASN (RD&A)’s letter did not mention the recent establishment of a new Deputy Assistant Secretary for Sustainment that we discuss in our report. We believe that this office has the opportunity to contribute to improved outcomes byp providing leadership to ensure that sustainment considerations are critically evaluated during the acquisition process. Absent such leadership, the Navy is at risk of continuing to provide ships to the fleet that are incomplete, unreliable, and cost more than expected to maintain. Appendix IV: GAO contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition the contact name above, the following staff members made key contributions to this report: Diana Moldafsky, Assistant Director; Laurier Fish, Analyst-in-Charge; Jillian Schofield; Sarah Evans; Lori Fields; Ann Halbert-Brooks; Joshua Garties; Laura Greifner; Tara Kumar; Shivarthn Maniam; Alexis Olson; Kya Palomaki; Anne Louise Taylor; and Tonya Woodbury. Carl Barden; Brian Bothwell; Anna Irvine; and Jean McSween also made contributions to this report. Related GAO Products Defense Acquisitions: Senior Leaders Should Emphasize Key Practices to Improve Weapon System Reliability. GAO-20-151. Washington, D.C.: January 14, 2020. Guided Missile Frigate: Navy Has Taken Steps to Reduce Acquisition Risk, but Opportunities Exist to Improve Knowledge for Decision Makers. GAO-19-512. Washington, D.C.: August 9, 2019. DOD Acquisition Reform: Leadership Attention Needed to Effectively Implement Changes to Acquisition Oversight. GAO-19-439. Washington, D.C.: June 5, 2019. Columbia Class Submarine: Overly Optimistic Cost Estimate Will Likely Lead to Budget Increases. GAO-19-497. Washington, D.C.: April 8, 2019. DOD Depot Workforce: Services Need to Assess the Effectiveness of Their Initiatives to Maintain Critical Skills. GAO-19-51. Washington, D.C.: December 14, 2018. Navy and Marine Corps: Rebuilding Ship, Submarine, and Aviation Readiness Will Require Time and Sustained Management Attention. GAO-19-225T. Washington, D.C.: December 12, 2018. Navy Readiness: Actions Needed to Address Costly Maintenance Delays Facing the Attack Submarine Fleet. GAO-19-229. Washington, D.C.: November 19, 2018. Navy Readiness: Actions Needed to Address Costly Maintenance Delays Affecting the Attack Submarine Fleet. GAO-19-192C. Washington, D.C.: October 31, 2018. Navy Shipbuilding: Past Performance Provides Valuable Lessons for Future Investments. GAO-18-238SP. Washington, D.C.: June 6, 2018. Columbia Class Submarine: Immature Technologies Present Risks to Achieving Cost, Schedule, and Performance Goals. GAO-18-158. Washington, D.C.: December 21, 2017. Navy Readiness: Actions Needed to Address Persistent Maintenance, Training, and Other Challenges Affecting the Fleet. GAO-17-809T. Washington, D.C.: September 19, 2017. Naval Shipyards: Actions Needed to Improve Poor Conditions That Affect Operations. GAO-17-548. Washington, D.C.: September 12, 2017. Weapon Systems Management: Product Support Managers’ Perspectives on Factors Critical to Influencing Sustainment-Related Decisions. GAO-17-744R. Washington, D.C.: September 12, 2017. Navy Readiness: Actions Needed to Address Persistent Maintenance, Training, and Other Challenges Facing the Fleet. GAO-17-798T. Washington, D.C.: September 7, 2017. Navy Shipbuilding: Policy Changes Needed to Improve the Post-Delivery Process and Ship Quality. GAO-17-418. Washington, D.C.: July 13, 2017. Navy Force Structure: Actions Needed to Ensure Proper Size and Composition of Ship Crews. GAO-17-413. Washington, D.C.: May 18, 2017. Navy Ship Maintenance: Action Needed to Maximize New Contracting Strategy’s Potential Benefits. GAO-17-54. Washington, D.C.: November 21, 2016. Littoral Combat Ship: Need to Address Fundamental Weaknesses in LCS and Frigate Acquisition Strategies. GAO-16-356. Washington, D.C.: June 9, 2016. Defense Inventory: Further Analysis and Enhanced Metrics Could Improve Service Supply and Depot Operations. GAO-16-450. Washington, D.C.: June 9, 2016. Military Readiness: Progress and Challenges in Implementing the Navy’s Optimized Fleet Response Plan. GAO-16-466R. Washington, D.C.: May 2, 2016. Navy and Coast Guard Shipbuilding: Navy Should Reconsider Approach to Warranties for Correcting Construction Defects. GAO-16-71. Washington, D.C.: March 3, 2016. Acquisition Reform: DOD Should Streamline Its Decision-Making Process for Weapon Systems to Reduce Inefficiencies. GAO-15-192. Washington, D.C.: February 24, 2015. Ford-Class Aircraft Carrier: Congress Should Consider Revising Cost Cap Legislation to Include All Construction Costs. GAO-15-22. Washington, D.C.: November 20, 2014. Weapon Systems Management: DOD Has Taken Steps to Implement Product Support Managers but Needs to Evaluate Their Effects. GAO-14-326. Washington, D.C.: April 29, 2014. Navy Shipbuilding: Opportunities Exist to Improve Practices Affecting Quality. GAO-14-122. Washington, D.C.: November 19, 2013. Trends in Nunn-McCurdy Cost Breaches for Major Defense Acquisition Programs. GAO-11-295R. Washington, D.C.: March 9, 2011. Defense Management: DOD Needs Better Information and Guidance to More Effectively Manage and Reduce Operating and Support Costs of Major Weapons Systems. GAO-10-717. Washington, D.C.: July 20, 2010. Best Practices: High Levels of Knowledge at Key Points Differentiate Commercial Shipbuilding from Navy Shipbuilding. GAO-09-322. Washington, D.C.: May 13, 2009. Defense Logistics: Improved Analysis and Cost Data Needed to Evaluate the Cost-effectiveness of Performance Based Logistics. GAO-09-41. Washington, D.C.: December 19, 2008. Defense Acquisitions: Cost to Deliver Zumwalt-Class Destroyers Likely to Exceed Budget. GAO-08-804. Washington, D.C.: July 31, 2008. Defense Acquisitions: Realistic Business Cases Needed to Execute Navy Shipbuilding Programs. GAO-07-943T. Washington, D.C.: July 24, 2007.