Background The Navy oversees the planning and execution of non-nuclear surface ship repair and maintenance through several organizations (see fig. 1), including the following: The Chief of Naval Operations is the senior military officer of the Department of the Navy and is responsible to the Secretary of the Navy for the command, utilization of resources, and operating efficiency of the operating forces of the Navy and of the Navy shore activities assigned by the Secretary. The Assistant Secretary of the Navy for Research, Development and Acquisition, as the Navy Acquisition Executive, has overall authority, responsibility, and accountability for all acquisition and sustainment functions and programs, including surface ship repair and maintenance. Naval Sea Systems Command (NAVSEA) and its subordinate organizations maintain ships to meet fleet requirements within cost and schedule parameters, among other duties for combat systems design and operation. Types of Availabilities for Surface Ship Repair The Navy contracts with private shipyards and other firms—collectively known as the ship repair industrial base—for the repair and maintenance of non-nuclear surface ships. This work may be performed in either government-owned or contractor-owned facilities, potentially including shipyards with piers, cranes or facilities for pipefitting and valve repair. Certain types of work, such as inspecting, repairing or otherwise maintaining a ship’s hull, might require placing a ship in the ship repair contractor’s dry dock. Ship repair availabilities can range from a few weeks to years depending on the extent of work required and degree of complexity. The types of availabilities include the following: Chief of Naval Operations (CNO) availabilities accomplish major repair work. This includes industrial maintenance requiring complex processes to complete restorative work, such as structural, mechanical, and electrical repairs. These may include modernization work to upgrade a ship’s capabilities along with repair work, and can last for over a year. Larger contractors typically execute these types of availabilities rather than small businesses. Continuous Maintenance availabilities (CMAV) accomplish non- major repair work, which includes routine maintenance work requiring relatively little time compared to CNO availabilities—typically only weeks to a few months in duration. Small business contractors commonly execute CMAVs, and, at some ports, larger companies that have contracts for CNO availabilities also take on this type of work. NAVSEA’s Oversight of MAC-MO Strategy Implementation Within NAVSEA, several organizations oversee MAC-MO strategy implementation (see fig. 2), including key functions such as contract administration, program management, and planning for future availabilities. Availability Planning under the MAC-MO Strategy Prior to awarding a contract for ship repair work under MAC-MO, the Navy plans and defines requirements for upcoming availabilities as depicted in figure 3 below. Availability Execution under the MAC-MO Strategy Contracting for availability execution under the MAC-MO strategy differs from that under the Navy’s previous strategy, known as Multi-Ship, Multi- Option (MSMO), in several key ways, including by calling for: establishment of fixed contractual prices and completion time frames for an upcoming availability, rather than payment of contractors’ incurred costs; use of a third-party planning contractor under a cost-reimbursement contract to define contract specifications, rather than relying on planners employed by ship repair contractors; and award of indefinite delivery contracts to multiple contractors that can then compete for future availabilities, rather than all availabilities for a particular class of ships going to one contractor. Under the MAC-MO strategy, the Navy normally places fixed-price orders for availabilities with expected durations of 10 months or less using indefinite delivery/indefinite quantity (IDIQ) contracts. IDIQ contracts do not specify exact times for delivery of supplies or services at contract award; the Navy establishes those via orders placed during contract performance. With MAC-MO, the Navy generally solicits and awards contracts for five-year periods to a set of qualified contractors at specific home ports. These periods include an initial execution year and four additional option years. As a result, several qualified contractors are available to subsequently compete for availabilities in a specific home port under firm-fixed-price availability delivery orders until contract expiration. Availabilities that the Navy expects to last more than 10 months are not restricted to the ships’ home port. This allows for contractors outside the home port to compete for this work. The Navy then awards contracts for these coast-wide availabilities as stand-alone contracts to a single prime contractor, potentially at a port different from the home port of the ship. Shorter availabilities may be limited to the home port area provided there is adequate competition, which the Navy defines as the presence of two or more qualified bidders. If adequate competition is not available in the home port area, the geographic area for solicitation is expanded equally in all directions until adequate competition exists. Figure 4 below depicts contracting processes used for soliciting and awarding work under the MAC-MO strategy. In November 2016, we reviewed the Navy’s implementation of the MAC- MO strategy through pilot maintenance periods, including its potential benefits and effects on the industrial base. We found MAC-MO had some potential benefits compared to the previous MSMO contracting strategy, including increased opportunities for competition and control of costs through fixed-price contracts. We additionally found that some contractors saw uncertainty associated with the need to continually compete for work, which could result in decisions to reduce their workforce and facilities, and the stability of ship repair workloads in their ports, irrespective of contract type. The Navy Has Met Most of Its MAC-MO Goals, but Schedule Delays Persist The Navy has achieved some, but not all, of the goals it set under the MAC-MO strategy. Among the achievements, the Navy provides more opportunities for competition—and received more offers—under MAC-MO than under the prior strategy. Further, MAC-MO’s fixed-price contracts help enable the Navy to ensure quality of work, and we found no evidence of deficient work at availability completion in our review of four completed case studies. At the same time, the Navy also desired improved availability cost and schedule outcomes under the MAC-MO strategy. The Navy’s results in these two areas have been mixed. Through April 2019, the Navy had completed 41 CNO availabilities under its MAC-MO strategy with, on average, 5 percent cost growth and 30 percent schedule growth. Unplanned work, which can often be unavoidable in ship repair, has detracted from both cost and schedule performance. MAC-MO Strategy Has Increased Opportunities for Competition and Helps the Navy Ensure Quality Standards Are Met The MAC-MO strategy has provided more opportunities than MSMO for competition by awarding a delivery order for each ship repair availability. The Navy has competed over 500 delivery orders under the MAC-MO strategy from April 2015 to March 2019. This represents a departure from the MSMO strategy under which a single contract was awarded to one contractor to execute multiple availabilities for a class of ship. The MAC- MO strategy also allows small businesses in Norfolk and San Diego to compete for noncomplex maintenance. Previously, under the MSMO strategy, small businesses said that they were more likely to work as subcontractors for the businesses that held one of the MSMO contracts. Navy officials have since stated that small businesses are now acting as prime contractors. The Navy has also achieved competition for soliciting its delivery orders under the MAC-MO strategy. According to our analysis of data from the Federal Procurement Data System-Next Generation (FPDS-NG), from the start of the MAC-MO strategy (April 2015) through March 2019, at least 78 percent (435 of 554) of MAC-MO awards solicited within home ports received two or more offers. Further, in the 18 percent of instances when the Navy awarded a delivery order after receiving only a single offer, it may have attained the benefits of having solicited that delivery order in a competitive environment. Table 1 shows the number of offers for both complex and noncomplex MAC-MO awards through March 2019. The MAC-MO strategy also gives the Navy flexibility to ensure that a contract’s quality requirements are met under a fixed price by the time of availability completion. The Navy identified improving the quality of workmanship as a goal when it switched from MSMO to MAC-MO. The previous MSMO contracting strategy relied on use of cost-reimbursement contracts, which only require the government to reimburse the contractor its allowable incurred costs, regardless whether the contractor completed the work. The MAC-MO strategy uses firm-fixed-price contracts, which provide for a price that is not subject to any adjustment on the basis of the contractor’s cost experience in performing the contract. This contract type places upon the contractor maximum risk and full responsibility for all costs and performance, including meeting the quality requirements of the contract. NAVSEA officials stated that in the event that the contractor doesn’t meet the quality terms of the contract, the Navy has two options: (1) require the contractor to complete the deficient work, at the contractor’s cost, to meet the specifications or (2) reduce the contract price to reflect the reduced value of the services performed and descope the related work requirements from the existing contract for performance on a future availability. Our review of availability completion reports from the four case study availabilities that were complete at the time of our assessment, out of six total, showed no instances where the Navy accepted quality deficiencies at availability completion. Navy contracting officials stated that in a firm- fixed-price contracting environment, they would not agree to accept deficient work without first obtaining concessions from the performing contractor, which would require modifying the delivery order. In one of these availabilities, we found evidence that the Navy elected to descope a non-option work item and defer it to a future availability. The maintenance team stated that this decision followed poor planning of the work item, which would have caused delays in completing the availability if not deferred. We also discussed these four availabilities with the responsible Navy maintenance teams, and none of those teams reported to us any deficient work at the time each availability completed. CNO Availabilities Have Experienced Limited Cost Growth under the MAC- MO Strategy, but Schedule Delays and Growth Work Persist Cost Between April 2015, when the Navy implemented the MAC-MO strategy, and April 2019, the Navy completed 41 CNO availabilities with an average cost growth per availability of 5 percent, or $1.7 million in fiscal year 2020 dollars. However, more than half of these availabilities (21 of 41) were completed at a lower cost than the Navy initially estimated. The cost growth of the remaining CNO availabilities (20 of 41) ranged between 1 percent and 78 percent and drove the aggregate average increase. Figure 5 shows the variation in cost performance, or the actual cost compared to the Navy’s estimate, for the 41 CNO availabilities. Figure 6 shows the cost performance, or actual cost compared to the Navy’s estimate, for the 41 CNO availabilities grouped by their location. Figure 7 shows cost performance, or actual cost compared to the Navy’s estimate, for the 41 CNO availabilities grouped by ship class. Schedule Between the start of the MAC-MO strategy in April 2015 and April 2019, the Navy completed 41 CNO MAC-MO ship repair availabilities with an average schedule growth, or actual number of days from availability start to completion, compared to the Navy’s estimate, of 30 percent, or 64 days. Twelve of 41 availabilities finished on time, and none finished ahead of schedule. In addition, two availabilities more than doubled in length, with one finishing with 123 percent schedule growth. We discuss some factors that can contribute to schedule growth below. Figure 8 shows the schedule growth for individual CNO availabilities. Figure 9 shows the schedule growth for the 41 CNO availabilities grouped by location. Figure 10 shows the schedule growth for the 41 CNO availabilities grouped by ship class. Navy officials stated that one potential source of delays is unplanned work, which consists of both growth work and new work. The Navy defines growth work as additional work that is identified or authorized after contract award that is related to a work item included in the original contract. We previously found that growth work contributed to cost and schedule increases, and it remains a contributing factor. Navy officials stated they expect some growth work in availabilities, as officials stated that certain tasks are difficult to fully scope within the original contract. As an example, one official stated that they cannot fully inspect ballast tanks and accurately write work specifications for their repair until the ship is at the repair yard and the availability has begun. Alternatively, the Navy defines new work as any additional work that is identified or authorized after contract award that is not related to a work item included in the original contract. Maintenance team officials stated that new work can originate when an item that needs repair breaks or the maintenance team first discovers it after the Navy awards the contract. The Navy can also add new work to an availability whenever it sees fit. In our six case study availabilities, we found that five added growth work, including examples of growth items that the Navy considered unavoidable. Our analysis of RMC data showed that the USS Stout (DDG 55) CNO Availability had 60 instances of growth work that the Navy considered unidentifiable prior to the start of the availability, including welding for the fuel tanks and repair to the bulkheads. The maintenance team did not consider these growth items to be unusual. Some non-CNO availabilities, like Continuous Maintenance availabilities, are smaller in scope and less susceptible to growth work. Maintenance team officials at SERMC consequently stated that they can often complete CMAVs on schedule. We found that the Navy completed one of our case study availabilities, the USS Iwo Jima CMAV, on schedule, and maintenance team officials stated they had time to add three new work items to the availability. Figure 11 describes the USS Stout (DDG 55) case study. Figure 12 describes the USS Iwo Jima (LHD 7) case study. According to Navy officials, managing growth work under firm-fixed-price contracts has contributed to schedule delays. In our November 2016 report on the Navy’s transition to the MAC-MO strategy, we described the importance of contractors and RMC staff negotiating contract changes and agreeing on costs in a timely manner in order to minimize schedule impact. In our current review, Navy officials stated that negotiating change orders for unplanned work under MAC-MO is more difficult and time consuming than under the prior MSMO strategy because the Navy can no longer direct the contractor to continue to work without agreeing on the cost. In one of our case study availabilities, the USS Whidbey Island (LSD 41) CNO Availability, the maintenance team officials stated that they had difficulties negotiating contract changes. As a result, the officials stated that the Navy used unilateral modifications to direct the contractor to execute growth work items and avoid further schedule disruptions. See Figure 13 below for more detail on the USS Whidbey Island (LSD 41) case study. The Navy Has Taken Action to Respond to MAC-MO Lessons Learned, but Funding Process Continues to Contribute to Delays The Navy recognizes the negative schedule outcomes it currently faces with MAC-MO strategy implementation and has worked to mitigate them. It has implemented new contracting provisions and is moving key availability milestones to earlier in the process in an effort to better plan availabilities and facilitate their on-time completion. The Navy has also tried to better coordinate with the third-party planner to plan for availabilities and improve schedule performance. Statutory requirements and their implementation, however, have hindered the Navy’s ability to further mitigate schedule delays. Specifically, the Navy must obtain approval from the Under Secretary of Defense (Comptroller) before funding growth work that occurs in subsequent fiscal years and exceeds $4 million—an amount established under a 1990 law. Late last year, Congress established a pilot program in fiscal year 2020 that affords the Navy the ability to use procurement funds for availabilities, and these funds remain available for obligation for three years. A congressional statement accompanying the appropriations law that established the pilot program states that the Navy is to submit quarterly reports on the execution of ship availabilities funded through the pilot program. The Navy Has Taken Action to Address Key Lessons Learned with MAC-MO Implementation In our November 2016 report, we identified several key lessons learned stemming from MAC-MO pilot maintenance availabilities. When we revisited these lessons learned during interviews with Navy officials, they discussed two persistent MAC-MO strategy attributes that remain points of emphasis for lessons learned from 2016. These strategy attributes, namely the use of firm-fixed-price contracts and the use of a third-party planner, led to two new key lessons learned and another ongoing lesson learned from our 2016 report. Most of these center on the importance of the Navy accurately planning for and anticipating needs during availabilities in order to avoid schedule delays—a theme that was also evident in our November 2016 report. According to NAVSEA leadership officials, the Navy primarily relies on two activities to determine lessons learned and identify actions that NAVSEA needs to take to improve ship repair maintenance, including under the MAC-MO strategy. Surface Team One compiles lessons learned that the individual RMCs recommend and reviews the implementation and status of actions to address those lessons learned. Performance to Plan (P2P) is a data-centric, analytical approach the Navy uses for a variety of improvement initiatives, including ship maintenance, to clearly characterize availability performance goals and develop solutions to improve availability duration outcomes. As shown in Table 2 below, the Navy has developed new contracting provisions and milestones to respond to lessons learned the Navy has identified. Additional information on each action follows the table. The Navy Has Recently Implemented New Contract Provisions and Revised Milestones Based on Lessons Learned to Reduce Disruptive Effects from Growth Work In 2018, the Navy began implementing two new contract provisions originating from lessons learned regarding the MAC-MO strategy—Small Dollar Value Growth and Level of Effort to Completion—in an effort to mitigate schedule delays typically associated with growth work. Small Dollar Value Growth (SDVG) The November 2018 SDVG provision specifically addressed schedule delays due to growth items that cost $25,000 or less. Under SDVG, during availability planning the Navy and contractor agree on a set price to be used anytime a growth work item equal to or under the $25,000 threshold is added to the work specification. This provision eliminates the need for the Navy and the contractor to engage in time-consuming negotiations on small dollar items during the availability. According to the Navy’s 2018 biennial assessment, small dollar growth work negotiations accounted for around 70 percent of all contract changes. According to Navy documentation, contract negotiations for small dollar growth work caused delays of up to a week. In our discussion with officials from the USS Whidbey Island maintenance team, they reported that the availability required 972 contract changes, which they suggested SDVG would have helped expedite. The Navy’s SDVG policy memo states that in using SDVG, the contractor can now typically begin work on the growth item 24 hours after discovery. Figure 14 describes the SDVG process. While it can expedite work on smaller dollar value items, the use of SDVG carries cost risk for the Navy and the executing contractors, which RMC leadership officials and contractor representatives acknowledged. According to these officials, under SDVG the Navy, at times, will likely pay more for growth items than it would if it devoted increased time to negotiate prices, with the same being true for the contractors. For example, the Navy awarded a contract delivery order for the USS Bulkeley (DDG 84) availability in February 2019 that included SDVG. The SDVG line item provided for up to 291 changes for growth during that availability at a firm-fixed-price of $7,144 per change based on historical needs of similar availabilities. This meant that the Navy could use SDVG up to 291 times during the availability, and each of those growth items would cost the Navy $7,144 regardless of whether the actual cost to the contractor underran or exceeded that amount. After the contractor identifies the in-scope growth item, the Navy only must determine that the cost is equal to or less than the $25,000 threshold in the contract. Nonetheless, Navy officials expressed that the benefit of significantly decreased negotiation time outweighs the potential cost risk. Level of Effort (LOE) to Completion As reflected in table 2, the Navy implemented a second new contract change process, known as LOE to Completion, in November 2018. This process is used for growth work items when the price exceeds the SDVG threshold of $25,000. LOE to Completion allows the Navy, within the already awarded contract for the availability, to fund growth work that contractors regularly discover during availability execution without having to separately negotiate each item. Through LOE to Completion, RMC leadership officials stated they have decreased negotiations and schedule delays during availability execution. LOE to Completion allows the Navy to obligate funding for labor-hours and material costs for estimated growth work at the time of award, rather than having to obtain appropriate funds after repair work begins. The Navy can then use those labor-hours and materials for individual growth work items over the course of the availability. According to RMC leadership officials, this provision allows them to avoid incurring additional delays. To establish the amounts of funding, the Navy reviews historical cost for growth work by class type and whether the availability is a docking or non-docking availability. For example, the Navy provided up to 134,002 work hours and $1.4 million for materials under the LOE to Completion contract process for the USS Bulkeley (DDG 84) availability. Figure 15 describes the LOE to Completion process. Because the Navy just recently implemented this process in November 2018, it has collected only limited data to date on its effectiveness. However, as described in figure 16, an availability involving complex ship repair work for the USS Princeton (CG 59) included contract terms that Navy officials described as a precursor to LOE to Completion. Revised Availability Milestones In August 2019, the Navy began targeting award of delivery orders for individual availabilities 120 days prior to the scheduled work start date. Previously, the Navy awarded these delivery orders 60 days prior to the scheduled work start date. According to Navy supply officials, awarding the delivery orders 120 days prior to the start of scheduled work allows the officials involved in the planning process to procure long lead-time materials early enough so that material delays do not impact schedule—a challenge they cited under the 60-day schedule. Figure 17 shows how the change awarding delivery order 120 days before work is scheduled to begin will affect availability milestones. As reflected in figure 17, another change is that long lead time materials are now ordered 365 days ahead of the start of work, as opposed to the prior schedule of 170 days ahead. Navy supply officials said that some materials require lead times from 1 year to 18 months. Consequently, ordering these materials 170 days before an availability begins increased the likelihood that they would arrive too late to fulfill the Navy’s stated goal of procuring all materials 30 days prior to the start of repair work. Unless repair work requiring these materials is nonessential and can be deferred to a future availability, these material delays can delay completion of availabilities by several months. Several ship repair contractor representatives we interviewed with pointed to long lead-time materials as drivers for schedule growth. While noting the potentially positive effects of shifting award date to 120 days before the availability begins, Navy officials also raised some challenges. They said that locking ship repair requirements almost a full year before an availability actually begins means that the Navy could finalize a ship’s upcoming availability work specifications before a ship even begins its next deployment. During this deployment, equipment breakages or other deficiencies not anticipated and subsequently not included in the work package could arise on the ship, all of which would likely become growth work during the availability. This new time frame for delivery orders has only recently been implemented. The first MAC-MO delivery order awarded 120 days prior to the start of work occurred in January 2020, with another awarded since then. The Navy was scheduled to award availabilities 120 days prior to the start of work in November 2019, but, according to Navy officials, lacked necessary funds to award several availabilities due to the continuing resolution in place at the time. The Navy is not yet certain whether awarding delivery orders earlier will improve the Navy’s ability to provide long lead-time materials on time. The Navy Has Taken Action to Address Availability Planning Lessons Learned, but Views Are Mixed on the Results Both the Navy and the third-party planner recognize the need for the two parties to work closely together to produce the best specifications and work packages possible under MAC-MO. As within the Navy, third party planner staff also seek to identify lessons learned, in order to improve the quality of ship repair specifications they produce. According to third-party planning contractor representatives, they monitor contract changes involving growth work, assess whether that growth is due to planning deficiencies or other causes, and then identify lessons learned, which they use to improve their specification writing process. For example, contractor representatives stated that they used lessons learned during the USS Bainbridge (DDG 96) availability to create a template for a section of the forecastle deck plate. This template could be used on future availabilities for ships of the same destroyer class, providing potential cost savings to future availabilities. However, RMC officials across the three ports implementing the MAC-MO strategy expressed concerns over the quality of third-party planning contractor specifications used in ship repair availability solicitations and contracts. They stated that the specifications developed by the third-party planning contractors have frequently included errors and discrepancies. As a result, the maintenance teams have had to work with the third-party planning contractor to resolve the issues prior to award. According to RMC officials, maintenance teams within a given port have their own preferences with regard to how the third-party planning contractor writes specifications. Consequently, a specification written and approved in one RMC is sometimes deemed inadequate within another RMC. Figure 18 describes how specification deficiencies and other events affected a USS Roosevelt (DDG 80) availability. Even with the issues that Navy maintenance teams have encountered with third-party planner-developed specifications, RMC officials stated that they continue to find ways to enhance their coordination with the third-party planning contractor. For instance, according to MARMC officials, they found that when availability maintenance teams physically worked alongside third-party planning contractor staff, the planning process went much more smoothly. After SWRMC officials learned of this practice, SWRMC’s maintenance teams were co-located with the third- party planning contractor staff in an effort to improve its process as well. According to RMC staff, they found that having all parties coordinating closely in the planning process to be an effective way to mitigate some of the specification writing issues. In contrast to RMC officials, from NAVSEA leadership officials’ perspective, the third-party planning contractor is currently accomplishing the goals the Navy has set forth and has provided accurate enough specifications to earn the incentive fees outlined in its contract. The NAVSEA officials noted that the contractor has also received annual incentive fees for providing recommendations to the Master Specification Catalog utilized by the Navy to incorporate lessons learned and improve specifications written at all RMCs. Congressional Action Offers Relief to the Navy’s Lengthy Funding Approval Processes, but Navy Does Not Have Plans to Assess Results Historically the Navy has used its operation and maintenance account to pay for ship repair. By law, those funds have generally only been available for new obligations for one fiscal year—which corresponds with the fiscal year in which the availability contract is awarded– after which the funds expire. In order for the Navy to use any remaining expired funds in the subsequent fiscal year for an in-scope contract change, the executing RMC must request what is called an upward obligation. The Navy can request an upward obligation at the fleet level as long as the request for a specific availability is less than $4 million. RMC officials stated this type of request involves a short process. However, if the upward obligations request exceeds $4 million for an availability, the executing RMC must receive approval from the Office of the Under Secretary of Defense (OUSD) Comptroller. According to RMC leadership officials, this process can take several months. We found that the Navy has requested upward obligations from the OUSD Comptroller 25 times across 14 ship repair availabilities since implementing the MAC-MO strategy in April 2015. In November 2016, we reported that the Navy identified the need for training for staff on how to obtain upward obligations funding. In our interviews with RMC leadership officials and Navy financial officials, some said they now had experience with upward obligations because of their regular need to obtain funding for ship availabilities that crossed fiscal years. Nonetheless, in our discussions with the RMC commanding officers, they described the upward obligations process to obtain OUSD Comptroller approval for upward obligations as cumbersome and unnecessarily complicated. Other Navy officials and contractors echoed these views and highlighted the upward obligations request process as a significant impediment to schedule performance. According to the RMC commanders, it requires several months to successfully execute and complete the upward obligations process for many availabilities because of reviews required within the Navy and the Office of the Secretary of Defense before approval is granted. Navy officials said the process also results in significant delays to the availabilities, as work cannot proceed without funding. For example, of six availabilities for which the Navy provided data, the shortest upward obligations request took 26 days, with the longest request spanning 189 days. Figure 19 describes how for one of our case studies, the USS Chosin (CG 65), the Navy experienced several months of schedule delay due in part to the upward obligations process. Navy officials stated they have attempted to identify legislative solutions to reduce the frequency under which they must obtain upward obligations, given the negative schedule effects this process precipitates. In 2018, the Office of the Assistant Secretary of the Navy (Financial Management and Comptroller), in conjunction with the Office of the Under Secretary of Defense (Comptroller), proposed two legislative initiatives to Congress intended to accomplish this goal. The first of these proposals seeks to raise the legal threshold for ship repair upward obligations requiring Navy and Defense Comptroller approval from $4 million to $10 million. The proposal also provides for a pilot ship availability with these new thresholds, which would allow the Navy to determine the proposal’s effectiveness before fully implementing the new threshold. According to Navy and DOD comptroller officials, this proposal holds merit on several levels. First, the upward obligations threshold has not changed since 1990, when the law implementing the process first passed. The proposed increase to the threshold would account for inflation and subsequent increases in the cost of ship repair over the last 30 years. For example, the average maintenance availability for a DDG 51 Arleigh Burke class destroyer cost $6 million in 1991, but costs $36 million when the Navy proposed the legislative change. Additionally, the scope of the Operations and Maintenance, Navy (O&M) budget has increased by a factor of 2.5 since the law’s 1990 passage. Navy officials believe that increasing the threshold to $10 million would potentially raise this amount to a level corresponding to increases in Navy ship repair budgets since that time. The second proposal would permit Navy O&M funds—which the Navy uses to fund ship repair, among other sustainment-related activities—to be available for the Navy to obligate for up to 2 fiscal years following their appropriation by Congress. Currently, these funds are available to be obligated by the Navy for only 1 year. According to Navy financial officials, since most ship repairs extend into a second year, this proposal would allow ship availabilities to avoid using upward obligations. A senior official with the Office of the Under Secretary of Defense (Comptroller) said that the threshold change was more logical, as the thresholds are no longer practical, and that the logistics of implementing 2-year funding were likely to be more complicated because of the various DOD software systems that would be affected. In December 2019, Congress and the President enacted legislation that— although differing from the Navy’s legislative proposals—is responsive to the Navy’s concerns relating to the process of approving upward obligations more than $4 million in its MAC-MO availabilities. In the Fiscal Year 2020 Consolidated Appropriations Act, Congress established a pilot program that allows the Navy to use the Other Procurement, Navy (OPN) account to fund Pacific fleet surface ship repair availabilities for 2020. Our review of Navy budget documentation shows that the Navy plans to execute 16 pilot availabilities using fiscal year 2020 OPN funds, and it has requested funding for another 26 pilot availabilities in fiscal year 2021. Unlike the Operations and Maintenance, Navy account, which the Navy typically uses to fund ship repair availabilities in 1-year increments, the OPN account provides the Navy with funding that will not expire for 3 years. Consequently, for availabilities the Navy funds through the pilot program, any growth work that necessitates an availability stretching into a second or even third year will avoid upward obligations and the related approval processes, provided sufficient funding remains in the OPN appropriation to cover the work. The joint explanatory statement accompanying the enacted legislation further stated that the Secretary of the Navy is to provide quarterly reports to Congress on the execution of ship availabilities funded through the pilot program in the OPN account. In these quarterly reports, the Navy is to report on the estimated or actual start or end dates of pilot availabilities, as well as the actual funded amount and estimate to complete. The Navy already completes systematic, biennial assessments of MAC- MO implementation, in response to our November 2016 report. While the Navy recognized upward obligations as an issue in its 2018 biennial assessment, the Navy did not examine potential solutions to the schedule delays that these obligations cause. Further, according to NAVSEA officials, the Navy has yet to determine whether it will address schedule outcomes and lessons learned from its pilot program availabilities within future biennial assessments. Our prior work identified leading practices for designing a well-developed and documented pilot program. These leading practices include the following: Establish well-defined, appropriate, clear, and measurable objectives Clearly articulate assessment methodology and data gathering strategy that addresses all components of the pilot program and includes key features of a sound plan Identify criteria or standards for identifying lessons about the pilot to inform decisions about scalability and whether, how, and when to integrate pilot activities into overall efforts Develop a detailed data-analysis plan to track the pilot program’s implementation and performance and evaluate the final results of the project and draw conclusions on whether, how, and when to integrate pilot activities into overall efforts Ensure appropriate two-way stakeholder communication and input at all stages of the pilot project, including design, implementation, data gathering, and assessment These practices enhance the quality, credibility, and usefulness of evaluations and help ensure that time and resources are used effectively. As the Navy moves into implementation of the OPN-funded pilot program, establishing a plan for analysis of the pilot program would provide a means to identify opportunities to take the data on availability schedules, which Congress directed, and compare it to the schedule performance the Navy has attained in its other non-pilot, MAC-MO availabilities. Such evaluations would provide information to the Navy and Congress to determine if the pilot approach should be expanded to help address persistent schedule challenges. In addition, similar to the lessons the Navy has learned in implementing the MAC-MO strategy, the Navy is likely to learn lessons from its OPN-funded pilot availabilities, including ones that relate to schedule drivers currently overshadowed by delays cast by the upward obligations process. Unless the Navy documents within an analysis plan a process for evaluating lessons learned, it runs the risk of missing opportunities to improve its overall performance outcomes across availabilities executed under the MAC-MO strategy. Navy Has Taken Action to Enhance the Predictability of Increasing Maintenance Workloads in Response to Contractor Concerns Representatives of private ship repair contractors that the Navy relies on to execute availabilities under the MAC-MO strategy told us that their workforce and facilities investment decisions are driven by two key considerations. First, the contractors seek visibility on planned workload within a given port, which, under current law, the Navy must publicly report on a quarterly basis. Second, the contractors assess that planned workload to determine what share of the work they are most likely to receive. This assessment affects whether a contractor hires more or fewer people, recapitalizes or expands facilities, and, ultimately, elects to remain part of the Navy’s industrial base for ship repair. In recognition of these considerations, the Navy has taken recent steps to increase predictability of workloads at each port, for example by bundling contracts for both sequential and concurrent availabilities. The Navy anticipates that these steps will help further increase contractors’ confidence in their ability to forecast their share of future workloads. Navy Forecasted Port Workloads Continue to Fluctuate but Are Expected to Exceed Ports’ Capacities in the Near- term As we found in our November 2016 report, various factors regarding the Navy’s level of demand for maintenance and repair work at each of the three home ports implementing MAC-MO, including the deployment of ships, can affect the demand for work in each of the home ports. Based on our analysis of Navy data, this workload remains cyclical in nature, and at times fluctuates above and below what port capacities ordinarily support, as it was under the prior contracting strategy. In May 2016, we found that wide swings in port workload can have a negative effect on the private-sector industrial base, and various factors can affect those workloads. Subsequent to that report, Congress required the Navy to publicly release on a quarterly basis workload projections covering the three ports implementing MAC-MO. Navy’s forecasts indicate that ports implementing MAC-MO will, at times during the next 3 years, be assigned workloads beyond their current capacity, particularly for the Southeast Regional Maintenance Center in Mayport, Florida. Figures 20, 21, and 22 identify the Navy’s port workload projections for each of the three ports as of December 2019. Large Contractors Offered Mixed Views on Workforce and Facilities Investment Planning under MAC-MO Although the Navy projects that overall workload at the ports implementing MAC-MO will fluctuate with periodic increases, lack of certainty about company-specific workload is driving mixed views among contractors on their willingness to make facility and workforce investments. Multiple contractor representatives we interviewed stated they have always worked within an environment of peaks and valleys of workload regardless of the Navy’s contracting strategy. Representatives of large ship repair contractors we interviewed commented on challenges and changes they have made to remain competitive in the MAC-MO strategy’s competitive, firm-fixed-price contracting environment. Under MAC-MO, which requires competition for every availability within a home port, large contractor representatives stated that they do not have a high level of confidence or visibility into future work that the Navy will award to their companies. They have noted that this uncertainty has affected their planning for hiring and facilities investments. Specifically, contractor representatives cited the following: Of the eight large MAC-MO contractors in our review, four reported that they have increased their full-time workforce and the other four have reported a decreased workforce since 2015. Representatives of three contractors selected in our review noted that they have had to rely more heavily on temporary labor to conduct work on Navy availabilities because of inability to predict workloads. For example, a representative of one large contractor noted that their company retains a permanent core workforce, which it then supplements with temporary labor, as needed, depending on the number of contracts it is awarded by the Navy. Representatives of another large contractor noted that the company recently reinstated a training program for new ship repair workers. A representative from the third contractor stated that the company is considering reinstating its equivalent training program based on workload forecasts and confidence in their amount of workload, which underpins investments in workforce training. Representatives of multiple large contractors in our review also stated that they increasingly rely on their subcontractors to execute ship repair work. For example, a representative from one noted that although the company reduced its full time workforce, it is still able to execute availabilities through their use of subcontractor labor. A representative of another large contractor noted that their company staffed a recent availability with about 70 percent subcontracted labor, in part to help the contractor work within the contract’s price as agreed to with the Navy and to help the company make a profit. Representatives of another large contractor stated their company’s preference is to use subcontractors rather than to surge its permanent staff, especially given the contractor’s uncertainty about its portion of future Navy ship repair and maintenance workloads. Representatives of three of the large contractors we interviewed also stated that unstable workloads have limited their plans for significant capital investments in new or expanded facilities. However, representatives of two large contractors reported making new investments in facilities due to high volume of work at their ports. For example, representatives of one large contractor noted that their parent company invested $100 million into building a new dry dock as part of the company’s commitment to win new availabilities and complete them on schedule. The company reported that it was willing to make this investment, in part, because Navy forecasts show an increase in ships being homeported at that location. These contractor representatives further stated their company is considering additional facilities investments. Apart from the considerations that affect their hiring and facilities investments, representatives from all of the large companies we interviewed told us that they plan to continue competing for Navy ship repair work under the MAC-MO strategy. For seven out of the eight of these contractors, the Navy is their primary customer. A representative of one large contractor noted their company’s preference for the MAC-MO strategy, as compared to earlier Navy contracting strategies, especially as a means to increase its ability to propose on and compete for availabilities. Representatives of two additional large contractors also echoed the positive effect of increased opportunities to propose on Navy ship repair and maintenance contracts as a means to potentially grow their workloads. Representatives of Selected Small Business Contractors Report Expanded Hiring and Facility Investments under MAC-MO Representatives of the three small business contractors we interviewed told us that they have each increased their workforces since 2015, when the Navy began implementing the MAC-MO strategy. Under this strategy, small businesses are able to compete for noncomplex ship repair work as prime contractors. Overall, these small business contractor representatives stated they intend to further grow their workforces and facilities, correspondent with the amounts of ship repair work they receive. Specifically, representatives of these small business contractors told us the following: Representatives of one small business prime contractor reported that their company grew its workforce from 625 to 982 between December 2015 and March 2019 as they stated that MAC-MO provided additional opportunities to propose on ship repair contracts. A representative of another small business prime contractor we interviewed estimated that their company hired an additional 100 personnel at two locations because of new, increased workloads related to MAC-MO’s implementation. One small business prime contractor included in our review completed a major facilities expansion, including the addition of a dry dock intended to serve all lines of business, including commercial business customers. A representative of one small business prime contractor stated that their company is considering significant infrastructure upgrades and plans to aggressively compete for noncomplex Navy ship repair and maintenance work. The Navy Is Taking Steps to Increase Predictability of Future Workloads The Navy has recently begun implementing two new contractual approaches—horizontal and vertical contract bundling—within its MAC- MO strategy, but has not yet had sufficient time to collect or assess results. These approaches are intended to increase contractors’ visibility into and confidence regarding future ship repair workloads. Navy leadership officials stated that by awarding multiple availabilities, industry receives a body of work that creates confidence in hiring and retaining a skilled workforce and investment in infrastructure. These approaches provide for contractors to propose on multiple ship repair availabilities that the Navy has bundled within a single request for proposal. Figure 23 illustrates these new contractual approaches. Horizontal Contract Bundling: Navy leadership officials testified to Congress in October 2019 that horizontal bundling helps them decide where to direct ship repair and maintenance work, especially as a means to not surpass capacity at a given port. A representative of one large contractor told us the company anticipates positive effects from horizontal bundling to include being awarded two availabilities from one proposal process and guarantees of work for a longer period than one availability. Another large business contractor representative noted that horizontal bundling would help in stabilizing workloads over a longer period of time, which would also help with its hiring planning. The Navy awarded its first horizontally bundled availabilities in September 2019, and the contractor is expected to complete work on the two ships at its shipyard in Seattle, Washington in June 2021 and May 2022, respectively. NAVSEA leadership officials noted that Navy intends to implement horizontal contract bundling at all of its ports in the future. Vertical Contract Bundling: This contract bundling approach has the potential to allow contractors to increase their workload through only one proposal process, as they may then have the possibility to work on two availabilities at one time. The Navy awarded its first vertically bundled availabilities in February 2019 to three contractors. The second award, in September 2019, resulted in one contractor receiving two simultaneous availabilities. Additionally, NAVSEA leadership officials state they are undertaking other initiatives intended to avoid (1) large fluctuations in ship repair work at individual ports, and (2) the need for contractor workforce layoffs and surge hiring. These initiatives are outlined in further detail below: Attempting to Level Port Workloads: Through its P2P initiative, the Navy intends to use historical timelines from recent availabilities to more accurately plan and forecast future availability time frames. This effort is using computer modeling to avoid either underutilizing or exceeding the available port loading capacity of the industrial base in any given timeframe. On average, NAVSEA leadership stated that they intend to lengthen planned availability timeframes by 56 days to more accurately reflect completion times. The officials assessed that this strategy will help ship repair contractors better manage their workforce planning. They further stated that if contractors have increased visibility in port loading, they will be more likely to hire an increased number of permanent staff in key ship repair trades. According to NAVSEA leadership officials, this could then allow for increased workload capacity at a given port, as those permanent— rather than temporary—staff would become more skilled over time and therefore would require less on-the-job training. Contractor Workforce Capacity Reporting: NAVSEA leadership officials also noted that the Navy is considering options for including language in future ship repair contracts requiring contractors to identify their workforce capacity, including by trade and skill set. NAVSEA leadership officials noted that the intention of such an initiative would be to obtain better workforce capacity data to better plan future port workloads. Conclusions Although the MAC-MO strategy appears to have stabilized the cost and quality components, completing maintenance availabilities within allotted schedules continues to elude the Navy. The Navy has taken steps to more readily accommodate growth work needs as they emerge, however these likely cannot completely eliminate the Navy’s need for upward obligations. The Navy has pointed to the low cost threshold and upward obligations approval process, as provided for in statute, as not providing it with the agility it needs to fund growth work on a schedule that minimizes disruption to an availability. Recently, Congress enacted legislation, signed into law by the President, which establishes an OPN-funded pilot program and provides the Navy a platform to potentially demonstrate that it can meet its MAC-MO schedule goals when freed from the time intensive process of upward obligations. Nonetheless, every pilot program should be thought out before it starts, including consideration of what data need to be collected and how the data will be analyzed. Otherwise, the pilot could be poorly run or could miss opportunities to gain information and lessons learned. Such planning for the OPN-funded pilot could enhance the quality, credibility, and usefulness of the pilot program. Recommendation for Executive Action The Secretary of the Navy should establish an analysis plan for the evaluation of OPN-funded pilot program availabilities, based on the leading practices for pilot programs. This analysis plan should identify opportunities to evaluate schedule outcomes of pilot program availabilities as compared to non-pilot program availabilities and document a process for evaluating lessons learned from the pilot program (Recommendation 1). Agency Comments We provided a draft of this report to the Navy for review and comment. In written comments provided by the Navy (reproduced in appendix II), the Navy concurred with our recommendation. We are sending copies of this report to the appropriate congressional committees; the Secretary of Defense; and the Secretary of the Navy. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-4841 or at oakleys@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Appendix I: Objectives, Scope, and Methodology In 2015, the Navy transitioned to the Multiple Award Contract, Multi Order (MAC-MO) contract strategy for the maintenance and modernization of surface ships. This report (1) examines competition, cost, schedule, and quality outcomes under the strategy; (2) evaluates actions the Navy has taken related to recent lessons learned; and (3) describes considerations informing contractors’ plans for future hiring and facilities investments. Outcomes of the Strategy To examine the competition outcomes of the MAC-MO strategy, as well as number of offers received, we analyzed delivery orders for all of the MAC-MO availabilities in Norfolk, Mayport, and San Diego from the start of the strategy in April 2015 through March 2019. Navy provided a list of MAC-MO Indefinite Delivery, Indefinite Quantity contracts and identified which were complex and noncomplex. We used the Federal Procurement Data System – Next Generation (FPDS-NG) to identify the delivery orders associated with these contracts, and the number of offers received for each order. To assess the reliability of the FPDS-NG data, we reviewed documentation, interviewed Navy officials, performed logic checks, and compared the FPDS-NG data to contract documents. To confirm that we had correctly identified orders related to MAC-MO availabilities, we reviewed the order description in FPDS-NG to confirm that it was a valid ship repair availability and the type of availability. For cases in which the FPDS-NG description did not contain the availability type, we obtained the contract to confirm that it was a valid ship repair availability. To assess the reliability of the number of offers, we performed a logic check to confirm the number of offers received for the delivery order was generally different from the number of offers received for the base contract. Documents reviewed included the FPDS-NG data dictionary, FPDS-NG data validation rules, and Fiscal Year 2013-2018 Federal Procurement Data Quality Summary, which contains results of agency testing of selected fields in FPDS-NG. We determined the FPDS-NG data were reliable for the purpose of assessing the competition outcomes of the MAC-MO strategy. To assess the quality outcomes of MAC-MO availabilities, we reviewed Federal Acquisition Regulations to identify differences between fixed- price and cost reimbursement contract types, and interviewed Navy officials regarding the steps the Navy takes to manage quality in a fixed- price environment. To examine the cost and schedule outcomes of the MAC-MO strategy, we collected ship maintenance availability data from NAVSEA and the Commander, Navy Regional Maintenance Center (CNRMC). This data contained the planned cost and schedule of Chief of Naval Operations (CNO) availabilities, as well as the actual cost and schedule for the availabilities that the Navy closed out between February 2, 2011 and January 15, 2019. While we were directed to assess the MAC-MO outcomes against the Multi-Ship, Multi-Option outcomes, differences in how the availability cost and schedule are estimated between the two strategies prevented us from comparing their cost and schedule outcomes. To assess the reliability of the data, we (1) gathered information from the Navy’s users of the data related to its reliability, (2) compared different snapshots of the data over time to check the consistency of completed entries, including the version that the Navy used to publish its first assessment of the MAC-MO strategy, and (3) compared availability documentation from our completed case study CNO availabilities. We determined the data were reliable for the purpose of assessing cost and schedule outcomes. To narrow our sample, we filtered the data to the ship classes and locations covered under the MAC-MO strategy and eliminated availabilities that had yet to report final cost and schedule entries. This yielded 41 closed out CNO availabilities since the start of the MAC-MO strategy in April 2015. We then adjusted all dollar values for inflation to fiscal year 2020 dollars by using the deflators for Operations and Maintenance funding found in table 5-9 of the Department of Defense budget estimates for fiscal year 2020. To calculate cost and schedule change, we determined the difference between the final cost and completion date, and the planned cost and completion date. The planned cost and schedule represents the Navy’s estimate at the time the Navy awarded the contract. We then calculated the average cost and schedule change for all 41 availabilities, as well as the availabilities at each of the three maintenance centers and classes of ships. To help examine the cost, schedule, and quality outcomes of the MAC- MO strategy, as well as to identify lessons learned, we selected six availabilities as non-generalizable case studies, four of which were completed at the time of our review. To select the availabilities, we used a list of MAC-MO Indefinite Delivery, Indefinite Quantity (IDIQ) contract numbers provided by the Naval Sea Systems Command. We used the Federal Procurement Data System, Next Generation (FPDS-NG) to collect the descriptions of contract actions to determine the ship and availability type, estimated cost, estimated completion dates, contractor, and place of performance. We selected a combination of six availabilities that provided a variety of the following characteristics: We selected two availabilities of each class of ship under the MAC- MO strategy, including destroyers, cruisers, and amphibious ships. We selected two availabilities from each maintenance center executing the strategy: Mid-Atlantic Regional Maintenance Center, Southeast Regional Maintenance Center, and Southwest Regional Maintenance Center. We selected availabilities awarded to a variety of ship repair contractors, including two from BAE Systems, two from General Dynamics NASSCO, one from Marine Hydraulics International, and one from Huntington Ingalls Industries. We selected a variety of availability types to describe different types of ship repair work, including two Selected Restricted Availabilities, a Special Selected Restricted Availability, Depot Modernization Period, Phased Maintenance Availability, and a Continuous Maintenance availability. For each of the case study availabilities, we collected and reviewed Navy availability documentation including the delivery order, correspondence between the maintenance teams and contractors, availability completion reports, weighted progress reports at the time of completion, and briefings containing lessons learned following completion of the availability. We reviewed the documents to: 1) confirm our selection criteria, 2) identify any deficiencies in quality of work and contract changes as a result, 3) identify the presence of growth work items, new work items, or deferred work items, 4) corroborate interview statements, and 5) identify any other issues during the availability and solutions that could be lessons learned for future availabilities. Actions Taken Related to Lessons Learned To evaluate the actions the Navy has taken related to recent MAC-MO strategy lessons learned, we analyzed Navy documentation containing lessons learned that aim to improve the Navy’s implementation of MAC- MO. We identified a total of three lessons learned as key based on our assessment of the Navy’s documentation of the MAC-MO contracting strategy. These three lessons learned were also identified as such in one or more interviews with NAVSEA officials knowledgeable about the challenges associated with MAC-MO implementation and the steps the Navy has taken to fix those issues. To evaluate the Navy’s progress in taking actions to address potential challenges posed by the key lessons learned, we reviewed Navy documents, including Navy assessments of the contracting strategy’s effectiveness, documents implementing revised planning milestones and contracting processes, strategy and planning documents, documents from availability completion meetings, case study contract file documents and other documentation related to lessons learned. To assess the extent to which the Navy has taken actions, we developed the following three-point scale: Not Complete—The Navy has not taken any action to respond to identified lessons learned. Partially Complete—The Navy has taken some action to respond to the identified lessons learned, but has not completed the action needed to address the identified risk. Complete—The Navy has completed the action needed to address the identified lesson learned. Considerations Informing Contractors’ Plans for Hiring and Facilities Investments To describe considerations informing ship repair contractors’ plans for future hiring and facilities investments under the MAC-MO strategy, we conducted semi-structured interviews with and reviewed questionnaire responses from 11 non-nuclear surface ship repair contractors. This included all eight contractors responsible for executing major ship repair work under this strategy at the three home ports implementing it, including Mayport, Florida, Norfolk, Virginia and San Diego, California. We randomly selected a non-generalizable sample of three small business contractors performing noncomplex ship repair work at the three home ports implementing MAC-MO, to obtain the views of small businesses executing MAC-MO contracts. We used FPDS -NG data to identify those small businesses that have been awarded MAC-MO delivery orders. Further, we used a data collection instrument to gather information from each of the selected 11 contractors on their facilities, workforce, and sources of revenue. For example, we collected contractor-reported information on what types of facilities the contractor owned, such as a dry dock or a pier, the number of the contractor’s full-time staff, and the percentage of revenue from entities other than from the Navy. To identify the Navy’s projected workload for non-nuclear surface ships where the MAC-MO strategy is implemented, we obtained data from the Navy from fiscal years 2019 through the end of 2023. Since the purpose of our analysis was to show the Navy’s projections in anticipated port workload, we did not conduct our own assessment of the accuracy of this data. We also interviewed key Navy officials and reviewed statements from testimonies to Senate subcommittees, including of the NAVSEA Commander and of the Assistant Secretary of the Navy for Research, Development, and Acquisition, on their approaches to provide increased visibility and avoid large fluctuations of workloads at Navy ports, including the three home ports implementing the MAC-MO strategy. We collected documentation on these approaches, such as for the Performance to Plan initiative on how the Navy intends to use computer modeling to more accurately plan and forecast future availability timeframes, leveraging Navy historical datasets to provide more accurate and realistic planning forecasts. In addition, for all three objectives, we interviewed officials responsible for overseeing, planning, administering, and funding the Navy’s ship repair contracts, including representatives of the Office of the Under Secretary of Defense (Comptroller); Office of the Assistant Secretary of the Navy (Financial Management and Comptroller); the Office of the Chief of Naval Operations; Commander, Navy Regional Maintenance Center (CNRMC) and Deputy Commander, Surface Ship Maintenance and Modernization (SEA 21); Surface Maintenance Engineering Planning Program (SURFMEPP); Commander, Naval Surface Force, Atlantic; Commander, Naval Surface Force, Pacific; Mid-Atlantic Regional Maintenance Center (MARMC) in Norfolk, Virginia; the Southwest Regional Maintenance Center (SWRMC) in San Diego, California; and the Southeast Regional Maintenance Center (SERMC) in Mayport, Florida. We additionally interviewed management representatives of 11 ship repair contractors included in our review and the third party planning contractor. We conducted this performance audit from November 2018 to May 2020 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Comments from the Department of the Navy Appendix III: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, GAO staff who made key contributions to this report include Christopher R. Durbin (Assistant Director); Sean Seales (Analyst-in-Charge); Pete Anderson; Sonja Bensen, Lorraine Ettaro; Lori Fields; Suellen Foth, Kurt Gurka; Cale Jones; Ethan Kennedy; Sophia Payind; and Carol Petersen.