Background Federal agency IT systems provide essential services that are critical to the health, economy, and defense of the nation. However, federal agencies increasingly rely on aging legacy systems that can be costly to maintain. As we previously reported in May 2016, our review of federal legacy systems found that 26 federal agencies reported spending almost $61 billion on operations and maintenance costs in fiscal year 2015. In addition, many of the government’s IT investments used hardware parts that were unsupported and outdated software languages, such as the common business oriented language (COBOL). In some cases, this lack of vendor support created security vulnerabilities and additional costs because these known vulnerabilities were either technically difficult or prohibitively expensive to address. Congress enacted the MGT Act in December 2017 and established the TMF to help agencies improve, retire, or replace existing systems. Congress appropriates money to the TMF, which is used to fund projects approved by the board. As of August 2019, Congress had appropriated $125 million to the TMF—$100 million was appropriated in fiscal year 2018 and $25 million in fiscal year 2019. Overview of the Technology Modernization Fund The MGT Act assigns specific responsibilities to OMB, GSA, and the Technology Modernization Board for the fund’s administration and also assigns responsibilities to federal agencies that received awarded funds. Among other things, OMB. The act requires the Director of OMB to issue guidance on the administration of the fund and report the status of the awarded projects on a public website. The information reported is to include a description of the project, project status (including any schedule delay and cost overruns), financial expenditure data related to the project, and the extent to which the project is using commercial products and services. GSA. The act designates the Administrator of General Services with responsibility for administering the fund. This includes, among other things: (1) providing direct technical support in the form of personnel services and other services; (2) assisting the Technology Modernization Board with the evaluation, prioritization, and development of agency modernization proposals; and (3) performing regular project oversight and monitoring of approved agency modernization projects. In March 2018, GSA established a TMF Program Management Office within the agency to manage these functions. An executive director leads the office and reports to the Office of the Deputy Administrator within GSA. The act requires the Administrator of General Services, in consultation with the Director of OMB, to establish administrative fees at levels sufficient to ensure the solvency of the fund in order to help offset GSA’s operating expenses for these functions. Agencies pay fees if they receive funding for a project. Technology Modernization Board. The board has responsibility for providing input to the Director of OMB for the development of processes for agencies to submit proposals, making recommendations to the Administrator of GSA to help agencies refine their submitted proposals, and reviewing and prioritizing submitted proposals. The board also is responsible for recommending the funding of modernization projects to the Administrator of GSA, and for monitoring the progress and performance of approved projects. In addition, the board is tasked with monitoring the operating costs of the fund. As part of its oversight of awarded projects, the board requires each project to present a quarterly update and report on the status of milestones achieved in order to ensure the project is on schedule. Other federal agencies. The act stated that any agency that submits an IT-related project proposal and receives TMF funding must repay the transferred amount as well as pay an administrative fee. After the board approves a project proposal, the respective agency is required to sign an interagency agreement with the TMF Program Management Office that specifies the terms of the TMF funding repayment, the administrative fee, and the repayment schedule before initial funds are disbursed and the project begins. Figure 1 provides an overview of key TMF activities that OMB, GSA, and the Technology Modernization Board have undertaken to meet the responsibilities outlined in the MGT Act. These include the establishment of TMF administrative processes and the Technology Modernization Board’s project award announcements, among other activities. These activities are also discussed in greater detail following the figure. In February 2018, OMB issued guidance on the implementation of the MGT Act that included instructions for agencies on submitting applications for TMF funding. Agencies were allowed to begin submitting initial application proposals on February 27, 2018. The guidance included an initial application template that agencies were required to complete. As part of the template, agencies were required to provide an estimate of the TMF funding request and the agency’s method used for cost estimation. Subsequently, in March 2018, OMB issued funding guidelines for projects receiving awards. The guidelines stated that project proposals must include a reliable estimate of any project-related cost savings or avoidance relative to pre-modernization activities using the templates provided. In addition, the guidelines stated that estimates must undergo appropriate due diligence and concurrence from the requesting agency’s Office of the Chief Financial Officer prior to submission to the board, in consultation with OMB’s Resource Management Office and GSA’s TMF Program Management Office. Further, the guidelines stated that the agency’s estimation process would be subject to GAO review, pursuant to the act. For agencies receiving a TMF award, the guidelines stated that agencies were required to repay all transferred funds as well as an administrative fee, which was determined based on the amount of awarded funding. As part of the process, agencies were required to establish a written agreement with GSA that set forth the terms for repaying the transferred funds and the administrative fee. Agencies were required to start making payments one year after the initial amount of award funding was transferred and complete all payments within five years, unless otherwise approved by OMB. While the guidelines noted that reimbursement was not contingent upon the achievement of project-related savings, agencies could use the project’s generated cost savings to repay the award. Agencies Follow a Two-Phase Proposal Process When Applying For a TMF Award The TMF application process occurs in two phases, each of which requires agencies to submit specific documents. During Phase 1, agencies are required to submit an initial project proposal providing preliminary information about the project, its purpose, and its anticipated benefits. Within this documentation, agencies must confirm that funding for this project has never explicitly been denied or restricted by Congress or OMB, in accordance with the MGT Act. Also during this phase, the Technology Modernization Board evaluates proposals and makes recommendations for project funding based on how well the project documentation demonstrates a strong execution strategy, technical approach, and includes a strong team with a demonstrated history of successful modernization efforts. The board encourages agencies to consider the adoption of commercial technology solutions in their proposals and present a strong technical approach and acquisition strategy to implement those solutions. In addition, agencies are encouraged to provide information on the potential impact of the modernization effort on the agency’s mission, feasibility, opportunity enablement (e.g. cost savings), and common solutions. If the board approves the Phase 1 initial project proposal, the project team will move on to Phase 2. In Phase 2, the agency must submit a financial plan showing a cost estimate and estimated savings from the implementation of the proposed project. Agencies must provide a more comprehensive project description than that provided in Phase 1, including discrete milestones, funding schedule, project plan, and financial plan. These documents must be approved by the agencies’ chief financial officer and CIO. Phase 2 proposals must also address any other areas identified by the board in the initial project review. Further, the agency proposal team must also prepare an in-person presentation for the board. OMB’s Resource Management Office reviews the proposal documentation to ensure that the proposed project aligns with the requesting agency’s mission. The office’s review is intended to ensure that the proposal does not duplicate funding provided through existing appropriations, or previously has been expressly denied funding or restricted by Congress. The review includes an assessment of the proposed project’s information on the reimbursement of the awarded funds, the project’s planned schedule, and out-year budget impacts. OMB also reported that the agency sends information on the proposed projects to Congressional appropriation committees for their review prior to the Technology Modernization Board’s approval of a project. Agencies with projects that the board recommends for TMF funding are required to sign an interagency agreement outlining the repayment terms. In addition, projects receive incremental funding contingent on the successful execution of milestones outlined in the written agreement for the transfer of funds. Figure 2 describes the steps in both phases of the TMF proposal process. As of August 2019, the Technology Modernization Board had awarded $89.36 million to seven projects. Table 1 lists the projects that have received funding (in alphabetical order by agency), descriptions of the projects, and when the TMF funding awards were announced. For more details on each of the awarded projects, see appendix II. OMB and GAO Have Issued Federal Cost Estimating Guidance OMB Circular A-11 directs agencies to follow the guidelines outlined in its appendix on cost estimating for all IT investments and acquisitions within the federal government. Since OMB first introduced its cost estimate appendix in 2006, as noted in the circular, the cost estimating appendix has been based on the GAO Cost Estimating and Assessment Guide. The appendix outlines a number of major steps in the cost estimating process and references the practices in GAO’s cost guide. Specifically, these steps include preparing a high-level work breakdown structure, defining ground rules and assumptions, developing the data by collecting information on the cost drivers, developing the estimate using various risk factors, performing a sensitivity analysis, documenting the estimate, and updating it on a regular basis. According to the GAO guidance, a cost estimate is considered reliable if it meets four characteristics and the specific set of best practices associated with each characteristic. Those characteristics are: Comprehensive: An estimate should include all life cycle costs (from the program’s inception and design through operations and maintenance), reflect the current schedule, and have enough detail to ensure that cost elements are not omitted or double counted. Specifically, the cost estimate should be based on a product-oriented work breakdown structure that allows a program to track cost and schedule by defined deliverables, such as hardware or software components. In addition, all cost-influencing ground rules and assumptions should be detailed in the estimate’s documentation. Well-documented: An estimate should be thoroughly documented, describe how it was developed; and include source data, clearly detailed calculations and results, and explanations of why particular estimating methods and references were chosen. Data should be traced to their source documents. Accurate: An estimate should be based on historical data or actual experiences on other comparable programs and an assessment of most likely costs, and be adjusted properly for inflation. In addition, the estimate should be updated regularly to reflect significant changes in the program—such as when schedules or other assumptions change—and actual costs, so that it should always reflect the current status. Credible: An estimate should discuss any limitations of the analysis because of uncertainty surrounding data or assumptions. In addition, the estimate should incorporate the results of a sensitivity analysis (that examine the effects of changing assumptions on the estimate), and risk and uncertainty analysis (that identifies all of the potential project risks and assesses how these might affect the cost estimate). The estimate’s results should be cross-checked, and an independent cost estimate should be conducted to see whether other estimation methods produce similar results. If any of the characteristics is not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high- quality estimate and cannot be considered reliable. Federal Law Generally Requires Agencies to Use Competitive Procedures When Awarding Contracts Federal agencies are generally required to use full and open competition to award contracts for the procurement of goods and services (including commercial IT products), with certain exceptions. The Competition in Contracting Act of 1984 requires agencies to obtain full and open competition through the use of competitive procedures in their procurement activities unless otherwise authorized by law. Using competitive procedures to award contracts means that all prospective contractors that meet certain criteria are permitted to submit proposals. While the Competition in Contracting Act generally requires federal agencies to award contracts using full and open competition, agencies are allowed to award contracts noncompetitively under certain circumstances. Generally, these awards must be supported by written justifications that address the specific exception to full and open competition that is being used in the procurement. An example of an allowable exception to full and open competition includes circumstances when the contractor is the only source and no other supplies or services will satisfy agency requirements. Federal agencies have the option to use a variety of contract types when purchasing IT products and services, including government-wide acquisition contracts, IT Schedule 70 contracts, and blanket purchase agreements. These contracts and agreements allow agencies to establish a group of prequalified contractors to compete for future orders under streamlined ordering procedures once agencies determine their specific needs. Agencies can then issue orders on these contracts and agreements, obligating funds and authorizing work to be performed. Agencies are required to publicly report their contract transactions in the FPDS-NG database. This contract transaction data includes information on the type of award made, the amount of the award, and whether competitive procedures were used. Specifically, agencies are required to identify the extent to which the contract was competed and what solicitation procedures were used. In addition, if an agency awards task orders on an existing contract, then the agency is required to identify whether competitive procedures were used. Further, if the contract did not use competitive procedures, then the agency is required to report the reason that the contract was not competed. About $1.2 Million Has Been Obligated to Cover TMF Operating Expenses and Agencies Expect to Realize Savings in Fiscal Year 2020 or Later As of August 31, 2019, GSA’s TMF Program Management Office had obligated about $1.2 million in operating costs for activities related to the establishment and oversight of the fund. While the office intended to assess administrative fees to fully recover its operating expenses, the actual amounts collected as of August 2019 had been less than planned. This was due to factors such as the office’s formulation of fee rates based on appropriations levels that were higher than what was ultimately received, along with changes to several projects’ scope and milestones. Further, cost savings have yet to be realized. Officials from the seven TMF-funded projects reported that they expect to begin realizing cost savings from their projects starting in fiscal year 2020 or later. TMF Operating Expenses Are to be Offset by Administrative Fee Collection, but Collected Fees Have Been Less Than Planned According to the MGT Act, the TMF Program Management Office may obligate funds to cover its operating expenses out of the appropriations received for the fund (totaling $125 million as of August 2019) in order to provide support to the Technology Modernization Board in meeting its responsibilities. To help offset TMF operating expenses, the act required that the GSA administrator, in consultation with the OMB director, to establish administrative fees at levels sufficient to ensure the solvency of the fund (so that obligations or transfers of funds to awarded projects never exceed the amount available in the fund for these obligations or award transfers). Subsequent OMB guidance, issued in March 2018, required TMF- awarded projects to pay an administrative fee on awarded funds, beginning the first year after the initial incremental amount of award funding was transferred to the agency. The TMF Program Management Office issued further guidance in June 2018 that established administrative fee rates based on a percentage of the amount transferred to an agency project and the payment period. During the time of our review, the office’s current administrative rate was for the period from July 2018 through September 2019. The fee rates were set in June 2018 with the intent to operate the fund as a full cost recovery model, meaning that the Program Management Office planned to fully recover all operating expenses through administrative fee collection by fiscal year 2029 if the office’s assumptions regarding appropriation levels and project selections were met. The office’s reported intention is to help preserve the capital of the fund, which would maximize the amount of appropriations available for award. Table 2 outlines the rates for TMF administrative fees based on the number of years to repay the awarded funds and the percentage of the transferred amount, for the period of July 2018 through September 2019. The TMF Program Management Office sets new rates annually after review from the Technology Modernization Board and approval by GSA’s Deputy Administrator; these rates go into effect in October of each year. As of August 31, 2019, the TMF Program Management Office had obligated about $1.2 million to cover its operating expenses and had begun to collect administrative fees from agency projects, consistent with the MGT Act. Specifically, from March 2018 (when the office began operations) through August 31, 2019, the office obligated approximately $409,000 in fiscal year 2018 and $797,000 for the first 11 months of fiscal year 2019. During the same period, the office collected $33,165 in administrative fees as of August 31, 2019. Based on this amount, the fund was able to only offset approximately 3 percent of its obligated operating costs as of August 31, 2019. The TMF Program Management Office’s administrative fee collection has been limited due to a number of factors that have affected the amounts scheduled to be collected: (1) no fees were collected in the first year of operation; (2) projects chose longer periods to make payments; (3) projects make payments based on funds transferred; (4) fee rates were determined based on assumptions regarding appropriations that were not met; and (5) project changes may affect fee collection. No fees were collected during the first year of operation. OMB’s funding guidelines allowed agencies to start paying administrative fees one year after a project received an award. Since the Technology Modernization Board began awarding funding in June 2018 (within fiscal year 2018), no projects were required to start paying administrative fees until fiscal year 2019, which deferred the start of the TMF Program Management Office’s fee collection by one year. Projects chose longer periods to make payments. When the TMF Program Management Office set administrative fee rates, agencies receiving awards were allowed to determine what rate they would pay according to how many years they planned to make payments. The office reported that a lower administrative fee rate was offered to projects that chose to repay awarded funds over a shorter period (3 years) rather than 5 years. All seven projects that have been awarded funding as of August 31, 2019, chose the longer repayment period of 5 years with a 3 percent rate. The Executive Director of the TMF Program Management Office reported that the office offered a lower administrative rate with the intent of making repaid funds available more quickly to be awarded to new projects. In doing so, the Technology Modernization Board expected to be able to make additional awards, which would increase the collection of administrative fees. Further, according to the Executive Director, the office did not expect that the agencies’ selection of a 5-year repayment term instead of a 3-year term to significantly affect the performance of the fund. However, as the Executive Director noted, these longer repayment terms do affect the collection of administrative fee payments because a longer repayment term means that these funds are not as readily available to award to new projects and generate new fees. Projects make payments based on funds transferred. Agencies receiving awards were only required to make administrative fee payments based on the amount of the award funding that was transferred, rather than based on the full awarded amount. As such, this reduced the amount of fees that the TMF Program Management Office could collect in the initial years that agencies made fee payments. As of August 31, 2019, the Technology Modernization Board had authorized the transfer of $37.65 million (of the $89.36 million awarded) to the seven projects. Based on the amounts transferred, the office is scheduled to collect $1.2 million in administrative fees through 2025 from the seven projects. Table 3 shows the current scheduled administrative fee payments that will be collected from the seven projects based on the amount of awarded funding that the projects had received as of August 31, 2019. Going forward, as the seven projects receive all of the remaining awarded funds, the projects are planning to pay a total of $2.68 million in administrative fees through 2025. However, the Technology Modernization Board had not made awards to any additional projects as of August 2019, and, as a result, the office will not likely be able to collect any additional fees from new projects until at least fiscal year 2021. Any newly awarded projects would be eligible to delay paying administrative fees until 1 year after the initial award date in accordance with the funding guidelines. Fee rates were determined based on assumptions regarding appropriations that were not met. The TMF Program Management Office set its current administrative fee rates in June 2018 based on the assumption that the fund would receive higher levels of appropriations than what was ultimately received. In doing so, the office projected that it would transfer more funds to projects, which would result in larger administrative rates over the initial years of the fund. Specifically, GSA requested $438 million in its fiscal year 2018 and 2019 budget requests for the TMF, but actually received $125 million in appropriations. Table 4 lists the amounts that GSA requested in its budget requests and the amounts appropriated for fiscal years 2018 through 2020. In making its June 2018 assumptions about the appropriations, the office projected that it would distribute larger amounts of funds in the first 2 years of operation and collect more administrative fees through fiscal year 2025. However, the office’s projected collection of administrative fees is less than what was scheduled as of the end of August 2019. In particular, while the office exceeded its projections for distributing funds in fiscal year 2018 ($1.93 million more than projected), the office had not yet met its projection of distributing $75 million in fiscal year 2019—specifically, as of August 31, 2019, the office had distributed only $25.71 million to awarded projects. Consequently, these lower levels of distributed funds decreased the amount of administrative fees scheduled to be collected. Table 5 shows the TMF Program Management Office’s projections for fund distribution for fiscal years 2018 through 2019 and its projected fee collection, compared to the current scheduled distributions and administrative fee collection for fiscal years 2018 through 2025, as of August 31, 2019. Going forward, the office had projected that it would distribute $75 million in fiscal year 2020. However, based on our analysis, only approximately $35.6 million was available in the fund as of August 31, 2019, to award to new projects. The Executive Director of the TMF Program Management Office stated that the office had to make assumptions about the TMF appropriation levels in order to develop the rate model. In doing so, all of the underlying assumptions and parameters related to determining the administrative fee rates and ensuring the fund operated at full cost recovery were reviewed by GSA’s Office of the Chief Financial Officer and Office of General Counsel, OMB, and the Technology Modernization Board before the GSA Deputy Administrator approved the fee rates in June 2018. In addition, the Executive Director noted that, at the time the rate model was developed, the office did not yet have information on the fiscal year 2019 appropriations and made the assumption that the fund would receive the same level of appropriations as in fiscal year 2018 ($100 million). However, based on the wide gap between the budget requests and what funds were ultimately appropriated in fiscal years 2018 and 2019, these assumptions regarding fund appropriation levels did not materialize and impacted the amount of fees that could be collected from awarded projects in fiscal year 2019. Four projects’ changes will affect fee collection. As of August 31, 2019, officials responsible for the management of four of the seven TMF- funded projects reported that they were planning to make significant changes to their projects’ approved scope or scheduled milestones. Officials from two projects reported that they had received approval for these scope changes from the Technology Modernization Board (in June 2019 and August 2019, respectively) and are currently waiting on approval for the repayment schedule changes as of August 31, 2019. Officials from the other two projects reported in August 2019 that they planned to present their changes to the board for approval. Based on our analysis, these changes are expected to affect the four projects’ administrative fee repayment schedules and reduce two projects’ administrative fee collection by $369,117. Table 6 lists the changes to the four TMF-funded projects as of August 31, 2019, as reported by the agencies; the status of the Technology Modernization Board’s approval of the changes; and the potential impacts these changes are expected to have on administrative fee collection. In addition, more details on the changes reported by the four projects are included in appendix II. The Executive Director of the TMF Program Management Office stated that the four projects’ reduction or delay in administrative fee payments should not affect administrative fee collection. The Executive Director explained that the return of prior awarded funds will allow the Technology Modernization Board to have more funds available to award to new projects, which would generate new administrative fees. However, these proposed changes to the four projects’ scope and schedule likely will affect upcoming administrative fee collection because additional time will be needed to review new project proposals. In addition, the agencies may delay administrative fee payments for one year after award issuance. As a result of the five factors that we identified that had impacted administrative fee collection as of August 2019, there is likely to be a period of time between when the office’s current administrative fee collection occurs and when the office can recover its operating expenses from this collection. Specifically, based on our analysis, it will take the TMF Program Management Office at least 5 years (until 2024) to recover the operating costs expended as of August 31, 2019, (over $1.2 million) with the current collection of administrative fees. In addition, once the two projects’ proposed scope and schedule changes are approved by the Technology Modernization Board (decreasing fees collected by $369,117), it is likely that the office will take longer than 5 years to recover these operating costs. Further, it is not clear when the TMF Program Management Office will recover future operating expenses incurred in fiscal year 2020 and beyond. Moreover, these factors will most likely continue to be a challenge for OMB and the office going forward if newly awarded projects choose longer repayment periods or more awarded projects make changes that affect fee collection. Consequently, OMB and the TMF Program Management Office are not currently on track to operate the fund at full cost recovery, as intended. The Executive Director of the TMF Program Management Office stated that the office had reduced its fiscal year 2019 operating expenses by almost 50 percent from the original planned operating level (in the fiscal year 2019 President’s Budget). In particular, the Executive Director reported that the office used temporary staff internally to deliver administrative and support activities, such as website updates and the preparation of meeting agendas and minutes, rather than rely on contractors. The office added that, using internal temporary employees had provided the office with the flexibility to scale operations up and down as appropriate. As of August 2019, the office was not pursuing a staff increase. Further, the Executive Director stated that, as of August 2019, the office was reassessing the assumptions for the administrative fee rate model for the upcoming year, including assumptions for fiscal year 2020 appropriations. The Executive Director added that the office would like to have more information on its fiscal year 2020 appropriations in order to help determine the new rate. These assumptions would be used to develop a new rate model that is expected to go into effect on October 1, 2019, for fiscal year 2020. As for the office’s ability to manage the fund at full cost recovery, the Executive Director stated that all of the assumptions would have needed to be met in order to ensure the TMF operated with full cost recovery. The Executive Director added that the office still intends to pursue full cost recovery going forward, but noted that this may change if the new set of assumptions is not met. Further, the Executive Director reported that four project proposals were in draft stages or pending a Technology Modernization Board determination as of August 2019. Since the fund was established in December 2017, OMB, the Technology Modernization Board, and the TMF Program Management Office have provided oversight of the fund’s awarded projects by requiring the respective agencies to provide quarterly updates on the status of project milestones and transferring additional funds only when milestones were reached. However, the board had not made a corresponding effort to ensure that the TMF Program Management Office’s operating costs and administrative fee collection remained on track to achieve full cost recovery as intended. In addition, the office’s plan to take 12 years—from the start of operations in fiscal year 2018 until fiscal year 2029—to fully recover its operating costs hinders GSA’s ability to maximize the amount of appropriations available for award due to the length of time necessary to recover its costs. As a result, as of August 2019, OMB and the TMF Program Management Office were not on track to recover all operating expenses related to fund administration and oversight, thereby leaving less of the fund’s capital available for project awards. The TMF Program Management Office’s authorized collection of administrative fees is intended to allow the office to offset expenses, which maximizes the amount of funding that can be awarded to projects. However, given the lower-than-expected collection of these administrative fees and the office’s lengthy time frame for recovering all costs, it may be prudent to review those fees and determine whether their rates are set appropriately. Unless OMB and the TMF Program Management Office take steps to develop a plan that outlines the actions needed to fully recover TMF operating expenses with administrative fee collection in a timely manner, there will be fewer funds available to award to projects that are intended to improve the efficiency and effectiveness of government IT systems. TMF Projects Plan to Begin Realizing Cost Savings in Fiscal Year 2020 or Later The MGT Act established the TMF to help improve, retire, or replace federal IT systems with more efficient and effective systems that would cost less money to operate and maintain. As part of its selection criteria, the Technology Modernization Board stated that the agency would need to clearly demonstrate in its proposal how the proposed project would generate cost savings or how the modernization of the system would dramatically improve the quality of service provided. In addition, OMB’s funding guidelines stated that the project proposal must include a reliable estimate of any project-related cost savings or avoidance using the templates provided. Agencies were required to identify what year their project would start to realize cost savings in the TMF application after receiving an award (the earliest year savings could begin to be realized was fiscal year 2019). Further, the guidelines stated that the agency’s estimation process would be subject to GAO review, pursuant to the act. As of August 31, 2019, officials responsible for project management for each of the seven TMF-funded projects reported that their projects had not yet begun to realize cost savings because either the project was still currently being implemented or the project had experienced changes to prior projections. Specifically, officials for four of the seven projects reported that their projects were currently meeting targeted milestones for implementation and would begin to realize cost savings starting in fiscal year 2020 or later as planned. Officials for the other three projects reported that they had recently made changes to the projects’ scope and scheduled milestones that delayed when the projects would begin to realize savings. For more details on the changes reported by these three projects, see appendix II. Table 7 shows the year that each of the seven TMF-funded projects report that they would begin to realize cost savings. One of the three projects that experienced changes, Agriculture’s Infrastructure Optimization project, had originally planned to begin realizing cost savings starting in fiscal year 2020; however, project scope and milestone changes delayed the expected date for realization of this savings. Officials from Agriculture’s Infrastructure Optimization project reported in August 2019 that the new time frame for realizing cost savings remained to be determined. In addition, Energy’s Enterprise Cloud Email project had originally intended to begin realizing cost savings in 2021, but changes to the project’s scope have delayed the realization of savings until 2024. The third project, GSA’s NewPay, had originally planned to begin realizing savings in 2024, but changes to the project’s technological implementation have delayed the realization of savings. In particular, officials from GSA’s NewPay project reported that the project still anticipates realizing cost savings, but the date for these savings remains to be determined. Congress established the MGT Act and the TMF to help agencies transform their legacy IT systems to be more cost effective and efficient. As the awarded projects complete implementation efforts, it will be critical for agencies to realize cost savings from these modernization efforts in order to help ensure the fund is successful. Savings Estimates for the Technology Modernization Fund Projects Are Not Reliable OMB’s Circular A-11 directs agencies to follow the guidelines outlined in its appendix on cost estimating for all IT investments and acquisitions within the federal government. Since 2006, as noted in the circular, the cost estimating appendix has been based on the GAO Cost Estimating and Assessment Guide. As discussed earlier, the appendix outlines a number of major steps in the cost estimating process and references the practices in GAO’s cost guide. According to GAO’s guidance, a reliable estimate should meet the criteria for four characteristics and the specific set of best practices associated with each of the characteristics. The four characteristics and the specific best practices, among others, are: comprehensive – the estimate should include all life cycle costs, a work breakdown structure, and ground rules and assumptions; well-documented – the estimate documentation should describe how the source data were used, the calculations that were performed and their results, and the estimating methodology used; accurate – the estimate should be based on historical data or actual experiences on other comparable programs and be updated regularly to reflect changes in the program; and credible – the estimate should incorporate the results of sensitivity, and risk and uncertainty analyses. According to the GAO guidance, if the overall assessment rating for each of the four characteristics is not fully or substantially met, then the cost estimate cannot be considered reliable. Based on our analysis of the cost estimates for the seven TMF-funded projects, the reported savings estimates that were derived from those estimates cannot be considered reliable. Officials responsible for developing the cost estimates for each of the projects did not incorporate all of the best practices for a reliable cost estimate, as defined in the GAO guidance and OMB Circular A-11. Table 8 describes the four GAO cost estimating characteristics, key practices associated with each characteristic (and the major steps in OMB Circular A-11), and the results of our analysis of the seven TMF- funded projects’ cost estimates. In addition, appendix III provides more details on our individual assessments of the seven projects’ cost estimates. In assessing the reliability of the projects’ cost estimates, we found that the TMF Program Management Office did not provide written guidance for developing the cost estimates in a manner consistent with federal requirements outlined in Circular A-11 or our best practices. Specifically, the only guidance that the Technology Modernization Board provided on the TMF website was the instruction to submit a project cost estimate using a template developed by the Program Management Office, and approved by OMB and the Technology Modernization Board. While the template provided a means to report costs for the proposed projects, the template did not require agencies to follow any of the best practices outlined in GAO’s Cost Estimating and Assessment Guide, and which is referenced by Circular A-11. Further, there were no written instructions for the template regarding the data elements or the fields required to be completed. Agency officials responsible for developing the cost estimate for each of the seven projects all confirmed that they were instructed to use the project cost estimate template to report their projects’ cost and savings estimates. In addition, these officials acknowledged that they did not follow their own internal cost estimate development processes or GAO best practices when developing their estimates. The Executive Director of the TMF Program Management Office stated that the project teams were expected to follow their own internal investment management process for developing the cost estimates. Additionally, the agencies’ chief financial officers and CIOs were required to review and approve the project proposal applications, including the completed cost estimate templates, prior to the agencies’ submissions to the Technology Modernization Board. Further, the Executive Director acknowledged that written guidance had not been developed for completing the project proposal documentation. Instead, the Executive Director stated that the office had held meetings, as requested by each project team, to provide assistance on how to complete the cost estimate template. The Executive Director stated that these meetings enabled the project teams to ask targeted questions on how to complete the template for their individual projects, which enabled the office to provide specific assistance on completing the template for each project. Staff in OMB’s Office of E-Government and Information Technology stated that agencies are required to follow the requirements outlined in Circular A-11 regarding the development of a cost estimate for all IT investments. In addition, the staff noted that each proposal is required to be approved by the agency’s Chief Financial Officer and CIO before being submitted to the Technology Modernization Board. The staff added that the information regarding the guidance for completing the proposal documentation and cost estimates is available on the TMF website. However, our review of the documentation provided on the TMF website did not identify any guidance regarding the development of the cost estimate as part of the proposal—except a statement requiring the completion of the provided template. The website also did not include any guidance instructing the agencies to follow the requirements outlined in Circular A-11, which references GAO’s cost estimating guidance. As noted in GAO’s cost estimating guide, reliable cost estimates can provide management the data necessary to make informed investment decisions, measure program progress, proactively correct course when warranted, and ensure overall accountability for results. Having a realistic estimate of projected costs also helps to ensure that projected cost savings are reliable. Building such quality into a cost estimate is addressed by the steps described in Circular A-11 (that references the practices outlined in GAO’s cost guide). Regardless of whether or not agencies were told to do so, it is an agency’s responsibility to follow these steps. Ensuring agencies understand the requirements they are supposed to follow when developing a cost estimate for their TMF proposal is critical to the success of the proposal process. If OMB and GSA do not clarify the requirement that agencies follow Circular A-11’s cost estimating process (that references GAO’s cost estimating guidance discussed in this report), agencies are at risk of continuing to provide unreliable cost information in their proposals to the Technology Modernization Board. Further, absent detailed guidance from the TMF Program Management Office on how to complete the cost estimate template, including information on the data elements and the fields required to be completed, agencies are at risk of providing incomplete or insufficient information in their project proposals. As a result, the board may not have sufficiently reliable project cost and savings information with which to make decisions on potential awards and whether these projects offer appropriate value for the investment being requested. TMF Project Acquisitions Used Full and Open Competition or an Authorized Exception The MGT Act requires the Administrator of GSA to ensure that the use of commercial off-the-shelf products and services are incorporated to the greatest extent practicable in agency projects awarded funding through the TMF. As required under the Competition in Contracting Act of 1984, all procurements, with certain exceptions, must be competed as full and open so that any qualified entity can submit an offer. Agencies are also required to publicly report their contract transactions in the Federal Procurement Data System-Next Generation (FPDS-NG), including information on the type of award made and whether competitive procedures were used. In addition, if an agency issues task orders on an existing contract, then the agency is required to identify whether competitive procedures were used. Further, if the contract did not use competitive procedures, then the agency is required to report the reason that the contract was not competed. As of August 31, 2019, six of the seven TMF-funded projects had awarded 23 contracts or task orders for work on the projects. Agency officials responsible for management of the six funded projects reported that 22 of the 23 awards used full and open competitive procedures, which we confirmed using acquisition data from FPDS-NG. HUD officials reported that the remaining award was based on a sole source contract that was not competed and an exception was documented. One project had not yet made an award. Table 9 lists the seven TMF-funded projects and the agencies’ reported use of full and open competitive procedures in FPDS-NG for the related awards, as of August 31, 2019. In making the 22 awards, agency officials responsible for the management of the six funded projects reported that they had relied on existing IT service contracts and blanket purchase agreements, or had established new blanket purchase agreements for these projects. Specifically, 11 awards were based on task orders issued on existing contracts. 9 awards were based on orders from existing blanket purchase agreements. 2 awards were made on new blanket purchase agreements. In making these awards using existing contracts and blanket purchase agreements that had followed full and open competitive procedures, the agencies complied with the requirements for using competitive procedures. In those cases where the agencies used existing blanket purchase agreements, these orders were coded as competitive based on data reported in FPDS-NG. For the one award where competitive procedures were not used, HUD completed a justification and approval for other than full and open competition, indicating that only one responsible source and no other supplies or services would satisfy the agency’s requirements. HUD officials stated that they chose a sole source contract because they wanted to retain the expertise of the existing contractors and maintain cohesion between the different phases of project work. For the project that had not yet made an award, officials responsible for the management of Agriculture’s Infrastructure Optimization project reported that, due to a change in the scope of the project made in June 2019, no contracts had been awarded yet for work on the project. The officials reported that they anticipated making an award by the end of December 2019 and that the contract is to be awarded using competitive procedures. Agencies’ continued adherence to federal acquisition requirements for full and open competition should help ensure that their TMF-funded investments deliver the intended services to benefit both the agencies and the public. Conclusions Since March 2018, when GSA established the TMF Program Management Office to administer fund operations, the office has obligated about $1.2 million to cover its expenses from managing the fund but has collected limited administrative fees to offset its expenses. As a result, the Technology Modernization Board has fewer funds than anticipated available to award to new projects. Going forward, OMB and the TMF Program Management Office are likely to face ongoing challenges in collecting administrative fees due to the factors that we have identified that affect fee collection and the office’s lengthy time frame for recovering all costs. While OMB and the TMF Program Management Office are not currently on track to recover all operating expenses in a timely manner, Program Management Office officials have expressed the intent to revisit their fee structure, in part to address the lower than anticipated amount of fiscal year 2019 appropriations. Because of the number of factors that are likely to affect fee collection, it will be critical that OMB and the TMF Program Management Office take steps to develop a plan that outlines the actions needed to fully recover TMF operating expenses with administrative fee collection in a timely manner in order to maximize the funds available for awards. By creating a new funding mechanism to help modernize federal IT systems, Congress intended that funds would be used to improve, retire, or replace existing federal IT systems to improve efficiency and effectiveness of these systems. However, since none of the seven TMF- funded projects’ cost savings estimates can be considered reliable, it is not clear whether the projects receiving funding to date will save the government as much money as was estimated. An important aspect to the success of the TMF will be clarifying the established requirement that agencies follow Circular A-11’s cost estimating process (that references GAO’s cost estimating guidance discussed in this report) in order to help ensure that the reliability of estimated savings for awarded projects is improved. Recommendations for Executive Action We are making five recommendations: two to OMB and three to GSA. Specifically: The Director of OMB should develop and implement a plan with GSA that outlines the actions needed to fully recover the TMF Program Management Office’s operating expenses with administrative fee collection in a timely manner. (Recommendation 1) The Director of OMB should work with GSA to clarify the requirement in the TMF guidance that agencies follow the cost estimating process outlined in Circular A-11 (that references GAO’s cost estimating guidance discussed in this report), when developing the proposal cost estimate. (Recommendation 2) The Administrator of General Services should develop and implement a plan with OMB that outlines the actions needed to fully recover the TMF Program Management Office’s operating expenses with administrative fee collection in a timely manner. (Recommendation 3) The Administrator of General Services should work with OMB to clarify the requirement in the TMF guidance that agencies follow the cost estimating process outlined in Circular A-11 (that references GAO’s cost estimating guidance discussed in this report), when developing the proposal cost estimate. (Recommendation 4) The Administrator of General Services should develop detailed guidance for completing the Technology Modernization Fund project cost estimate template, including information on the data elements and the fields required to be completed, in order to help ensure the accuracy and completeness of the provided information. (Recommendation 5) Agency Comments and Our Evaluation We provided a draft of this report to OMB and the five agencies for their review and comment. In response, of the two agencies to which we made recommendations, GSA stated that it agreed with one recommendation and partially agreed with the remaining two recommendations; and OMB did not state whether it agreed or disagreed with the recommendations. In addition, of the four agencies to which we did not make recommendations, one agency stated that it concurred with information presented in the report, two other agencies stated that they had no comments on the report, and a fourth agency did not state whether it had comments on the report. Further, four agencies provided technical comments on the report, which we incorporated as appropriate. The following discusses the comments received from each agency to which we made recommendations. GSA provided written comments in which it agreed with our recommendation to develop detailed guidance for completing the TMF project cost estimate template. Additionally, the agency partially agreed with our recommendation to develop and implement a plan with OMB that outlines the actions needed to fully recover TMF operating costs with administrative fee collection, stating the agency had concerns with our discussion of this topic in the report. Among the concerns was that we clearly did not acknowledge that GSA is on track to meet the requirement codified in the statute to maintain the solvency of the fund. However, our report did not make a conclusion that the fund was insolvent, or that the fund was on track to being insolvent. Rather, we discussed the factors that have affected administrative fee collection to date. In our discussion, we noted that as a result of these factors, it will take the TMF Program Management Office at least 5 years (until 2024) to recover the operating expenses expended as of August 31, 2019 (over $1.2 million) with the current collection of administrative fees. Consequently, as of August 2019, OMB and the TMF Program Management Office were not on track to recovering all operating expenses in a timely manner, thereby hindering GSA’s ability to maximize the amount of appropriations available for award. As such, we continue to believe our assessment is accurate. GSA also had concerns that we did not state that the TMF Program Management Office’s goal of full cost recovery for operating expenses was over the lifetime of the fund. In our report, we discuss that the TMF Program Management Office planned to fully recover all operating expenses through administrative fee collection by fiscal year 2029. In doing so, we noted that the office’s plan to take 12 years to fully recover its costs hinders GSA’s ability to maximize the amount of appropriations available for award due to the length of time necessary to recover its costs. Therefore, we believe that we have sufficiently discussed the time frame GSA plans to take to fully recover its costs. Further, GSA stated that our discussion of the TMF Program Management Office’s operating costs would be improved if we noted the large percentage of fund administrative costs was devoted to salaries for a limited number of staff. In determining the cost of administering the TMF, we analyzed the costs of establishing and overseeing the TMF and evaluated the collection of administrative fees from projects awarded funding, consistent with the MGT Act. In doing so, we noted the steps taken by the TMF Program Management Office to reduce its operating expenses, including reducing costs by 50 percent for fiscal year 2019, and not pursuing a staff increase in fiscal year 2019. We did not analyze any individual operating expenses and therefore, have no basis to comment on current salary expenses and whether they could or could not be reduced. As such, we believe that we appropriately discuss the costs of establishing and overseeing the TMF and the relationship of those costs to the goal of fully recovering all operating expenses. Accordingly, we believe our recommendation to develop and implement a plan to fully recover office operating expenses with administrative fee collection is still warranted. The agency also partially agreed with our second recommendation to work with OMB to clarify the requirement in TMF guidance that agencies follow the federal cost estimating guidance discussed in this report. GSA stated that the agency does not set cost estimating policy requirements for agencies, as that is the responsibility of OMB and agency CIOs. In our report, we discuss the MGT Act’s requirement that the Administrator of GSA, in consultation with the CIO Council and with the approval of the Director of OMB, administer the TMF. Because the GSA Administrator has been designated responsibility for administering the fund, the agency has a role in clarifying what guidance agencies should follow when developing their cost estimates for the TMF proposal application. Further, we acknowledge GSA’s statement that the agency will commit to working with OMB and the Technology Modernization Board to identify necessary updates to the cost estimating guidance as a positive step towards addressing our recommendation. Consequently, we believe our recommendation for GSA to work with OMB to clarify the requirement in TMF guidance that agencies follow Circular A-11’s cost estimating process (that references GAO’s cost estimating guidance discussed in this report), when developing the proposal cost estimate, is still appropriate. GSA’s comments are reprinted in appendix IV. OMB provided written comments in which the agency did not state whether it agreed or disagreed with our recommendations; however, OMB stated that the agency remains concerned with the facts, characterizations, and opinions in the draft report. The agency further stated that the draft report contains many key assumptions and recommendations that are misleading and paints an incomplete picture of the TMF. OMB then stated that while we met with the agency twice during the course of the audit, we engaged with GSA multiple times in contrast. According to OMB, many of the questions we posed to GSA would have been better answered by OMB, whose authorities in the budget, apportionment, and approval process for TMF proposals could have enabled us to state items in the report with greater accuracy. In addition, the agency stated that many of its corrections and suggestions offered in its review of the statement of facts were rejected by us, although the agency offered no examples to support its comments. We disagree with OMB’s statements regarding our audit methodology for several reasons. First, in meetings with staff from OMB’s Office of E-Government and Information Technology, we obtained information from the staff in all of the areas noted by OMB in its letter. In our report, we discuss OMB’s role in the fund’s administration and the approval process for TMF proposals, as well as OMB’s guidance in these areas. Further, we made ourselves available to engage with OMB throughout the course of the audit. For example, we arranged a meeting with the Federal CIO and her staff to discuss the administration of the TMF and to present our preliminary observations, but the meeting was cancelled by the Federal CIO’s office due to scheduling constraints and not rescheduled. Second, we incorporated many of OMB’s comments on the statement of facts related to OMB’s role in fund administration and the approval process into our draft report. For example, although we had included information in the statement of facts regarding the requirement that agency CIOs and chief financial officers approve TMF proposals prior to submittal to the Technology Modernization Board, OMB requested that we include this information in other sections throughout the report. OMB also requested that we include language in the report to ensure that it was understood that TMF projects began after an interagency agreement was signed between the TMF Program Management Office and the agency and not when TMF awards were announced. We incorporated these changes into the background and other relevant report sections. However, in cases where OMB asked us to incorporate the entirety of language from the MGT Act—rather than summarizing the law’s key requirements—we chose not to do so for the purposes of conciseness. In addition, OMB also requested that we update the status information for the TMF awarded projects in our report to be closer to the report’s issuance. However, as we had told OMB staff during our review, we intended to report project information as of August 31, 2019, based on our audit methodology and reporting timeframes. Consequently, we believe that we have accurately characterized the facts related to OMB’s role in TMF administration and sufficiently incorporated OMB’s relevant comments into our report. OMB also disagreed with our characterization of the TMF repayment process and the assumptions about potential insolvency of the fund. As noted above in our response to GSA’s comments, our report did not make a conclusion that the fund was insolvent, or that the fund was on track to being insolvent. Rather, our report discusses the factors affecting administrative fee collection and the impact these ongoing challenges have on the TMF Program Management Office’s ability to pursue a full cost recovery model and recover all costs by fiscal year 2029, as GSA intended. In addition, we acknowledged the Program Management Office’s efforts to reduce its operating costs in fiscal year 2019 (to under $1 million). OMB also stated that the primary shortcoming has been the fact that the TMF has been underfunded by Congress, leading to slower than anticipated project volume. In our report, among the factors that we discussed as affecting TMF fee collection, we noted that the initial TMF fee rates were determined in June 2018 based on assumptions regarding appropriations that were not met. We also noted the impact that these assumptions had on the TMF Program Management Office’s projected collection of administrative fees in the first two years of operation and for fiscal year 2020. Specifically, we noted that the office projected it would distribute $75 million in fiscal year 2020 but had only approximately $35.6 million available in the fund as of August 31, 2019. We concluded that OMB and the TMF Program Management Office were not on track to recovering all operating expenses in a timely manner, thereby leaving less of the fund’s capital available for project awards. At no point did we assert the fund was insolvent, or was in danger of becoming so. As such, we continue to believe our assessment of the fund’s ongoing fee recovery is accurate and that our recommendation for OMB and GSA to work together to develop and implement a plan to use administrative fee collection to fully recover operating expenses is still warranted. OMB also challenged our analysis of agency projects’ cost estimates using our Cost Estimating and Assessment Guide because, according to the agency, we had asserted that federal agencies must follow the cost guide when developing cost estimates for federal projects. OMB stated that all projects, including those submitted for consideration, must follow OMB Circular A-11, not the GAO guide. Since OMB first introduced its cost estimating appendix to Circular A- 11 in 2006, the circular has stated that the appendix is based on the GAO cost estimating guide. Specifically, the circular stated that the appendix is based on GAO’s “guide to their auditors on how to evaluate an agency's cost estimating process, and the reliability and validity of the data used to develop the cost estimates. Following these guidelines will help agencies to meet most cost estimating requirements.” Further, we reported that OMB’s Circular A-11 cost estimating appendix outlined a number of major steps in the cost estimating process, and referenced the practices outlined in GAO’s cost guide. As our report states, OMB Circular A-11 directs agencies to follow the guidance outlined in the appendix on cost estimating for all IT investments and acquisitions within the federal government, and as mentioned above, is based on GAO’s cost estimating guidance. We noted that OMB’s guidance referenced GAO’s cost guide; however, we did not assert that agencies were required to follow GAO’s cost guide independent of Circular A-11. Further, our analysis of the cost estimates for the seven projects found that none of the projects incorporated all of the best practices for a reliable estimate cost estimate, as defined in either OMB Circular A-11 or GAO guidance. We noted that the TMF’s website did not include any guidance instructing agencies to follow the requirements outlined in Circular A-11; however, we stated that, regardless of whether or not agencies were told to do so, it was an agency’s responsibility to follow these steps. Further, we noted that ensuring agencies understand the requirements they are supposed to follow when developing a cost estimate for the TMF proposal process is critical to the success of the proposal process. Accordingly, we continue to believe our assessment of the seven projects’ cost estimates is accurate and based on appropriate and generally-accepted criteria, and that our recommendations to OMB and GSA in this area are still warranted. However, in the interest of ensuring that our recommendations are explicit about clarifying which requirements agencies are to follow when developing cost estimates, we have modified the language of our related recommendations to more directly address Circular A-11. OMB also noted the additional requirements—beyond those found in Circular A-11—imposed on agency submissions by the Technology Modernization Board, including authoritative signoff by the agency chief information officer and chief financial officer for schedule and repayment documentation. The agency further asserted that the characteristics of the TMF, including the ability to incrementally fund projects and to adjust project scope and timing of project transfers, means that projects funded by the TMF are more likely to succeed. We agree that agencies’ executive review of submissions to the board is an integral part of ensuring the quality of those submissions. Such reviews, coupled with more clear direction to agencies on what federal guidance they are required to follow, as discussed above, will further strengthen the quality of the supporting documentation submitted to the board. Further, OMB also stated that the board takes seriously its responsibilities to make sure approved projects meet the requirements of the MGT Act, the guiding principles established by the board, and to ensure that projects repay all required amounts while successfully delivering smarter, more secure commercial capabilities to improve citizen services. In addition, OMB stated that the board requires that all approved projects have requirements to provide information, best practices, playbooks, and other supporting documentation. OMB also stated that the board has managed the TMF both in alignment with industry-wide best practices for iterative, agile financing for technology projects, and has been judicious and discerning in how it invests TMF funds. We agree with the importance of ensuring approved projects meet the requirements of the MGT Act. In our report, we acknowledged OMB, the Technology Modernization Board, and the TMF Program Management Office’s efforts to provide oversight of the fund’s awarded projects. However, our report also identified ongoing challenges with the TMF Program Management Office’s fee collection, including the office’s plan to take 12 years to fully recover its operating costs—a plan that was reviewed by the Technology Modernization Board and OMB—that will hinder GSA’s ability to maximize the funds available for awards. We also agree that it is important that all approved projects have requirements in place related to providing information and supporting documentation. In our report, we discussed that OMB’s funding guidelines required projects to include a reliable estimate of project- related savings. However, as we also noted, none of the seven projects’ reported savings estimates were reliable because they did not incorporate all of the best practices for a reliable cost estimate as defined in OMB Circular A-11 and GAO’s cost estimating guide. Therefore, it was not certain whether the projects that we reviewed would save the government as much money as was estimated. While it is important that the board have requirements in place, it is equally vital that agencies clearly understand the requirements they are supposed to follow—and that these requirements are clearly articulated on the TMF website—for the proposal process to be successful. As such, we continue to believe our recommendations to OMB and GSA are appropriate. OMB’s comments are reprinted in appendix V. In addition to the aforementioned comments, the four agencies to which we did not make recommendations provided the following responses. In an email received on November 22, 2019, a Director of Strategic Planning, Policy, Egovernment and Audits in the Office of the CIO at Agriculture stated that the agency concurred with the information presented in the report. In an email received on November 7, 2019, an audit coordinator in Energy’s Office of the CIO did not state whether the agency had comments on the report and provided technical comments, which we incorporated as appropriate. In written comments provided on November 19, 2019, the department stated that it had no comments to provide on the written report. HUD’s comments are reprinted in appendix VI. In an email received on November 6, 2019, an economist in Labor’s Office of the Assistant Secretary for Policy stated that the agency had no comments on the report. We are sending copies of this report to the appropriate congressional committees; the Director of the Office of Management and Budget; the Secretaries of the Departments of Agriculture, Energy, HUD, and Labor; the Administrator of GSA; and other interested parties. This report will also be available at no charge on our website at http://www.gao.gov. If you or your staffs have any questions on matters discussed in this report, please contact me at (202) 512-4456 or harriscc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VII. Appendix I: Objectives, Scope, and Methodology Our objectives were to: (1) determine the costs of establishing and overseeing the Technology Modernization Fund (TMF), as compared to the savings realized by projects that have received awards; (2) assess the extent to which cost savings estimates for awarded projects are reliable; and (3) determine the extent to which agencies have used full and open competition for any acquisitions related to the awarded projects. The scope of our review included the Office of Management and Budget (OMB) and the General Services Administration (GSA) TMF Program Management Office, the two organizations responsible for TMF administration, as well as the five agencies that had received the seven awards from the fund as of August 2019—the Department of Agriculture (Agriculture), Department of Energy (Energy), Department of Housing and Urban Development (HUD), Department of Labor (Labor), and GSA. For our first objective, we obtained and analyzed financial data from GSA related to actual and planned operating costs for establishing and overseeing the TMF for fiscal years 2018 through 2025 (fiscal year 2018 was the first year that the TMF was in operation). To ensure the accuracy and completeness of GSA’s financial data on the operating costs for TMF administration, we obtained information from officials within GSA’s Office of the Deputy Administrator on the controls in place for ensuring the reliability of the financial data. We also reviewed GAO, GSA Office of Inspector General, and GSA reports that discussed the results of prior reviews of internal controls for GSA financial systems. Based on discussions with agency officials and our reviews of these prior reports, we did not identify any specific findings that would affect our reporting of these data. In addition, we reviewed GSA-provided data for obvious errors and inconsistencies and identified no significant errors related to the accuracy or completeness of the data. Based on these steps, we determined that these data were sufficiently reliable for us to be able to report accurately on GSA’s operating costs for TMF administration. We also obtained and analyzed agency documentation from, and interviewed officials within, GSA’s TMF Program Management Office regarding the fund’s actual and planned operating expenses as of August 31, 2019. We assessed the collection of administrative fees used to ensure the solvency of the fund during the period from June 2018 (when projects first began to receive awards) through August 31, 2019. In addition, we interviewed staff in OMB’s Office of E-Government and Information Technology regarding OMB guidance and its administrative responsibilities for the fund. Further, we obtained and analyzed TMF project proposal documentation and signed interagency agreements and interviewed officials in charge of the TMF-funded projects within the Office of the CIO and other appropriate offices at each of the five agencies to determine the scheduled repayment transfers, administrative fee payments, and whether awarded projects had realized cost savings for fiscal year 2019. (Fiscal year 2019 was the first fiscal year that awarded projects could have realized cost savings as a result of receiving TMF funding.) In doing so, we confirmed that none of the seven projects had begun to realize cost savings; therefore, it was premature to compare the projects’ realized savings to TMF administrative costs. For the second objective, we analyzed TMF project proposals, including cost estimates and supporting documentation, from the five agencies that received the seven awards. In addition, we interviewed the agencies’ project officials responsible for developing the overall TMF cost savings estimate and associated cost estimates regarding their estimation processes. We compared each TMF-funded project team’s estimating methodologies and documentation to the best practices of a reliable cost estimate discussed in GAO’s Cost Estimating and Assessment Guide. Our analysis enabled us to determine whether each project’s cost estimate, used to determine the project’s cost savings estimate, was comprehensive, well-documented, accurate, and credible. The GAO Cost Estimating and Assessment Guide considers an estimate to be comprehensive if its level of detail ensures that all pertinent costs are included and no costs are double-counted or omitted; well- documented if the estimate can be easily repeated or updated and can be traced to original sources through auditing; accurate if it is not overly conservative, is based on an assessment of the most likely costs, and is adjusted properly for inflation; and credible if the estimate has been cross-checked with an independent cost estimate and a level of uncertainty associated with the estimate has been identified and quantified. For each characteristic, our analysis had five possible assessment categories: Not met. The estimate provided no evidence that satisfies any of the characteristic’s set of best practices. Minimally met. The estimate provided evidence that satisfies a small portion of the characteristic’s set of best practices. Partially met. The estimate provided evidence that satisfies about half of the characteristic’s set of best practices. Substantially met. The estimate provided evidence that satisfies a large portion of the characteristic’s set of best practices. Met. The estimate provided complete evidence that satisfies the characteristic’s entire set of best practices. A cost estimate is considered reliable if the overall assessment for each of the four characteristics are met or substantially met. We presented the results of our initial analysis of each TMF project cost estimate to its respective agency in July 2019. We asked the agencies to verify the information presented in the analysis and provide any updates or additional supporting documentation, as appropriate. Each of the agencies provided updated information, which we incorporated into this analysis, as appropriate. In addition, we interviewed staff in the Office of E-Government and Information Technology, as well as officials from the TMF Program Management Office, about the process for the review and approval of TMF-funded project cost savings estimates and cost estimate documentation. Because the Technology Modernization Board required agency project teams to use a template to submit the project cost savings estimates and because we learned from project officials at each of the five agencies that they did not rely on data from agency financial systems when completing the template, we took additional steps to assess the reliability of the data in the completed templates. First, we interviewed officials in the TMF Program Management Office responsible for developing the template in order to understand the purpose of each template data field and what information was required to be completed. We took this step because there were no written instructions for the template regarding the data elements or the fields required to be completed. We also interviewed officials in the Office of the CIO and other appropriate offices at each agency, who were in charge of completing the TMF cost estimate template. We discussed with these officials how the template was filled out and what sources of data were used. Because project teams did not rely on data from agency financial systems when completing the spreadsheet template, we reviewed agency responses and other supporting documentation to determine how the estimated costs and savings were derived and whether there were any qualifications of the provided data. This included whether certain costs were excluded from the program cost estimate, how up-to-date the data were, or whether there were other qualifications of the provided data. We followed up with agency officials regarding these qualifications as appropriate. Further, we reviewed the completed templates to identify missing data, or other errors, and consulted with our cost estimation specialists about these issues, as appropriate. Based on our assessment of each project’s cost estimate (used to derive the cost savings estimate) and the other measures we took to assess the reliability of the data included in the completed templates, we determined that the cost savings data for all seven TMF projects were not sufficiently reliable; thus, we did not include the estimated savings amounts in our report. In addition, we discuss the data’s shortcomings in the report. To accomplish the third objective, we obtained and analyzed contract documentation for each of the seven awarded projects. We also interviewed officials in charge of the TMF-funded projects within the Office of the CIO and other appropriate offices at each of the five agencies about acquisitions related to the awarded projects. Using the agency provided contract information, we obtained and analyzed data from the Federal Procurement Data System-Next Generation (FPDS- NG)—the government’s procurement database—for the period of June through August 2019. We assessed whether each awarded acquisition used full and open competition in accordance with the Competition in Contracting Act of 1984 and the federal acquisition regulation. To ensure the accuracy and completeness of the awarded projects’ contract information related to the use of full and open competition, we searched FPDS-NG data to confirm that all contracts and task orders related to the projects had been provided. We then presented the results of our analysis to officials in charge of project acquisitions at each agency and asked these officials to verify the completeness and accuracy of the FPDS-NG data and provide any updates, as appropriate. Officials in charge of all of the awarded projects confirmed the contract information related to the use of full and open competition and provided additional contract acquisition data, as appropriate. Based on these steps, we determined that these data were sufficiently reliable to report on the TMF-funded project acquisitions’ use of full and open competition. We conducted this performance audit from March 2019 to December 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Description of Projects Receiving Technology Modernization Fund Awards, as of August 2019 As of August 31, 2019, seven projects had been awarded funding from the Technology Modernization Fund (TMF). Once an award had been made, TMF funds were distributed to project teams incrementally based on each project’s performance against the milestones established in the project’s written agreement. These seven projects had received incremental funding of approximately $37.65 million and, of that amount, had obligated $18.05 million towards project implementation. The following description of each of the seven projects includes an overview of the awarded project, funding transfer, and project status information as of August 31, 2019, and how the project intends to repay the funds awarded. Department of Agriculture’s Farmers.Gov Portal Project The Department of Agriculture’s (Agriculture) Farmers.Gov Portal project is intended to help update and modernize conservation financial assistance and payment operations within the department’s Farm Service Agency and National Resources Conservation Service. These two agencies provide financial and technical assistance to farmers and ranchers through related conservation programs. While separately authorized and appropriated, the programs share common customers and also share interconnected systems. The project is intended to work to reengineer related financial assistance business processes at these agencies and update the agencies’ legacy systems so that the systems can be properly connected with the department’s common financial system. Due to changes to the project’s schedule, an official responsible for the management of the Farmers.Gov Portal project reported that the agency plans to delay requesting the remaining balance of $6 million in awarded funds from the Technology Modernization Board until fiscal year 2020. Figure 3 provides a summary of the Farmers.Gov Portal project. Officials from the Farmers.Gov Portal project reported that the department intends to repay the TMF funds awarded using annual appropriations from each of the two agencies involved in the project. Agriculture’s Infrastructure Optimization Project Agriculture’s Infrastructure Optimization project, managed by the Office of the Chief Information Officer (CIO), was originally intended to migrate 10 applications within the department to cloud services by the end of fiscal year 2019. However, officials responsible for the management of the project reported that they began working with the TMF Program Management Office to make changes to the project’s scope in June 2019, changing which applications would be migrated and reducing the number of applications to be migrated to one. Officials reported that the project now intends to migrate the Farm Production and Conservation’s Emergency Watershed Protection Program to cloud services but has not yet determined when the project will be completed. The program helps landowners, operators, and individuals to implement emergency measures after a natural disaster in order to help relieve imminent hazards to their life or property. Due to the change in scope for the project, officials responsible for the management of the Infrastructure Optimization project reported that planned to request a total of $500,000 for the project from the Technology Modernization Board ($4.5 million less than the original award amount). As a result of this change in scope, officials reported that the repayment period, administrative fee, and the time frames for repaying the transferred amount and associated fee, was being reevaluated by the agency. Project officials reported in August 2019 that they planned to present their revised project plan to the Technology Modernization Board for consideration and approval. If approved by the board, the project would likely reduce its administrative fee from $150,000 to $15,000. Figure 4 provides a summary of the Infrastructure Optimization project. Officials from the Infrastructure Optimization project reported that the department originally intended to repay the TMF awarded funds by using the planned cost savings and avoidances accrued from not having to pay the costs for the maintenance of these 10 applications. In fiscal year 2018, the department reported spending approximately $4 million to cover labor costs for maintaining these 10 on-premise applications. However, project officials reported that, with the change in scope to the project, the details for how they will repay the awarded funding are currently under reevaluation. Department of Energy’s Enterprise Cloud Email Project The Department of Energy’s (Energy) Enterprise Cloud Email project, managed by the Office of the CIO, was originally intended to complete the consolidation, upgrade, and migration of 26 of the department’s on- premises email systems to cloud email software as a service by fiscal year 2021. However, the department made changes to the project’s scope in February 2019, reducing the number of mailboxes that would be migrated from approximately 47,080 to 24,531. Officials responsible for the management of the Enterprise Cloud Email project within Energy’s Office of the CIO reported that the department was able to migrate 22,549 mailboxes to cloud services using department funds prior to receiving TMF-awarded funds. Due to the change in scope for the project, officials from the Enterprise Cloud Email project reported that they planned to request a total of $7.41 million in funding for the project from the Technology Modernization Board ($7.80 million less than the original award amount). As a result of this change in scope, officials reported that the repayment period, administrative fee, and the time frames for repaying the transferred amount and associated fee, will change from what was originally approved by the Technology Modernization Board. Project officials reported in August 2019 that they intended to present their revised plan to the Technology Modernization Board for consideration and approval. If approved by the board, the project would reduce its administrative fee from $456,510 to $222,406 and would complete the fund repayment in 2024 rather than 2025. Figure 5 provides a summary of the Enterprise Cloud Email project. Officials from the Enterprise Cloud Email project reported that the department intends to repay the TMF funds awarded by using the planned cost savings and avoidances accrued from future operations and maintenance costs for these email systems. In fiscal year 2018, the department reported spending approximately $4.78 million to cover operations and maintenance costs for the 26 on-premise email systems originally in scope for the project. However, the department could not provide an update on the operations and maintenance costs for the current email systems that are to be migrated using TMF funds. Department of Housing and Urban Development’s Unisys Migration Project The Department of Housing and Urban Development’s (HUD) Unisys Migration project managed by the Office of the Chief Technology Officer was originally intended to migrate five of the department’s most critical business systems from an on-premise mainframe database to cloud computing services by the end of fiscal year 2020. These systems help manage the Federal Housing Administration’s mortgage insurance program as well as over one hundred HUD grant, subsidy, and loan programs managed through the Office of the Chief Financial Officer. Due to delays in awarding contracts for the project, a HUD official reported that the department had submitted a request to the Technology Modernization Board in August 2019 for the project to be rebaselined. The official reported that the project planned to delay requesting the next disbursement of $5 million from fiscal year 2019 to fiscal year 2020 and the project is now intended to be completed by March 2021. Figure 6 provides a summary of the Unisys Migration project. Officials from the Unisys Migration project reported that the department intends to repay the TMF funds awarded by using the planned cost savings accrued from reducing the department’s overall operations and maintenance costs for these systems. In fiscal year 2018, the department reported spending approximately $11.6 million in operations and maintenance contract costs for maintaining these five legacy systems. Department of Labor’s Visa Application Transformation Project The Department of Labor’s (Labor) Visa Application Transformation project, managed by the Office of the CIO, is intended to replace a paper- based labor certification process for certain types of work visas with an E-Certification process. The new system is intended to enable the department to issue a labor certification securely and electronically to employer applicants, similar to an electronic boarding pass issued by airlines. In addition, this project is expected to streamline and improve data accessibility and reporting capabilities by creating a data hub at Labor. This hub is expected to allow the department to securely transmit these labor certifications and other necessary documentation to the Department of Homeland Security’s U.S. Citizenship and Immigration Service, with an eventual linkage to the Department of Agriculture and the Department of State. Figure 7 provides a summary of the Visa Application Transformation project. Officials responsible for the management of the Visa Application Transformation project within the Office of the CIO reported that the department intends to repay the TMF funds awarded by using the planned cost savings accrued from eliminating the costs of procuring security paper and printers for printing the certifications as well as reduced costs for contractor and federal employee support of the paper process. In fiscal year 2019, the department reported spending approximately $1.9 million on these costs for the paper-based process. General Services Administration’s Application Modernization Project The General Services Administration’s (GSA) Application Modernization project, managed within the Office of the Chief Technology Officer, is intended to modernize 11 applications currently using proprietary vendor technology by converting them to use open source technologies. GSA currently has 88 applications that are in need of modernization and intends to use the lessons learned and new capabilities as a repeatable process that will be used for future migrations of other proprietary applications to open source technologies. Figure 8 provides a summary of the Application Modernization project. Officials responsible for managing the Application Modernization project reported that it intends to repay the TMF funds awarded through: (1) its existing working capital fund and (2) the planned cost savings and avoidances accrued from reducing operations and maintenance costs, and eliminating hardware and operating system software costs for these proprietary applications. In fiscal year 2018, the agency reported spending approximately $23.9 million to cover these costs. GSA’s NewPay Project The NewPay project, managed within GSA’s Office of the CIO, is intended to modernize GSA’s payroll system for its 21,000 users and replace it with a cloud-based software as a service solution. This is expected to lay the foundation for modernizing federal legacy payroll systems to a cloud-based solution for approximately 2.1 million federal civilian employees. Currently, four federal agencies (Agriculture, Department of Defense, Department of the Interior, and GSA) serve as payroll providers for federal civilian employees. NewPay also is intended to encompass time and attendance solutions which are intended to be implemented in later project phases. Project officials reported that they originally planned to complete the migration to NewPay and shut down GSA’s legacy systems by 2023 and consolidate all other government legacy provider payroll operations into NewPay. However, officials reported that the strategy for transitioning other legacy payroll providers to NewPay was revised in mid-summer 2019. Going forward, GSA and the other federal payroll providers plan to focus on completing the migration of all systems to NewPay prior to transitioning and consolidating payroll operations within GSA. Project officials reported that GSA is working with OMB and the other agency payroll providers to identify funding available for these efforts so that a new schedule can be developed. Figure 9 provides a summary of the NewPay project. Officials responsible for managing the NewPay project within the Office of the CIO reported that the agency intends to repay the TMF funds awarded through subscriptions and fees that federal agencies are to pay to utilize the software as a service solution and through fees NewPay intends to collect for serving as a payroll operations provider. In fiscal year 2018, the four federal agency payroll providers spent approximately $300 million providing payroll services for approximately 2.1 million federal civilian employees. Appendix III: Analysis of Cost Estimates for Projects Receiving Technology Modernization Fund Awards Agencies submitting full project proposals to the Technology Modernization Board during phase II of the proposal process for the Technology Modernization Fund (TMF) were required to submit information on the project’s cost estimate and cost savings estimate using a spreadsheet template (known as appendix B). We compared each TMF-funded project team’s estimating methodologies and documentation to the best practices of a reliable cost estimate discussed in the GAO Cost Estimating and Assessment Guide. According to GAO’s guidance, a reliable estimate should meet four characteristics and the specific set of best practices associated with each of the characteristics. Those four characteristics are: Comprehensive: An estimate should include all life cycle costs (from the program’s inception and design through operations and maintenance), reflect the current schedule, and have enough detail to ensure that cost elements are not omitted or double counted. Specifically, the cost estimate should be based on a product-oriented work breakdown structure that allows a program to track cost and schedule by defined deliverables, such as hardware or software components. In addition, all cost-influencing ground rules and assumptions should be detailed in the estimate’s documentation. Well-documented: An estimate should be thoroughly documented; describe how it was developed; and include source data, clearly detailed calculations and results, and explanations of why particular estimating methods and references were chosen. Data should be traced to their source documents. Accurate: An estimate should be based on historical data or actual experiences on other comparable programs and an assessment of most likely costs, and be adjusted properly for inflation. In addition, the estimate should be updated regularly to reflect significant changes in the program—such as when schedules or other assumptions change—and actual costs, so that it should always reflect the current status. Credible: An estimate should discuss any limitations of the analysis because of uncertainty surrounding data or assumptions. In addition, the estimate should incorporate the results of a sensitivity analysis (that examine the effects of changing assumptions on the estimate), and risk and uncertainty analysis (that identifies all of the potential project risks and assesses how these might affect the cost estimate). The estimate’s results should be cross-checked, and an independent cost estimate should be conducted to see whether other estimation methods produce similar results. In assessing each project’s estimate against the components of the four characteristics, we assigned one of five assessment categories: Not met. The estimate provided no evidence that satisfies any of the characteristic’s set of best practices. Minimally met. The estimate provided evidence that satisfies a small portion of the characteristic’s set of best practices. Partially met. The estimate provided evidence that satisfies about half of the characteristic’s set of best practices. Substantially met. The estimate provided evidence that satisfies a large portion of the characteristic’s set of best practices. Met. The estimate provided complete evidence that satisfies the characteristic’s entire set of best practices. A cost estimate is considered reliable if the overall assessment ratings for each of the four characteristics are met or substantially met. The following discusses in detail our assessment of the seven TMF awarded projects’ cost estimates. Department of Agriculture’s Farmers.Gov Portal Project Table 10 includes our detailed assessment of the Department of Agriculture’s (Agriculture) Farmers.Gov Portal project. Based on the overall assessment ratings for each of the four characteristics, Agriculture’s project cost estimate is not considered reliable. Agriculture’s Infrastructure Optimization Project Table 11 below includes our detailed assessment of Agriculture’s Infrastructure Optimization project. Based on the overall assessment ratings for each of the four characteristics, Agriculture’s project cost estimate is not considered reliable. Department of Energy’s Enterprise Cloud Email Project Table 12 includes our detailed assessment of the Department of Energy’s (Energy) Enterprise Cloud Email project. Based on the overall assessment ratings for each of the four characteristics, Energy’s project cost estimate is not considered reliable. Department of Housing and Urban Development’s Unisys Migration Project Table 13 includes our detailed assessment of the Department of Housing and Urban Development’s (HUD) Unisys Migration project. Based on the overall assessment ratings for each of the four characteristics, HUD’s project cost estimate is not considered reliable. Department of Labor’s Visa Application Transformation Project Table 14 includes our detailed assessment of the Department of Labor’s (Labor) Visa Application Transformation project. Based on the overall assessment ratings for each of the four characteristics, Labor’s project cost estimate is not considered reliable. General Services Administration’s Application Modernization Project Table 15 includes our detailed assessment of the General Services Administration’s (GSA) Application Modernization project. Based on the overall assessment ratings for each of the four characteristics, GSA’s project cost estimate is not considered reliable. GSA’s NewPay Project Table 16 includes our detailed assessment of GSA’s NewPay project. Based on the overall assessment ratings for each of the four characteristics, GSA’s project cost estimate is not considered reliable. Appendix IV: Comments from the General Services Administration Appendix V: Comments from the Office of Management and Budget Appendix VI: Comments from the Department of Housing and Urban Development Appendix VII: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the individual named above, the following staff made key contributions to this report: Dave Hinchman (Assistant Director), Jason Lee (Assistant Director), Jessica Waselkow (Assistant Director), Chris Businsky, Jennifer Echard, Emile Ettedgui, Valerie Hopkins (Analyst in Charge), Anna Irvine, Julia Kennon, Sandra Kerr, James MacAulay, Priscilla Smith, and Mary Weiland.