Background Signed into law on May 9, 2014, the DATA Act expands on previous federal transparency legislation. It requires federal agency expenditures to be disclosed and agency spending information to be linked to federal program activities so that policymakers and the public can more effectively track federal spending. The DATA Act also requires government-wide reporting on a greater variety of data related to federal spending, such as budget and financial information, as well as tracking these data at multiple points in the federal spending life cycle. To accomplish these goals, among others, the act gives OMB and Treasury responsibility for establishing government-wide financial data standards for any federal funds made available to or expended by federal agencies. These standards specify the data to be reported under the DATA Act and define and describe what is to be included in each data element with the aim of ensuring that information reported will be consistent and comparable. As Treasury and OMB implemented the DATA Act’s requirement to create and apply data standards, the overall data standardization effort has been divided into two distinct, but related, components: (1) establishing definitions which describe what is included in each data element with the aim of ensuring that information will be consistent and comparable, and (2) creating a data exchange standard with technical specifications which describe the format, structure, tagging, and transmission of each data element. In the implementation of the DATA Act, OMB took principal responsibility for the definitions, while Treasury took principal responsibility for the technical standards that express these definitions, which federal agencies use to report spending data for publication on USAspending.gov. The act also holds agencies accountable for submitting complete and accurate data to USAspending.gov and requires that agency-reported award and financial information comply with the data standards established by OMB and Treasury. The Importance of Data Governance for Ensuring Data Quality One of the purposes of the DATA Act is to establish government-wide data standards to provide consistent, reliable, and searchable spending data that are displayed accurately for taxpayers and policymakers on USAspending.gov (or a successor website). As we have reported previously, establishing a data governance structure—an institutionalized set of policies and procedures for providing data governance throughout the life cycle of developing and implementing data standards—is critical for ensuring that the integrity of the standards is maintained over time. The need for a data governance structure is underscored by our previous analyses of the quality of the federal spending data available on USAspending.gov and inconsistencies we identified in how agencies report data according to data standards. A data governance structure could be useful for adjudicating revisions, monitoring, and ensuring compliance with the standards over time. As we have noted, such a structure, if properly implemented, would greatly increase the likelihood that the data made available to the public will be accurate. A data governance structure can also provide consistent data management during times of change and transition. We have previously reported that gaps in leadership can occur as administrations change. This can impair the effectiveness and efficiency of complex government- wide efforts, potentially resulting in delays and missed deadlines. Accordingly, in 2015, we recommended that OMB, in collaboration with Treasury, establish a set of clear policies and processes for developing and maintaining data standards that are consistent with leading practices. OMB and Treasury did not comment on our recommendation. We plan to conduct work intended to help inform OMB’s and Treasury’s efforts. This work may include the development of a maturity model that could provide a framework for assessing data governance activities related to federal spending data. This work may also have broader government-wide implications as agencies begin implementing the requirements of the Foundations for Evidence-Based Policymaking Act enacted on January 14, 2019, including designating Chief Data Officers with data governance and implementation responsibilities. Although Some Governance Procedures Are in Place, a Formal Structure for Governing Data Standards Continues to Evolve Roles of Data Governance Interagency Advisory Groups Have Shifted During DATA Act Implementation In December 2018, OMB staff told us that they are transitioning from the governance structure used for initial DATA Act implementation to a new structure for managing data standards within the broader context of efforts to establish a federal data strategy. According to OMB staff, the initial data governance structure reflected OMB’s and Treasury’s focus on creating the design and build functions required to meet the statutory requirements of the DATA Act. The President’s Management Agenda (PMA), released in March 2018, outlines a long-term vision for modernizing federal operations. To address the issues outlined in the PMA, the administration established a number of cross-agency priority (CAP) goals. These goals, required by the GPRA Modernization Act of 2010, are to address issues in a limited number of policy areas requiring action across multiple agencies, or management improvements that are needed across the government. According to OMB staff, several of the 2018 goals relate to data standardization, and a new governance structure is needed to achieve those goals. OMB staff informed us in July 2018 that the governance structure used for initial implementation efforts, which included the DATA Act Interagency Advisory Committee and Data Standards Committee, had been disbanded, and that the advisory roles of these groups were assumed by the Chief Financial Officers Council’s DATA Act Working Group (CFOC Working Group). According to OMB staff, the working group includes four subgroups, which focus on Policy, Internal Controls and Data Quality, Audit Coordination, and the DATA Act Information Model Schema (DAIMS), respectively. OMB staff also told us that by December 2018 an interagency board and council, both led by the General Services Administration (GSA), had begun to advise OMB on policy matters. According to an action plan that OMB and GSA released along with the March 2018 PMA, the new interagency Shared Solutions Governance Board (SSGB) and Business Standards Council (BSC) are responsible for setting goals and providing advice to promote a government-wide marketplace for shared services. Specifically, they cover mission-support services such as human resources and financial management that a small number of providers offer to many agencies. According to OMB staff, this oversight function involves creating and administering government-wide data standards, including data standards established to support the DATA Act. The SSGB includes executives from across the federal government and is responsible for making recommendations to OMB on shared services policy. The BSC provides expertise on various subject matter areas (e.g., procurement and financial assistance) to promote the development of common business capabilities and data standards. The action plan does not discuss how the functions carried out by the SSGB and BSC apply specifically to the data standards established under the DATA Act. In commenting on a draft of this report, OMB staff told us that the “Governance Ecosystem” page on the website of Unified Shared Services Management (USSM) describes the SSGB and links functions of the SSGB and BSC to the DATA Act. They said it does this by showing that OMB and Treasury have key roles in all three entities. However, this common membership does not, in itself, provide the transparency and clarity of documented policies and procedures for governing DATA Act standards. Treasury officials said that the CFOC Working Group is involved in aligning DATA Act data standards across various functional communities, including procurement and financial assistance. Further, the group is considering making recommendations to OMB regarding changes to data definitions and other policy matters. For example, Treasury officials told us that in fall 2018, the DAIMS Subgroup identified difficulties in aligning different definitions of Period of Performance Start Date used for procurement and in financial assistance awards, and plans to elevate this issue to the Policy Subgroup for review. Specifically, the DAIMS Subgroup found that it is not always clear whether the start date should be reported as the date when a specific transaction occurred or the date when the original underlying award was made. This choice about how to interpret the data element can have substantial consequences for the consistency of the data reported. For example, in some cases, the underlying awards for recent transactions were made in the 1960s or 1970s. According to OMB staff and Treasury officials, at the center of this shifting array of advisory bodies, the DATA Act Executive Steering Committee (ESC) has continued to meet regularly and to serve as the top-level governance body for DATA Act implementation. OMB staff told us that the ESC is chiefly responsible for setting government-wide policy for the data standards based on the recommendations from various advisory bodies. In addition to the ESC, Treasury has continued to maintain and update the DAIMS and DATA Act Broker, following a set of change control procedures that involve consultation with stakeholders and public release of information about updates. OMB and Treasury Have Instituted Some Data Governance Activities but Have Not Established a Set of Clear Policies and Processes Although OMB has taken some steps to address our recommendation, efforts are still needed to establish a clear set of policies and procedures for governing the data standards established under the act. The key practices for data governance that we identified in our previous work are shown in table 1. In the specific context of the DATA Act standards, Treasury and OMB have taken steps to enforce the use of data standards by directing agencies to develop and maintain data quality plans and requiring agencies to submit data through the DATA Act Broker. The broker performs validations to improve data quality and ensure the consistent application of data standards. However, because the approach to governing DATA Act data standards has continued to evolve during the past few years, and because a set of data governance policies and procedures is not documented, we were unable to conduct a comprehensive assessment of OMB’s and Treasury’s data governance efforts against leading practices. While some data governance activities have been undertaken within the specific context of DATA Act data standards, others are part of broader efforts under the PMA. In July 2018, OMB staff told us that governance over the DATA Act data standards would be accomplished within the broader context of the CAP goals established under the PMA. For example, OMB established a governance structure to achieve the objectives of CAP goals related to “Results-Oriented Accountability for Grants.” As part of this broader effort to standardize grants management business processes and data to increase efficiency and reduce reporting burden, OMB, the Department of Health and Human Services, and other federal agencies have published a list of draft grants management data standards for public comment. However, published documents describing this effort do not explain how the process for developing grants management standards under this CAP goal would apply specifically to the data standards established under the DATA Act. Nor do they address if or how these new standards align with those established under the act. Further, none of the documentation on the PMA’s governance structure for grants management mentions the DATA Act. In commenting on a draft of this report, OMB staff told us that the staff members from OMB and Treasury who are responsible for the grants management standards are the same people involved in managing the DATA Act standards. While this connection between the two efforts may provide adequate communication in the short term, staffing is likely to change over time, and there is no assurance that the same people will always be involved. As we have reported previously, having documented policies in place that delineate clear roles and responsibilities for decision-making could help to ensure continuity into the future. As the Comptroller General testified in 2015, in the absence of a clear set of institutionalized policies and processes for developing standards and for adjudicating necessary changes, the ability to sustain progress and maintain the integrity of established data standards may be jeopardized as priorities and data standards shift over time. OMB Does Not Have Procedures for Updating Data Definition Standards OMB Has Not Established Procedures for Making Decisions about Changes to Existing Data Definitions Managing and controlling changes to data standards is a key activity for data standardization and effective data governance. The DATA Act requires OMB and Treasury, in consultation with the heads of federal agencies, to establish government-wide financial data standards that include common data elements for financial and payment information required to be reported by federal agencies and entities receiving federal funds. Among other requirements, these standards, to the extent reasonable and practicable, must be capable of being continually upgraded as necessary. According to key practices for data governance that we identified in our previous work, organizations should have documented policies and procedures for making decisions about changes to existing data standards. In June 2018, OMB staff changed certain data definitions in the publicly accessible website that serves as the official repository for the data definitions. However, OMB does not have a documented procedure for updating or making changes to these definitions. In commenting on a draft of this report, OMB staff stated that the DATA Act Information Model Schema (DAIMS) change control procedures were the method for updating data standards. However, OMB’s website for data definitions is maintained separately from the DAIMS, and the DAIMS procedures only address changes to the DAIMS, and do not address this separate repository of data definition standards. OMB staff said that the June 2018 revisions were made in response to the findings of our November 2017 report. Specifically, OMB revised the Primary Place of Performance Address definition to no longer include a street address or county. OMB amended the definition of Record Type to clarify that it applies to financial assistance awards only. As shown in figure 1, OMB also amended the explanatory text preceding the definitions to revise and clarify its policy regarding agencies’ use of data definitions. OMB staff described the changes to definitions as minor technical corrections to align with the reporting instructions in the DAIMS. In December 2018, OMB staff informed us that OMB’s procedure for making changes to the data definitions it maintains in the official repository can be found on the “Governance Ecosystem” page of the website of Unified Shared Services Management (USSM). However, our review of that page in January 2019, including the links it provides to other pages, found no evidence that the website provides any documentation related to the DATA Act. In particular, we found no evidence of a documented procedure for making changes to the data definitions in OMB’s official repository. The staff were unable to provide documentation to show that any standard procedure was followed in making the June 2018 changes, or that the DATA Act Executive Steering Committee approved the changes. As discussed earlier in this report, that committee is the top-level governing body for DATA Act implementation and is responsible for approving changes to data standards. The evolution of OMB’s approach to developing a governance structure to maintain the integrity of the DATA Act data standards could in part explain the lack of a documented procedure for updating the definitions. As discussed above, OMB has created and disbanded various advisory bodies for DATA Act data standards and has only recently decided on an approach for formalizing governance over the standards, namely the decision to integrate governance of these standards with the governance processes administered by the SSGB. In 2015, we reported that establishing a formal framework for providing data governance throughout the life cycle of developing and implementing these standards is critical for ensuring that the integrity of the standards is maintained over time. Without established written procedures for making revisions to data definitions, needed changes may not be made in a timely manner, which could impair data quality. For example, if the definitions in the DATA Act official repository and definitions in other sources are not aligned, then agency staff responsible for DATA Act compliance and reporting may make inconsistent choices about which definitions to apply when creating and submitting data. As we have previously reported, the current data governance structure did not prevent inconsistencies between the DAIMS and the official repository for data definitions. OMB Revised Data Definition Standards without Transparently Communicating the Changes to Stakeholders Changes to data standards for federal spending data should be transparently communicated to stakeholders, including the public. The DATA Act requires OMB and Treasury to consult with public and private stakeholders in establishing data standards. In addition, according to key practices for data governance that we identified in 2016, organizations should have documented policies and procedures for managing, controlling, monitoring, and enforcing consistent application of data standards and for obtaining input from stakeholders and involving them in key decisions, as appropriate. Standards for internal control in the federal government state that management should externally communicate the necessary quality information to achieve the entity’s objectives. These objectives can include those relating to the release of reliable information in accordance with appropriate standards, applicable laws and regulations, and expectations of stakeholders. In the context of standards for transparently reporting federal spending data, stakeholders include the general public as well as staff at federal agencies. OMB did not transparently communicate the June 2018 revisions. OMB staff said that the changes were communicated in OMB Memorandum M- 18-16, which was issued on June 6, 2018. As shown in figure 2, a footnote in that memorandum contains a link to the official web page for OMB’s Office of Federal Financial Management. That page includes a link, labeled “DATA Act Data Standards,” to the public MAX.gov page that serves as the official repository for the data definition standards. However, neither this footnote nor other text in the memorandum makes reference to changes made to the definitions and policy. As of March 18, 2019, the official repository did not indicate that any changes have been made since the initial creation of the definitions in 2015. OMB did not provide documentation showing that the revisions were communicated to the public or to specific categories of stakeholders, such as users of the data standards within the federal government. As described below, the procedures that Treasury has implemented for managing changes to technical guidance, including publishing revision histories for guidance documents, represent one potentially effective approach to informing stakeholders, including the public, about changes to data standards. OMB staff viewed the revisions made in June 2018 as minor technical corrections that were needed to align the definitions with other OMB policies and with the consensus view of stakeholders at the time the data standards were first established. Consequently, they did not believe it was necessary to communicate these revisions publicly or indicate in the official repository that changes had been made. However, these revisions required significant changes in some federal agencies’ use of data definitions. As we reported in November 2017, some agencies applied DATA Act definitions directly when generating data to be reported to USAspending.gov. The new explanatory text added to the data definition repository instructs agencies not to apply these definitions directly, but instead to apply the more detailed definitions contained in regulations and policies governing the making and management of awards. Without transparent communication of changes to data definition standards, stakeholders—including staff at federal agencies required to report data according to these definitions—may miss important information relating to changes in how, when, and by whom data definitions are to be applied. The staff may then report data that are not consistent and comparable across the federal government. Such inconsistent reporting can undermine the transparency goals of the DATA Act, particularly when it affects key data elements, such as those describing geographical information. For example, we found in November 2017 that inconsistent data were reported about the locations where the federal government spends money, because some agencies used OMB’s DATA Act definition of the Primary Place of Performance data element, while other agencies used definitions from other sources, such as the data dictionary for the Federal Procurement Data System – Next Generation (FPDS-NG). In addition, a revision history showing when clarifications of policy and corrections to data standards were made could assist users of federal spending data, including historical data, in interpreting those data and assessing their reliability and quality. Treasury Has Procedures in Place for Communicating with Stakeholders Regarding Changes to Technical Guidance Treasury has established procedures for consulting with and informing stakeholders, including the public, about changes to technical guidance and reporting processes. Treasury’s stakeholder engagement process includes regular review of and revisions to its technical guidance. Before revisions to guidance are put into effect, Treasury staff circulate proposed changes through an email list that any member of the public can subscribe to, discuss these changes at frequent meetings with federal agency staff responsible for DATA Act reporting, and provide opportunities for agencies to test reporting under the new rules and provide feedback from this testing to Treasury. In addition, the guidance documents provide logs of all changes that have been made since the documents were created. According to Treasury staff, the most important tools for ensuring that agencies report consistent and comparable data are the DATA Act Information Model Schema (DAIMS) and the DATA Act Broker. Treasury’s documentation states that the DAIMS is “the data standard of the DATA Act” and contains standardized data elements that are complete and reflect the requirements of the act. The DAIMS includes reporting guidance that provides agencies with a complete listing of data elements they must report as well as a complete listing of data elements that will be extracted from government-wide systems, such as FPDS-NG. The DAIMS also includes a validation rules document that describes the business rules employed by the DATA Act Broker, which is Treasury’s system for collecting and validating agency data. Treasury provides federal agencies with detailed procedures for submitting DATA Act data to the broker. Before making changes to the DAIMS and DATA Act Broker, Treasury provides stakeholders with information about the planned changes and an opportunity to comment on them. For example, in June 2018, Treasury released DAIMS 1.3, an updated version of the DAIMS to be implemented during fiscal year 2019. Before releasing the final version of DAIMS 1.3, on June 29, 2018, Treasury shared its plans for the release with stakeholders through the Chief Financial Officers Council’s DATA Act Working Group (CFOC Working Group) and office hours calls. Treasury also transmitted a notice of proposed changes to federal agencies, collected comments from agencies during a designated comment period, and included responses to these comments in the final version of the release. Before implementing any of the changes in DAIMS 1.3 in the DATA Act Broker, Treasury provided agencies with a testing environment that allowed agency staff to identify any issues with the changes before those changes were applied to data published on USAspending.gov. Treasury’s documentation for the public and for federal agencies includes detailed information about the history of changes to the DAIMS. Each of the DAIMS guidance documents includes a change log that shows revisions made since the document was first created. The detailed information Treasury provides about changes to technical guidance can promote data quality and transparency by ensuring that federal staff are aware of reporting requirements, and that users of the data understand how those data are created and reported. Conclusions Since 2014, OMB and Treasury have made significant strides to address the DATA Act’s requirements for standardization of federal spending data. As they move forward, appropriately and effectively managing changes to data standards will be critical to ensuring the quality and comparability of the data across the federal government. Treasury has instituted regular procedures for making changes to technical data standards, including procedures for consulting with stakeholders and for recording and communicating changes. OMB has taken responsibility for maintaining an official list of data definition standards separate from the technical data standards maintained by Treasury. However, OMB lacks comparable procedures for maintaining these data definition standards. OMB made changes to some of the definitions and clarified policies about how they are to be applied, but did not communicate those changes to stakeholders, including the public. Definitions of data elements and policies about how those definitions are to be applied are a key component of the management of federal spending data under the DATA Act. Although OMB has completed the task of creating an initial set of definitions, it has not formally and explicitly documented a consistent approach for maintaining the integrity of the data definition standards over time as we previously recommended. Until OMB establishes procedures to ensure that changes are controlled, it will continue to be a challenge to apply and interpret these definitions consistently, presenting risks to data quality. In addition, clearly identifying the changes that have already been made in the official repository could aid agency officials in reporting data and users in understanding the context in which past data have been reported. These actions would be important steps toward improving control over the data standards, creating an effective governance structure, and ultimately improving the consistency and quality of federal spending data. Recommendations for Executive Action We are making two recommendations to the Office of Management and Budget: The Director of OMB should clarify and document OMB’s procedure for changing official data definition standards for DATA Act reporting, for example, by explicitly describing how change procedures developed for other government-wide initiatives apply to DATA Act definition standards in a public source of guidance or information. (Recommendation 1) The Director of OMB should ensure that the June 2018 policy changes regarding DATA Act data definition standards are clearly identified and explained in the official repository or another authoritative public source of DATA Act standards and guidance, such as by including a revision history along with the current version of the definitions. (Recommendation 2) Agency Comments and Our Evaluation We provided a draft of this report to OMB and Treasury for review and comment. OMB neither agreed nor disagreed with our recommendations, and OMB staff from the Office of Federal Financial Management provided oral comments, which are summarized below and incorporated as appropriate in the report. Treasury informed us that they had no comments on the draft report. In their oral comments, OMB staff stated that on the whole, the report correctly described the complex ecosystem of governance over data standards for federal spending data. However, OMB staff stated that in certain places the report did not fully capture the extent of OMB’s actions related to data governance for the DATA Act data standards. According to OMB staff, the Shared Solutions Governance Board (SSGB), under OMB’s direction, plays an important role in governing DATA Act data standards. OMB staff said that this relationship exists because the same agencies and staff participate in both the SSGB and the governance of the DATA Act data standards. In addition, OMB staff confirmed that descriptions of the specific roles and responsibilities of the SSGB, CFOC Working Group, and the Treasury office that manages the DAIMS have not been documented. OMB staff said that many of the same agency personnel participate in all of these groups, and therefore work closely together on a regular basis. OMB staff stated that this close involvement results in effective communication and a consistent approach to governance, and ensures an understanding of the procedures for changing data standards even though those procedures are not formally documented. We acknowledge OMB’s assertion that the various groups for creating and administering government-wide data standards (including data standards established to support the DATA Act) share many of the same staff. However, OMB’s approach relies on the continued participation of the same staff in order to maintain continuity rather than relying on documented policies, procedures, roles, and responsibilities for data governance functions. A key benefit of having a robust, institutionalized data governance structure is to provide consistent data management during times of change and transition, such as during staffing transitions or administration changes. It is important for OMB to clearly delineate roles and responsibilities so stakeholders understand how governance of the DATA Act standards is accomplished within the broader context of the PMA and other efforts. OMB staff also said they have communicated all changes to DATA Act data standards that have been made since the standards were created. OMB staff told us that the DAIMS is the official data standard for DATA Act reporting and, as such, includes logs that record all changes to the standards. According to OMB staff, OMB updated its public data standards web page on www.max.gov in June 2018 to fix an error and ensure that the page matched the DAIMS. Staff stated their belief that such a correction did not represent an actual change to a data standard and therefore did not need to be recorded in the DAIMS change log or communicated publicly. However, guidance issued in June 2018, OMB Memorandum M-18-16, identifies the MAX.gov web page as the official repository of the data standards. Specifically, the guidance directs agencies to report data in accordance with the standards maintained by OMB and Treasury pursuant to FFATA, as amended by the DATA Act, and provides a link to the OFFM website’s listing of data standards definitions. If OMB chose to identify the DAIMS—instead of the MAX.gov page—as the official source of data standards, then the issue about changes not being identified on the MAX.gov page would not be important. Although OMB made conforming changes based on our input to align the definition of Primary Place of Performance on the MAX.gov web page, having clearly documented procedures for making changes to the data standards and for ensuring that changes are communicated widely is important for ensuring the consistent and comparable reporting envisioned under the act. Additionally, in June 2018, OMB made an important change to the explanatory text that precedes these official data definitions as posted on the MAX.gov website, clarifying OMB’s policy regarding the use of the DATA Act data definitions. OMB staff acknowledged that that this clarification could have been publicized more effectively, which is why we continue to believe that our second recommendation—to include a revision history along with the current version of the DATA Act data definitions—remains valid. We are sending copies of this report to the Secretary of the Treasury and the Acting Director of OMB, as well as interested congressional committees and other interested parties. This report will be available at no charge on our website at http://www.gao.gov. If you or your staff have any questions about this report, please contact Michelle Sager at 202-512-6806 or SagerM@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report. Key contributors to this report are listed in appendix III. Appendix I: Interagency Groups Responsible for DATA Act Governance Appendix I: Interagency Groups Responsible for DATA Act Governance GitHub is a web-based software repository hosting service. The Federal Spending Transparency website can be found at: http://fedspendingtransparency.github.io/. JIRA is an online software development tool that Treasury uses to provide responses to stakeholder questions and comments related to the development of the broker. Appendix II: Status of Open GAO Recommendations Related to the DATA Act Appendix III: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Peter Del Toro, Assistant Director, and Kathleen M. Drennan, Analyst-in-Charge, supervised the development of this report. Theodore Alexander and Sherrice Kerns made major contributions to this report. Also contributing to this report in their areas of expertise were David M. Ballard, Ann Czapiewski, Jenny Chanley, Robert Gebhart, Michael LaForge, Carl Ramirez, Andrew J. Stephens, and James Sweetman, Jr. Related GAO Products Open Data: Treasury Could Better Align USAspending.gov with Key Practices and Search Requirements. GAO-19-72. Washington, D.C.: December 13, 2018. DATA Act: Reported Quality of Agencies’ Spending Data Reviewed by OIGs Varied Because of Government-wide and Agency Issues. GAO-18-546. Washington, D.C.: July 23, 2018. DATA Act: OMB, Treasury, and Agencies Need to Improve Completeness and Accuracy of Spending Data and Disclose Limitations. GAO-18-138. Washington, D.C.: November 8, 2017. DATA Act: As Reporting Deadline Nears, Challenges Remain That Will Affect Data Quality. GAO-17-496. Washington, D.C.: April 28, 2017. DATA Act: Office of Inspector General Reports Help Identify Agencies’ Implementation Challenges. GAO-17-460. Washington, D.C.: April 26, 2017. DATA Act: Implementation Progresses but Challenges Remain. GAO-17-282T. Washington, D.C.: December 8, 2016. DATA Act: OMB and Treasury Have Issued Additional Guidance and Have Improved Pilot Design but Implementation Challenges Remain. GAO-17-156. Washington, D.C.: December 8, 2016. DATA Act: Initial Observations on Technical Implementation. GAO-16-824R. Washington, D.C.: August 3, 2016. DATA Act: Improvements Needed in Reviewing Agency Implementation Plans and Monitoring Progress. GAO-16-698. Washington, D.C.: July 29, 2016. DATA Act: Section 5 Pilot Design Issues Need to Be Addressed to Meet Goal of Reducing Recipient Reporting Burden. GAO-16-438. Washington, D.C.: April 19, 2016. DATA Act: Progress Made but Significant Challenges Must Be Addressed to Ensure Full and Effective Implementation. GAO-16-556T. Washington, D.C.: April 19, 2016. DATA Act: Data Standards Established, but More Complete and Timely Guidance Is Needed to Ensure Effective Implementation. GAO-16-261. Washington, D.C.: January 29, 2016. DATA Act: Progress Made in Initial Implementation but Challenges Must Be Addressed as Efforts Proceed. GAO-15-752T. Washington, D.C.: July 29, 2015. Federal Data Transparency: Effective Implementation of the DATA Act Would Help Address Government-wide Management Challenges and Improve Oversight. GAO-15-241T. Washington, D.C.: December 3, 2014. Government Efficiency and Effectiveness: Inconsistent Definitions and Information Limit the Usefulness of Federal Program Inventories. GAO-15-83. Washington, D.C.: October 31, 2014. Data Transparency: Oversight Needed to Address Underreporting and Inconsistencies on Federal Award Website. GAO-14-476. Washington, D.C.: June 30, 2014. Federal Data Transparency: Opportunities Remain to Incorporate Lessons Learned as Availability of Spending Data Increases. GAO-13-758. Washington, D.C.: September 12, 2013. Government Transparency: Efforts to Improve Information on Federal Spending. GAO-12-913T. Washington, D.C.: July 18, 2012. Electronic Government: Implementation of the Federal Funding Accountability and Transparency Act of 2006. GAO-10-365. Washington, D.C.: March 12, 2010.


Background Signed into law on May 9, 2014, the DATA Act required OMB, or an agency it designated, to establish a pilot program to facilitate the development of recommendations for (1) standardized reporting elements across the federal government, (2) elimination of unnecessary duplication in financial reporting, and (3) reduction of compliance costs for recipients of federal awards. To meet these requirements, OMB established a pilot program with two components—one that focused on federal grants and another on federal contracts (procurement). OMB designated HHS as the executing agency of the grants portion of the Section 5 Pilot with oversight from OFFM. OFPP was responsible for designing and leading the procurement portion of the pilot focusing on reporting of Federal Acquisition Regulation (FAR) procurement requirements. OFPP collaborated with the Chief Acquisitions Officers’ Council and GSA on specific aspects of implementation including the development of the Central Reporting Portal, a reporting tool which is intended to centralize FAR reporting. See figure 1 for a timeline of the activities undertaken by the grants and procurement portions of the pilot as well as deadlines required by the act. As part of our ongoing oversight of the DATA Act’s implementation, we have monitored OMB’s efforts to meet its statutory requirements related to the Section 5 Pilot. In April 2016, we reported on the design plans for the Section 5 Pilot. We found that HHS’s design for the grants portion of the pilot was generally on track to meet statutory requirements and partially adhered to leading pilot design practices. However, we also reported that the procurement portion was not on track to meet requirements, and that its plans did not follow leading pilot design practices. In response to a recommendation in our report, OMB revised its plan for the procurement portion to better reflect leading practices for pilot design identified in our April 2016 report. These changes included more fully documenting its data collection plans and including a sampling plan to meet diversity requirements for pilot participants. According to OMB staff, the ongoing work and related grants guidance resulting from the Section 5 Pilot reflects a broader strategy for reducing federal recipient reporting burden that is outlined in the President’s Management Agenda (PMA). Released in March of 2018, and led by the Executive Office of the President and the President’s Management Council, PMA is a strategy to modernize how federal agencies deliver mission outcomes and provide services in three key areas: (1) modern information technology; (2) data, accountability, and transparency; and (3) the workforce for the 21st Century. Several Cross-Agency Priority (CAP) goals include PMA’s milestones and activities. These CAP goals identify opportunities for multiple agencies to collaborate on government-wide efforts and report on goal progress quarterly. Two of these, CAP Goals 5 and 8, include strategies for reducing federal award recipient reporting burden. OMB staff told us that some of the findings from the Section 5 Pilot and recommendations from their subsequent report to Congress informed the focus of these CAP goals. For example, according to OMB staff, the grants portion of the Section 5 Pilot focused on identifying how changes in grants data collection and grant management may reduce federal recipient reporting burden. PMA CAP Goal 8 is described as building on these efforts by shifting the focus toward the life cycle of grants management and standardizing grants management activities using agile technology. Section 5 Pilot Met Many but Not All Statutory Requirements We determined that the Section 5 Pilot fully met three of the DATA Act’s statutory requirements, substantively met one, and partially met two others. The Section 5 Pilot fully met the following statutory requirements: (1) that pilot data collection cover a 12-month reporting cycle; (2) timely issuance of OMB’s report to Congress in August of 2017 to select congressional committees; and (3) that the report to Congress contain a discussion of any needed legislative actions as well as recommendations related to automating and streamlining aspects of federal financial reporting to reduce the reporting burden of federal award recipients. We found that the pilot also substantively met the requirement that the pilot program include a combination of federal award recipients and an aggregate value of awards of not less than $1 billion but not more than $2 billion. Although the $122 billion in grants included in the pilot greatly exceeded the upper bound, this was principally a result of the decisions by OFFM and HHS to pilot different test models for reducing reporting burden, and to include a wide range of different types of grants. The total value of grant awards exceeded the amount envisioned by the act. OMB’s August 2017 report stated that the decision to go beyond the minimum requirement of testing one approach was made in the interest of achieving the DATA Act’s objective to identify ways to reduce reporting burden as well as the effect this decision would have on the aggregate value of grants sampled. We believe that the pilot substantively met this requirement and did not identify any negative effects related to the larger aggregate value of grants, contracts, and subawards included in the grants portion of the pilot. We found that the approach followed by OMB and HHS furthered the broader objective identified by this section of the act. In addition, we determined that the pilot partially met two of the act’s requirements. The first of these requirements concerns the act’s requirement that OMB’s report to Congress include a description of the data collected, the usefulness of the data provided, and the cost to collect pilot data from participants. The report that OMB issued to Congress in August 2017 included information on the first two of these but only partly addressed the third. Specifically, it contained cost information for only the grants portion of the pilot, stating that the cost associated with executing this portion during fiscal years 2015 through 2017 was more than $5.5 million. The report did not contain any cost information on the procurement portion of the pilot. The DATA Act also required that OMB issue guidance to agencies for reducing reporting burden for federal award recipients—including both grantees and contractors—but the guidance subsequently issued only pertained to the grants community. We determined that OMB only partially met this requirement. On September 5, 2018, OMB issued M-18- 24: Strategies to Reduce Grant Recipient Reporting Burden. Among other things, this memorandum contained guidance to federal agencies making the SF-424B form optional based on findings from the grants portion of the pilot. Form SF-424B is used by grantees to document assurances regarding their compliance with a wide range of rules and regulations. Figure 2 summarizes our assessment. The Grants Portion of the Pilot Identified Several Ways to Reduce Reporting Burden and Provided Support for Government-Wide Streamlining Efforts All Six Grant Test Models Reported Evidence of Reducing Burden, Increasing Accuracy, or Both As the agency designated by OMB to execute the grants portion of the Section 5 Pilot, HHS developed and analyzed six “test models” to determine if adopting the proposed changes would contribute to the pilot program’s objectives of reducing reporting burden and duplication. These test models examined a variety of grant reporting issues that HHS had identified as presenting challenges. All but one of the test models, the Common Data Element Repository (CDER) Library 2, based their findings on data collected from grantees. The text box below provides high-level summaries of each of the six models. Additional details on the approach followed for each model, as well as reported results, can be found in appendix II. OMB Used Findings from the Grants Portion of the Pilot to Support Recommendations and Government-wide Guidance for Reducing Grantee Reporting Burden OMB’s August 2017 report to Congress on the findings of the Section 5 Pilot contained three broad recommendations and stated that OMB plans to take action on these recommendations. These recommendations covered (1) standardizing core data elements, (2) eliminating duplication through auto-population of data, and (3) leveraging information technology open data standards to develop new tools across the federal government. We found that evidence from the grant test models supported all three recommendations for streamlining federal reporting discussed in the report. For example, OMB recommended that its staff standardize core data elements used for managing federal financial assistance awards based on reductions in administrative burden experienced in the CDER Library 1 test model. In another example, four test models supported OMB’s recommendation for increased use of data auto-population from existing federal data sources as a way to reduce duplication in reporting. Findings from the grants portion of the Section 5 Pilot also provided support for government-wide efforts to streamline reporting and reduce recipient reporting burden. These include OMB’s memorandum M-18-24: Strategies to Reduce Grant Recipient Reporting Burden, which discusses efforts to automate and centralize grant management processes. Among other things, M-18-24 requires that federal agencies evaluate the systems and methods currently used to collect information from grant recipients to eliminate duplicative data requests. OMB staff confirmed that M-18-24 incorporates findings from some of the test models of the grants portion of the pilot such as the Single Audit test model, which examined reducing duplicative reporting of grant recipients’ data. The efforts to reduce duplicative reporting in M-18-24 also align with OMB’s recommendation in its August 2017 report to Congress to eliminate unnecessary duplication in reporting by leveraging information technology that can auto-populate from existing data sources. In addition, OMB staff told us that findings from the grants portion of the pilot contributed to broader, government-wide initiatives related to federal reporting. For example, according to OMB staff, the three recommendations from the August 2017 report to Congress are reflected in CAP Goal 8 of the President’s Management Agenda, which focuses on results-oriented accountability for grants. These OMB staff also told us that findings from the grants portion of the pilot informed two CAP Goal 8 strategies. For example, the CAP Goal 8 grants management strategy focuses on standardizing grants management business processes and data. OMB developed a comprehensive taxonomy for core grants management data standards that is currently available for public comment. In addition, a second strategy focuses on incorporating a risk- based performance management approach to metrics in grant award operations to determine low-risk and high-value federal awards. CAP Goal 8 also states plans to streamline the 2019 Single Audit Compliance Supplement to focus on requirements that inform grant award performance. Procurement Portion of Pilot Did Not Result in Sufficient or Appropriate Data to Assess Changes in Contractors’ Burden Reduction Lack of Contractor Participation and the Absence of Iterative and Ongoing Stakeholder Engagement Limited the Ability of Procurement Pilot to Achieve its Objectives Unlike the grants portion of the pilot, the procurement portion did not result in data collection that could be used for an evidence-based assessment of ways to reduce reporting burden. OMB’s Office of Federal Procurement Policy (OFPP) sought to assess five test models that, according to the report to Congress, were essential to centralized procurement reporting. However, the pilot did not fully test any of the hypotheses associated with those test models. The reasons for not testing the hypotheses included a lack of contractor participation and a lack of iterative and ongoing stakeholder participation and engagement throughout the course of the pilot. See appendix III for additional information regarding the various procurement test models, associated hypotheses, and additional details regarding our assessment. The procurement portion of the pilot focused entirely on the development and testing of a central reporting portal to consolidate FAR reporting requirements. According to OFPP staff, the pilot intended to eventually identify ways to centralize a wide range of reporting requirements that contractors currently meet through decentralized methods. Contractors must report many types of information depending on the contract. Toward that end, OFPP, with the assistance of GSA, created a procurement reporting website called the Central Reporting Portal. To test the efficacy of this portal for reducing burden, OFPP initially decided to examine how well it handled a specific FAR reporting requirement—the reporting of payroll data in accordance with the Davis-Bacon Act. According to pilot plans, Davis-Bacon reporting requirements were selected because they were identified by contractors as “pain points” during initial stakeholder outreach conducted in 2014 and 2015. OFPP planned to collect and analyze 1 year of weekly Davis-Bacon wage reporting data from at least 180 contractors through the Central Reporting Portal to identify how centralized reporting might reduce contractor reporting burden. However, during the 12-month procurement data collection period, no contractors agreed to submit their Davis-Bacon data as part of the pilot. Consequently, OFPP did not collect any wage data. Despite OFPP stating in its plans and reiterating to us as late as September 2017 that it expected to be able to secure at least 180 pilot participants, only one contractor expressed interest in reporting its Davis-Bacon information using the portal. This contractor withdrew from the pilot before submitting any data through the Central Reporting Portal. OFPP staff told us they were aware of the potential for low pilot participation for Davis- Bacon reporting when pilot testing began in February 2017 because contractors already had established processes for fulfilling the highly complex Davis-Bacon reporting requirements, and pilot participation was optional. According to GSA contracting staff, the one contractor who initially expressed interest ultimately decided not to participate because the format in which the contractor tracked and reported payroll data was incompatible with that used by the pilot portal, resulting in additional burden. However, it was not until August 2017—approximately 7 months into its year-long data collection period—that specific steps were taken to address the fact that the procurement portion of the pilot had not collected any data from Davis Bacon contractors. During this period OFPP did not conduct pilot outreach activities with the contractors, who were key to successful implementation of the pilot. OFPP staff told us that at the time of the pilot launch they learned that contractors were interested in having the Central Reporting Portal be able to communicate with third-party payroll reporting systems to automate reporting. OFPP staff said that although they are exploring this possibility, it was not a capability that was included as part of the pilot. Had this type of feedback on stakeholder needs been obtained sooner, OMB could have explored the feasibility of adding this capability to the portal or engaged in communication with stakeholders to develop alternate approaches that might have persuaded more contractors to participate. The usefulness of iterative and ongoing communication is recognized by the Standards for Internal Control in the Federal Government. Those standards state that management should use quality information to achieve its objectives, and that management should collect quality information by engaging with stakeholders through iterative and ongoing processes and in a timely manner. In this case, key stakeholders include relevant agencies, contracting officials, and contractors using the system. OFPP’s plan for the procurement portion of the pilot recognized the importance of stakeholder engagement stating that, to include a diverse group of recipients in the pilot, they should identify eligible participants for the pilot, conduct outreach to identify participants, and repeat this process as necessary until they achieved the sample necessary to test the Central Reporting Portal. However, as previously stated, no contractors agreed to submit their Davis-Bacon data as part of the pilot. Therefore, OFPP did not repeat this process until the pilot obtained the necessary sample size. Such interactions could have provided important information on contractors’ needs and concerns that OFPP could have used to inform their decisions regarding the pilot’s implementation. Expansion of Procurement Pilot to Include Hydrofluorocarbon Reporting Had Limitations In November 2017, OFPP expanded the type of data accepted by the pilot to include hydrofluorocarbon (HFC) reporting, a new FAR reporting requirement. However, this choice had limitations in its suitability for providing useful data for testing the hypotheses of the five procurement test models. Unlike Davis-Bacon reporting, where contractors submit weekly reports, HFC is an annual reporting requirement for contractors that emit HFC gases over a certain threshold. The Central Reporting Portal is the only location where contractors can submit HFC reporting. For the purposes of the pilot, the Central Reporting Portal accepted HFC submissions from November 2017 through February 2018. During the pilot, 11 HFC annual reports were submitted to the portal (see figure 3). As a result of the small number of reports collected, OMB collected much less data than it had initially expected to receive to test the capabilities of the Central Reporting Portal. If the procurement portion of the pilot had been executed as planned, it could have theoretically resulted in 9,360 Davis-Bacon submissions for analysis. A larger data set of contractors’ experiences using the Central Reporting Portal could have informed OMB’s decision-making process through analysis of more, and potentially more varied data. In addition to the small number of submitted HFC annual reports, the decision to switch to using HFC data had another limitation. These data could not be used to examine changes in reporting burden as a result of using the Central Reporting Portal. This is because HFC reporting was a new reporting requirement, and as such, it did not have an established reporting process to use as a point of comparison to assess changes in reporting burden. The objective of the procurement pilot was to assess how centralized reporting can reduce reporting burden. This objective could not be achieved without data on the existing reporting burden. OMB’s Recommendations for Streamlining Reporting Were Not Supported by Findings from the Procurement Portion of the Pilot Evidence from the procurement portion of the pilot did not support OMB’s government-wide recommendations for reducing reporting burden in its August 2017 report to Congress. As previously stated, OMB’s report to Congress included three recommendations that focused on (1) standardizing core data elements, (2) eliminating duplication by using data auto-population, and (3) leveraging information technology open standards to develop new tools. As support for the first recommendation, the report stated that results from the procurement pilot test models demonstrated that standard data elements—coupled with uniform data adoption—and the ability to centrally collect and share information reduces administrative burden. Since the procurement portion of the pilot did not gather or analyze any pilot data from the Davis-Bacon participants, OMB did not assess the extent to which the ability to centrally collect data actually reduces burden. Recommendation two stated that support from the procurement test model demonstrated that recipient burden is reduced when identical data can be entered once in one place and reused. However, the HFC data collection process did not reuse data when capturing information and did not have the ability to auto-populate data. HFC data collection was the only part of the procurement portion of the pilot that collected information that could have been used to inform this recommendation. According to OFPP staff, the Davis-Bacon portion of the portal had the capability to auto-populate data. However, no Davis-Bacon data were collected that would have allowed quantification of the effects of reusing data on reporting burden. OMB stated that support for the third recommendation included data and information collected from the pilot. Although there was some consultation with stakeholders during initial planning and design of the procurement portion of the pilot and the early development of the portal, the pilot did not actually collect any data from either Davis-Bacon contractors or through the HFC portion of the pilot in the data gathering and analysis portion of the pilot related to this recommendation. OMB Plans to Expand Use of the Central Reporting Portal to Streamline Reporting of FAR Requirements In August 2018, OMB announced plans to expand the use of the Central Reporting Portal for FAR reporting, stating that the portal allows contractors to report data to one central location. OFPP staff told us that they are considering centralizing a third FAR requirement using the portal in the future but have not yet determined what that will be. As discussed above, the procurement portion of the pilot did not collect sufficient data to test the effect of the portal on reporting burden. In addition, the plan for the procurement portion states that OFPP intended to analyze feedback on pilot data collection and, depending on that feedback, decide whether to expand the pilot to other FAR reporting requirements. However, the pilot did not collect any such feedback to inform its determination to expand the Central Reporting Portal in the future. As a result, OFPP has limited information regarding issues that could affect expanded use of the Centralized Reporting Portal. In the absence of such information, it is difficult for OFPP to determine whether continued or expanded use of the Central Reporting Portal will reduce reporting burden, and which additional FAR requirements, if any, to include. Conclusions To reduce the burden and cost of reporting for recipients of federal funds, Congress included specific provisions in the DATA Act to encourage OMB to take a deliberate and evidence-based approach toward developing guidance for federal agencies in this area. The Section 5 Pilot offered OMB a valuable opportunity—namely, to test a variety of methods and techniques at a small scale before applying them more widely. Such a process may enhance the quality, credibility, and usefulness of evaluations in addition to helping to ensure that time and resources are used more effectively. Similar to what we found when we analyzed the design of the Section 5 Pilot in 2016, our review of its implementation and the results it produced found differences between the grant and procurement portions. OMB and HHS designed and executed a robust grants portion of the pilot that tested several different approaches for reducing the reporting burden experienced by federal grant recipients. The resulting findings were used to develop OMB’s government-wide recommendations, and to inform two subsequent goals in the 2018 President’s Management Agenda related to reducing recipient reporting burden. In contrast, OMB did not fully implement the procurement portion of the pilot consistent with its plans. The procurement portion did not collect data to test the hypotheses associated with any of its five test models, and therefore could not provide empirical support for either OMB’s government-wide recommendations or guidance related to reducing reporting burden. Among the factors responsible for this were the lack of Davis-Bacon contractor participation and OMB’s inability to find a suitable alternative. OMB has announced its intention to expand centralized reporting for FAR requirements across government. In the absence of timely information regarding the needs and concerns of stakeholders, OMB faces the risk of experiencing implementation challenges similar to those it experienced during the pilot. Although the use of a centralized reporting portal could ultimately prove useful for reducing burden, the lack of information from stakeholders—including the contractors who would use it—raises concerns about the future success of plans for expanding the Central Reporting Portal. Recommendation for Executive Action The Director of OMB should ensure that information is collected regarding how centralized reporting of procurement requirements might reduce recipient reporting burden—including input from stakeholders such as contractors through an iterative and ongoing process—to inform OMB’s planned expansion of the Central Reporting Portal. Agency Comments and Our Evaluation We provided a draft of this report to OMB, HHS, and GSA for review and comment. HHS and GSA informed us that they had no comments. OMB provided technical comments, which we incorporated as appropriate. OMB neither agreed nor disagreed with our recommendation. We are sending copies of this report to the appropriate congressional committees, The Secretary of Health and Human Services, The Acting Director of OMB, the Administrator of GSA, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at 202-512-6806 or sagerm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report. GAO staff who made key contributions to this report are listed in appendix IV. Appendix I: Objectives, Scope, and Methodology This report assesses the extent to which (1) the Section 5 Pilot met the statutory requirements of the act, (2) the grants portion of the Section 5 Pilot demonstrated changes in federal award recipients’ reporting burden, and (3) the procurement portion of the Section 5 Pilot demonstrated changes in federal award recipients’ reporting burden. To assess the extent to which the pilot met statutory requirements we reviewed section 5 of the Federal Funding Accountability and Transparency Act of 2006, as amended by the Digital Accountability and Transparency Act of 2014, to determine the legal requirements set forth in the act pertaining to establishing, designing, and executing the Section 5 Pilot. We compared these requirements to documents from the Office of Management and Budget (OMB) and designated agencies. These documents included pilot plans for the grants and procurement portions of the pilot, OMB’s August 2017 report to Congress, M-18-23: Shifting from Low-Value to High-Value Work and M-18-24: Strategies to Reduce Grant Recipient Reporting Burden. We also interviewed staff from agencies involved in administering and executing the pilot on how they carried out their responsibilities. These agencies included the Department of Health and Human Services (HHS), OMB’s Offices of Federal Financial Management (OFFM) and Federal Procurement Policy (OFPP), and the General Services Administration (GSA). To assess the extent to which the grants portion of the Section 5 Pilot demonstrated changes in federal award recipients’ reporting burden, we reviewed HHS’ plans. We analyzed the plans compared to information collected from the various test models throughout the pilot. The data we assessed included survey data and analyses. We also assessed whether statements on changes in grantees’ reporting burden made in OMB’s August 2017 report to Congress were supported by documentation. We did this by verifying the statements against supporting information. We determined that the pilot data we reviewed were reliable for the purposes of our work by reviewing the data, tracing them back to underlying agency source documents, and interviewing relevant agency staff. We also interviewed OFFM staff and HHS officials on how the grants portion of the pilot was executed. To assess the extent to which the procurement portion of the pilot demonstrated changes in reporting burden, we reviewed OMB’s plans and compared them to actions OMB took to execute the pilot. We compared OMB’s actions to execute the procurement portion of the pilot against criteria identified in Standards for Internal Control in the Federal Government. We viewed a demonstration of the Central Reporting Portal tool for reporting Davis-Bacon and hydrofluorocarbon (HFC) submissions. GSA developed the portal and OFPP provided oversight for the portal’s development. We also reviewed documentation including HFC reporting submissions made through the portal. In addition, we interviewed OFPP staff, GSA officials responsible for administering the portal, and three contracting officials from GSA who were assigned to participate in the Davis-Bacon component of the procurement portion of the pilot regarding their actions related to implementing the procurement portion of the pilot. We conducted this performance audit from November 2017 to April 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Description of Test Models from Grants Portion of the Section 5 Pilot This appendix provides detailed information regarding the test models from the grants portion of the Section 5 Pilot. The Common Data Element Repository Library 1 Test Model The Common Data Element Repository (CDER) Library is an online repository for federal grants-related data standards, definitions, and context. The library is intended to be an authorized source for data elements and definitions for use by the federal government and for recipients reporting grant information. Hypothesis: If grant recipients are provided with definitions of data elements through the CDER Library, then they will be able to accurately complete forms in a timely manner. Methodology: The Department of Health and Human Services (HHS) divided test model participants into two groups to read a scenario based on the grants lifecycle and complete a data collection tool. The first group used the CDER Library to complete the data collection tool while the second group used all other available sources to complete the data collection tool. After completion of the data collection tool, test model participants filled out a survey about their experiences using the CDER Library. Test Model Metrics: Accuracy and completeness of captured data within a period of time and survey results. Example of Test Model Results: On average, test model participants that completed a data collection tool using the CDER Library scored 11 percent higher in the accuracy of information requested and, on average, spent 6 fewer minutes when completing the tool. Number of Test Model Participants: Fifty-nine. The Common Data Element Repository Library 2 Test Model The CDER Library 2 Test Model focused on identifying duplication in grant forms and data elements across the federal government based on the data standards, definitions, and context within the CDER Library 1. Hypothesis: If duplication across forms can be identified using the CDER Library, then agencies can update or reduce forms to reduce grant recipient burden. Methodology: HHS conducted an internal analysis of SF-424 form families, using the CDER Library, to identify duplication in data elements to determine which forms could be consolidated. Test Model Metrics: Number of duplicative fields within form families and across forms for selected federal entities Example of Test Model Results: The internal analysis conducted by HHS identified 371 instances of data element duplication across 10 agency grant funding applications when using standardized data elements from the CDER Library 1. Number of Test Model Participants: Not Applicable; the CDER 2 Library Test model did not collect information from test model participants because the test model was an internal document review. The CDER Library 2 test model tested the utility of the data element definitions within the CDER Library 1. The Consolidated Federal Financial Report Test Model The Consolidated Federal Financial Report Test Model focused on examining the potential early validation of consolidated CFFR data and potential future streamlining of the close-out process by allowing the submission of Federal Financial Report (FFR) data in one system, rather than in multiple entry systems. Hypothesis: If grant recipients can enter complete FFR information systematically through one entry point instead of multiple different avenues and that information could be shared electronically from that point forward, then grant recipient burden will be reduced and data accuracy will be improved. Methodology: HHS surveyed Administration for Children and Families grant recipients on their experience submitting a consolidated FFR via HHS’s Payment Management System, and grantees on their perceptions of the process for using a consolidated FFR through facilitated discussions. Test Model Metrics: Survey results. Example of Test Model Results: Sixty-four percent of the CFFR test model participants reported that submitting their FFR through a single system would result in reduced reporting time. In addition, 65 percent of the CFFR test model participants believed using the payment management system for submitting FFR data would improve the accuracy of the information they submitted. Number of Test Model Participants: One-hundred fifteen tested the pilot environment and 30 participated in the facilitated discussions. The Single Audit Test Model The Single Audit Test Model consisted of (1) an audit and opinions on the fair presentation of the financial statements and the Schedule of Expenditures of Federal Awards; (2) gaining an understanding of and testing internal control over financial reporting and the entity’s compliance with laws, regulations, and contract or grant provisions that have a direct and material effect on certain federal programs (i.e., the program requirements); and (3) an audit and an opinion on compliance with applicable program requirements for certain federal programs. The Single Audit Test Model focused on reducing reporting of data on duplicative forms. Hypothesis: If grant recipients do not have to report the same information on duplicative forms—for example, the SEFA compared to the Single Audit Report Package and Data Collection Form—then grant recipients’ burden will be reduced. Methodology: HHS collaborated with the Office of Management and Budget’s Office of Federal Financial Management and the Department of Commerce Federal Audit Clearinghouse (FAC) to create a pilot environment for test model participants to submit key portions of a modified Standard Form—Single Audit Collection. HHS conducted two focus groups with test model participants subject to the Single Audit. The first focus group discussed and completed a survey on the new form. The second group, a sample of test model participants who are subject to perform a Single Audit submitted the existing form in the FAC pilot environment, completed a separate data collection form similar to the new form, and completed a survey on the effectiveness and burden of the new form. Test Model Metrics: Focus group feedback and survey results. Example of Test Model Results: All test model participants with access to the Single Audit’s pilot environment believed the upload feature for reporting requirements could decrease duplication in required grant reporting. Number of Test Model Participants: Thirteen tested the pilot environment and 123 participated in facilitated discussions. The Notice of Award Test Model This model focused on the feasibility of developing a standardized Notice of Award (NOA) to reduce reporting burden and facilitate access to standardized data needed to populate Single Audit information collection. Hypothesis: If grant recipients have a standardized NOA for federal awards, then grant-reporting burden may be reduced for recipients by standardizing access to data needed to populate information collections. Methodology: HHS divided test model participants into two groups and completed a data collection tool. The first group completed the data collection tool using three standardized NOAs, while the second group completed the data collection tool using three non-standardized NOAs. After completion of the data collection tool, test model participants self-reported their respective times to complete the data collection tool. They also filled out a survey about the standardized NOA’s impact on reporting burden and provided input on elements to include in a standardized NOA. Test Model Metrics: Self-reported form completion time, accuracy, and survey results. Example of Test Model Results: Test model participants with access to the standardized NOA coversheets spent an average of 3 minutes less when completing the test model’s data collection tool. Number of Test Model Participants: One-hundred four. The Learn Grants Test Model The Learn Grants Test Model is a website on Grants.gov that summarizes and provides links to new and important grants information such as policies, processes, funding, and other information needed throughout the grants life cycle. The website intended to make it easier for stakeholders to find, learn about, and apply for federal grants and promote the standardization of grants terminology and data. Hypothesis: If grant recipients are supplied with grants life cycle information in one website, then they will have increased access to grants resources and knowledge of the grants life cycle process. Methodology: HHS developed a grants knowledge quiz from information on the Learn Grants website. HHS administered the knowledge quiz to test model participants in two phases. First, test model participants completed the knowledge quiz using existing knowledge and without the Learn Grants website. Next, test model participants completed the knowledge quiz with access to the Learn Grants website. HHS compared the results from both knowledge quizzes. After completion of the knowledge quiz, test model participants completed a survey on the usefulness of the Learn Grants website and its impact on increasing knowledge quiz scores. Test Model Metrics: Knowledge quiz accuracy and survey results on the usefulness of Learn Grants website. Example of Test Model Results: Test model participants experienced an average 10 percent (one quiz point) increase in their grant knowledge quiz scores when using the Learn Grants website. New grantees who participated in the test model also reported that the Learn Grants website provided useful grants information. Number of Test Model Participants: Fifty-seven. Appendix III: Assessment of Test Models in the Procurement Portion of the Section 5 Pilot Appendix III: Assessment of Test Models in the Procurement Portion of the Section 5 Pilot Hypothesis not tested. Hypothesis: Verification of FAR standards for post award reporting will confirm the value of existing data standards and reduce variations that will, in turn, reduce contractor burden and cost. Original plan (Davis-Bacon): OFPP planned to execute this test model through focus groups. According to OFPP, no focus groups were conducted. Revised Strategy (HFC): This hypothesis could not be tested through HFC reporting because it was a reporting requirement without an existing reporting method through which to compare reporting burden. 3. Prepopulate data into the Central Hypothesis not tested. Reporting Data Original Strategy (Davis-Bacon): OFPP planned to test this hypothesis by gathering data on the time it takes to submit reporting data through the Central Reporting Portal and outside of the portal, with self-reported data from contractors. According to OFPP, data were not collected due to a lack of participation in the Davis-Bacon portion of pilot. Revised Strategy (HFC): This hypothesis could not be tested through HFC reporting because it was a reporting requirement without an existing reporting method through which to compare reporting burden. Procurement Test Model and Hypothesis GAO’s Assessment 4. Consolidate data collection and Hypothesis not tested. access (proof of concept) Hypothesis: If contractors can enter FAR-required reporting data systematically through one entry point instead of multiple different avenues, and that information can be shared electronically with appropriate individuals, then contractor burden will be reduced and data access improved. Assessment Rationale Original plan (Davis-Bacon): OFPP planned to test this hypothesis by gathering data on the time it takes to submit reporting data through the Central Reporting Portal and outside of the portal, with self-reported data from contractors. OMB also planned to conduct guided discussions. According to OFPP, data were not collected due to a lack of participation in the Davis-Bacon portion of pilot. Revised Strategy (HFC): This hypothesis could not be tested through HFC reporting because it was a reporting requirement without an existing reporting method with which to compare reporting burden. 5. Central Reporting Portal can Hypothesis not tested, but metric associated with test model was met. Hypothesis: If interfaces can be built to support access to other reporting systems, contractor burden will be reduced. Original plan (Davis-Bacon): According to OFPP staff, the Davis-Bacon part of the Central Reporting Portal was able to provide prepopulating of data by interfacing with other reporting systems or drop down menus for all reporting fields. However, it could not demonstrate that such prepopulation resulted in a reduction of contractor burden. Revised Strategy (HFC): This is not applicable for HFC reporting which is reported through open fields. Although OFPP did not actually test the hypothesis associated with this test model, it did meet the metric that it had associated with the test model in its pilot plan. That metric is to develop prepopulating capabilities in the Central Reporting Portal by interfacing with other reporting systems. Appendix IV: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Peter Del Toro, Assistant Director; Silvia Porres-Hernandez, Analyst-in-Charge; Jazzmin Cooper; and Jimmy Nunnally made major contributions to this report. Also contributing to this report in their areas of expertise were Michael Bechetti, Jenny Chanley, Mike LaForge, Carl Ramirez, Stewart Small, Andrew J. Stephens, James Sweetman Jr., and Tatiana Winger.