Background CMS and private payers use a variety of quality measures to assess different aspects of health care quality. Process measures assess the extent to which providers effectively implement clinical practices (or treatments) that have been shown to result in high-quality or efficient care, such as the percentage of patients with a myocardial infarction who receive an aspirin prescription on discharge. Others are outcome measures, which track the results of health care, such as mortality, infections, and patients’ experiences of that care. To calculate providers’ performance on quality measures, CMS and private payers ask providers to report a variety of clinical data. Historically, providers have collected data for quality measures through a detailed, manual review of paper medical records. Other quality measures use data from billing records and patient surveys. More recently, a limited number of electronic quality measures have been developed to allow providers to report data electronically using electronic health records. Medicare Quality Programs Since the early 2000s, CMS has created a number of distinct quality reporting programs within Medicare. These programs generally focus on different sites of care, such as hospitals, physician offices, and nursing homes. Beginning in the early 2000s, CMS launched a number of related programs that offer financial incentives to providers receiving Medicare payments to report their performance on specified quality measures. Some of these programs, such as the Hospital Inpatient Quality Reporting program, are pay-for-reporting programs, in which providers may receive higher payments if they report their performance on the quality measures used in the programs. Others, such as the Hospital Value-based Purchasing program, are pay-for-performance programs, in which the level of providers’ performance on the quality measures affects the amount of the payment they receive. CMS also incorporates pay-for- performance in various alternative payment models, such as accountable care organizations—where CMS pays groups of providers based in part on the collective performance of those providers, rather than the fee-for- service traditionally paid in Medicare. Developing and Adopting New Quality Measures At any given point in time, CMS has a set of quality measures it is currently using in its various Medicare quality programs as well as efforts underway to identify different quality measures to better meet program needs. These quality measures may either already have been developed or potentially could be developed. A variety of different entities may develop new health care quality measures, such as the Joint Commission, the National Committee for Quality Assurance, and various medical specialty societies. In some cases CMS itself contracts with entities for the development of measures for use in its Medicare quality programs. CMS has developed a set of guidelines for developing new quality measures that are described in its Blueprint for the CMS Measures Management System. The Blueprint lays out the steps measure developers should follow to first identify health care topics or conditions where new measures are needed, and then develop and test specific new measures to fill those identified gaps. According to CMS estimates, it can take 2 years or more to complete all of these steps. As part of this process, CMS encourages entities that develop measures to submit them to the National Quality Forum (NQF), a nonprofit organization that evaluates and endorses measures—that is, determines which measures should be recognized as the best available for a given aspect of care. NQF has endorsed over 700 quality measures. In addition, NQF plays a major role in CMS’s process for determining which measures to use in its Medicare quality programs. Since 2009, NQF has been the sole organization to function under contract to CMS as the consensus-based entity as described by the provisions of sections1890 and 1890A of the Social Security Act (SSA). The consensus-based entity manages the Measure Applications Partnership, which is a formal process for obtaining stakeholder input on proposed new measures for Medicare quality programs, along with other measure endorsement and maintenance activities. CMS also relies on other contractors to conduct analyses or disseminate information related to the development and use of quality measures in its Medicare quality programs. CMS Quality Measurement Strategic Objectives CMS has established strategic objectives for the measures CMS develops or uses in its Medicare quality programs. CMS’s quality measurement strategic objectives have evolved over the last decade as CMS has expanded Medicare quality programs and has collaborated with other organizations that use or develop quality measures, such as private insurance companies. In 2017 CMS announced a revised version of these objectives in its Meaningful Measures Initiative. These eight quality measurement strategic objectives are for CMS to adopt measures that are patient-centered and meaningful to patients, clinicians, and address high-impact measure areas that safeguard public health, are outcome-based where possible, fulfill each program’s statutory requirements, minimize burden for providers, create significant opportunity for improvement, address measure needs for population-based payment through alternative payment models, and align across programs and/or with other payers. In addition, to provide greater specificity for its objective to address high- impact measure areas that safeguard public health, CMS has designated 19 specific meaningful measure areas. See appendix I for the list of these meaningful measure areas and the six broad quality priority areas that they address. CMS Funding for Quality Measurement Activities CMS’s quality measurement activities are funded through the federal budget and appropriations process. Each appropriation includes language that describes an authorized purpose or purposes for which the funds may be used. Such language may specifically reference certain activities such as quality measurement or could refer to a broad purpose under which activities such as quality measurement may have been authorized. Available funds are first obligated—that is, committed to a specific purpose—and then expended when an actual payment is made. Expenditures can occur one or more fiscal years after the obligation was incurred. Funds that are available in a given fiscal year but not obligated during that year are known as unobligated balances. Unobligated balances can be carried over to the next fiscal year, unless their availability expires under the terms of their appropriation. Most CMS funding that is explicitly appropriated for quality measurement activities is available indefinitely, until obligated and expended. CMS Lacks Complete Information on Its Quality Measurement Funding and on How It Uses Funding to Achieve Its Strategic Objectives CMS maintains information in its core budget database on the amount of funding for its quality measurement activities, such as when funding for that purpose is specifically authorized by appropriations. However, CMS’s database does not capture all of the funding the agency has obligated that pays for quality measurement activities or the extent to which this funding has supported CMS’s quality measurement strategic objectives. Our review of CMS’s quality measurement funding information also shows that CMS maintains a substantial amount of unobligated balances—funding that CMS has not yet used and remains available—for quality measurement activities. CMS Maintains Information on Funding for Some Quality Measurement Activities in CMS’s Core Budget Database CMS officials report that the agency records funding information for its quality measurement activities in its core budget database, HIGLAS. CMS has information on quality measurement funding primarily when the appropriation is specifically authorized for that purpose. CMS officials identified eight appropriations that specifically designate funding for Medicare quality measurement activities over the 10-year period we reviewed (fiscal years 2009-2018). These include five appropriations that have funded the consensus-based entity established under sections 1890 and 1890A of the SSA, to carry out various activities under contract with CMS in accordance with those provisions. CMS officials identified another three appropriations that focused on more discrete aspects of quality measurement, such as developing new quality measures for clinicians. From fiscal years 2009 through 2018, a total of $429.9 million was authorized for these eight appropriations (see table 1). In addition, CMS officials identified some funding used—that is, obligated—for quality measurement activities, from appropriations authorized for more general purposes. They obtained information on such usage from HIGLAS based on the presence of labels, such as “quality measure development,” in the project code and project description data fields in HIGLAS. According to CMS officials, these data fields provide the most detailed categorization of activities in HIGLAS. Table 2 shows the specific project codes and project descriptions used in HIGLAS to characterize use of quality measurement funding in fiscal year 2018. These obligations are from both appropriations that specifically authorize quality measurement activities and also from general appropriations whose authorized purposes do not specifically mention quality measurement activities. As shown in table 2, the project codes and their descriptions used in HIGLAS provide high-level information that largely matches the information known from the appropriation authorizing such use. CMS Lacks Information on the Total Amount of Quality Measurement Funding and the Extent to Which This Funding Supports Its Strategic Objectives Our review of the funding information in HIGLAS found that the data do not capture the total amount of funding CMS has obligated that pays for quality measurement activities. As we have noted, CMS officials identified funding obligated for quality measurement activities in HIGLAS either because 1) the funding came from appropriations specifically designated for quality measurement purposes, or 2) the funding came from appropriations for more general purposes but had specific HIGLAS project codes to identify its use for quality measurement activities. However, CMS officials told us that they thought there were additional quality measurement activities funded from appropriations for general purposes that could not be identified by project codes in HIGLAS. As a result, they could not determine from HIGLAS what amount of these funds paid for quality measurement activities as opposed to other activities. CMS officials stated that while they do not have information on the amount of this unidentified quality measurement funding, they estimated that it was less than the amount of quality measurement funding identified in HIGLAS. Furthermore, CMS’s funding information in HIGLAS also is not sufficiently detailed to show the extent to which the funding was used for activities that support CMS’s eight quality measurement strategic objectives. While some HIGLAS project descriptions—like “Hospital Outcome Measures”— correspond with one of these objectives, as shown in table 2 most do not. In addition, the documents that CMS uses to plan and monitor spending for quality measurement activities generally do not include information showing how much funding CMS has obligated for activities related to CMS’s quality measurement strategic objectives. CMS officials stated that they considered it unduly burdensome to attempt to use HIGLAS to track quality measurement funding according to their strategic objectives. First, they said that quality measurement activities overall constitute a small portion of the funding recorded in HIGLAS. In addition, officials noted that CMS’s strategic objectives change over time. Finally, CMS officials stated their belief that all of CMS’s quality measure activities help to address the agency’s objectives. As a result, CMS cannot determine how its specific funding for quality measurement activities addresses each of its quality measurement strategic objectives and how possible changes in its funding allocations among those activities could help to promote its objectives more effectively. Federal standards for internal control call for agencies to use complete and accurate information and to identify types or categories of information that enable the agency to achieve its objectives. Without more complete information on the total amount of funding obligated to quality measurement activities, CMS officials cannot accurately assess the magnitude of resources they have provided for quality measurement. In addition, even if CMS quality measure activities generally address one or another of its strategic objectives, having information on the extent of funding for each quality measurement strategic objective could help CMS officials assess the amount of funding each of the agency’s priorities is receiving. Doing so would enable CMS officials to make adjustments in accordance with their objectives. While collecting more complete and detailed information on funding for quality measurement activities in HIGLAS—or using some other method that CMS determines is feasible—would require additional effort, CMS could realize corresponding benefits. CMS officials told us that at present, when they need to obtain a higher level of detail about funding for quality measurement activities, they do not use HIGLAS and instead typically conduct a manual review of any available underlying documentation, such as documents related to individual contracts. For example, in order to respond to a statutory requirement to report on its spending to develop certain quality measures for physicians, CMS officials told us they needed to review a set of individual contracts associated with those measures. CMS officials noted that this process is often laborious and that the content of available documents may not enable them to obtain all the desired funding information for the specific quality measurement activities in question. Collecting more information routinely about funding for quality measurement activities has the potential to make such manual reviews of documents less necessary and burdensome. The limitations in CMS’s information on funding for quality measurement activities have implications for CMS’s ability to communicate information outside the agency. As required by the Congress, CMS issued its first annual report on quality measurement funding in March 2019. In this report, CMS itemized information on such funding into four broad categories: “Duties of the consensus-based entity,” “Dissemination of quality measures,” “Program assessment and review,” and “Program oversight and design.” CMS’s report listed a number of more specific activities within these categories without providing the amount of funding it allocated for each of the described activities. More detailed funding information could help the Congress to better understand how CMS is using appropriations for quality measurement, and could assist with effective oversight of these activities. Internal control standards call for agencies to consider the needs and expectations of external users, such as Congress, when collecting and communicating information. CMS’s Funding Information Shows Substantial Unobligated Balances in Its Quality Measurement Funding Our review of the funding information CMS provided determined that the agency has maintained substantial unobligated balances related to its quality measurement activities from fiscal years 2010 through 2018. Unobligated balances represent funding that CMS did not use in the year it was appropriated, and that remains available for use in future years. All but one of the eight appropriations that specifically authorize spending for quality measurement activities are available indefinitely. Five of these appropriations funded quality measurement activities under sections 1890 and 1890A of the SSA. In the case of these five appropriations, with the exception of fiscal year 2009, CMS had unobligated balances each year that were larger than or similar to the total amount the agency had obligated from those appropriations that year (see figure 1). Figure 1 also shows three other appropriations more narrowly focused on developing new measures for clinicians and post-acute care providers under Medicare (appropriated by MACRA section 102 and the IMPACT Act sections 2a and 2d). Since 2015, unobligated balances for these appropriations also generally exceeded annual obligations. See appendix II for more detailed information. CMS officials stated that unobligated balances reflect broader spending decisions—for quality measurement as well as other activities—the agency makes to meet its strategic objectives and any related legislative requirements. CMS officials said that in general, they chose to use the available quality measurement funds conservatively to ensure there were no gaps in funding to carry out their statutory responsibilities, in view of uncertainty about the availability and timing of funding in future years. They also said that they took into account the total amount of appropriated funds—including unobligated balances—in developing the scope and duration of quality measurement activities. The officials noted that it often takes more than one year to implement these activities, in order to gather information, select contractors, or solicit and award grant applications. Regarding the level of unobligated balances to be carried over from one fiscal year to the next, CMS officials told us that they work to obligate all appropriations in accordance with statutory requirements, and do not have thresholds for maximum unobligated balances. Maintaining large unobligated balances means that CMS is retaining funds for future quality measurement activities rather than using them for current quality measurement activities. One example of how such choices can affect the scope and timing of CMS’s quality measurement activities was the outcome of a CMS competition for cooperative agreements, announced in March 2018, to develop new clinician quality measures to address identified measurement gaps. Drawing on funds from the appropriation dedicated to developing, improving, updating, or expanding new clinician quality measures (MACRA 102) that were available for use until 2022, CMS set a maximum amount for the awards of $30 million over three years. CMS officials determined that the $30 million ceiling meant that there was adequate funding for seven awardees, while CMS indicated that additional applicants scored well on CMS’s selection criteria and addressed areas of need. For fiscal year 2018, MACRA 102 had an unobligated balance of $42 million, with an additional $15 million appropriation in place for fiscal year 2019. As of May 23, 2019, CMS officials told us that they had not announced new competitions to develop clinician quality measures. CMS Lacks Assurance That the Quality Measures It Decides to Use or Develop Effectively Promote Strategic Objectives CMS takes different approaches in deciding which Medicare quality measures to use in its programs, which to remove, and which new measures to develop. However, CMS lacks procedures to ensure that these decisions are consistent with its quality measurement strategic objectives, and CMS has not yet developed or implemented performance indicators to evaluate its overall progress toward achieving these objectives. CMS Takes Different Approaches in Deciding Which Quality Measures to Use and Develop For selecting measures to be used in its Medicare quality programs, CMS has an annual process, as defined by the Patient Protection and Affordable Care Act. CMS makes a number of decisions that influence measure selection throughout the process. Each year CMS asks measure developers to submit candidate quality measures to CMS for potential selection. CMS makes preliminary decisions on which of these measures to use in its quality programs, and it publishes this selection of measures in its annual Measures under Consideration list (MUC). The MUC list then undergoes public review by multiple stakeholders. After this review, CMS chooses which measures to include in the formal rulemaking processes that ultimately determine which measures are added to its quality programs. See table 3. To make decisions on which measures to include in the MUC list, CMS officials review the submissions. According to CMS, officials from each Medicare quality program, referred to as quality program leads, separately review each measure submitted for use in that program. CMS officials told us that as necessary, they consult with technical experts and with other CMS or Department of Health and Human Services (HHS) officials. According to CMS officials, the program leads make recommendations to higher level officials, such as division directors, on whether CMS should accept or reject each measure. CMS internal guidance outlines factors that, among other things, officials should consider. Some of these factors reflect the strategic objectives laid out in the Meaningful Measures Initiative, and the guidance also indicates that officials may consider additional factors in their decision-making. CMS officials told us that, when making measure selection decisions, program teams are given the flexibility to develop criteria that best suits their programs’ needs, noting that some programs are intended to address a broad range of areas, such as the Inpatient Quality Reporting Program, while others have a more limited focus, such as the Hospital Readmissions Reduction Program. CMS officials told us that the director of CMS’s Center for Clinical Standards and Quality, which is responsible for quality measurement, makes the final measure selection decisions and, in doing so, generally accepts the recommendations of the program teams. Our analysis of CMS’s quality measures indicates that the number of candidate quality measures submitted to CMS for the MUC list has decreased from 335 measures in 2014 to 67 in 2018. CMS officials told us the decline in the number of candidate measures submitted reflected CMS efforts to more clearly define a targeted set of quality measurement priorities for measure developers and to reduce provider reporting burden. Minimizing provider burden is one of CMS’s strategic objectives, and, according to CMS officials, it represents a priority communicated by the CMS administrator. For more information about CMS’s measure selection decisions for its annual MUC list in 2014 through 2018, see appendix III. CMS officials also make decisions annually about which existing measures CMS will remove from its Medicare quality programs. According to CMS officials, the process for deciding which measures to remove is an ongoing, iterative process, and discussions on which measures to remove generally occur in parallel with discussions for selecting measures, with discussions on both measure selection and removal coming to a conclusion in the drafting of the annual proposed and final rules for each program. For measures that are being used in its quality programs, CMS relies on measure developers to monitor the performance of their measures based on principles defined in CMS’s Blueprint. According to the Blueprint, information from developers’ monitoring efforts, including recommendations from technical experts, should be conveyed to and evaluated by CMS officials. CMS officials told us that their decisions to remove measures often take into account the recommendations made by technical experts. In addition, CMS has promulgated through federal rulemaking eight factors for determining whether to remove existing measures from its Medicare quality programs, some of which reflect its quality measurement strategic objectives. CMS officials also said that in deciding to remove measures from CMS quality programs in 2018 they, in part, considered an assessment of the costs of reporting measures relative to the benefit of continued use of the measures. CMS decisions to remove measures have been included in notices of proposed rulemaking in the Federal Register, which allows for public comment and further consideration before issuance of final rules to that effect. In addition to making decisions on the selection and removal of measures, CMS officials also make decisions regarding which new measures to develop. Our review of CMS contract documents, including task orders, indicates that CMS typically awards multiple year contracts to conduct ongoing assessments of quality measures and to develop measures for specific Medicare quality programs, such as inpatient psychiatric facilities or post-acute care providers. Those task orders often call on contractors to convene technical expert panels and conduct additional analyses to assess what measures are currently available for use and what gaps exist in available measures. CMS officials told us they review these reports and provide informal feedback to the contractors. CMS also establishes parameters that guide these efforts. For example, in its 2016 Measure Development Plan for Medicare’s new physician payment system, after soliciting public input, CMS designated six medical specialty areas in which to focus its measure development efforts, and subsequently added five more specialties on which to focus the work of its contractors. For more information about outside entities that perform quality measurement activities under contract with CMS and the efforts CMS has taken to coordinate these activities across its contractors, see appendix IV. CMS Lacks Procedures for Systematically Assessing Whether the Measures It Decides to Develop and Use Address Its Strategic Objectives CMS has taken some steps that provide opportunities for CMS officials to consider how quality measures may help address the agency’s quality measurement strategic objectives. CMS officials said that in 2018 they began using the Measure Review Template, a spreadsheet used to consolidate information on quality measures submitted to CMS by measure developers. CMS officials told us that they use the spreadsheet to inform their discussions, such as by considering how measures are distributed across the 19 meaningful measure high-impact areas. CMS is also developing another tool, the Quality Measure Index, that is intended to provide a standard methodology to score measures on dimensions that include several of CMS’s eight quality measurement strategic objectives. In addition, CMS officials told us that on occasion they have made limited assessments across measures concerning specific strategic objectives. CMS officials told us that these limited assessments across measures are generally performed when a measure submitted for use in its Medicare quality programs is closely related to another measure, which affects the CMS objective to increase measure alignment. In addition, they said they have identified a few indicators that they use to continuously assess their decision-making process, such as the percentage of outcome measures. CMS also documents some information about its quality measurement decisions. For example, the agency announces its final selection of quality measures to be added to and removed from its Medicare quality programs in the annual federal proposed and final rules for each of those programs. The rationale for selecting each measure is provided as a summary of the peer-reviewed evidence of the impact that use of the measure will have on clinical care. In addition, CMS maintains an internal tracking system, which assembles the information that measure developers provide about the measures they submit to CMS. This system includes some information related to CMS’s quality measurement strategic objectives, such as the meaningful measures high-impact area the measure is intended to address. While these steps provide some information about the linkages between certain quality measures and some of CMS’s quality measurement strategic objectives, CMS lacks procedures to ensure systematic assessment of each quality measure against each of its eight quality measurement strategic objectives. For example, while CMS has implemented the Measure Review Template to consolidate some information on measures, the template does not provide procedures for systematically assessing how each measure will help CMS achieve all eight of its quality measurement strategic objectives. The Quality Measure Index—currently under development—has the potential to be used in a systematic assessment of each measure, but according to CMS officials, as of March 2019 the agency had not yet determined how it planned to use this tool once its testing was complete. Furthermore, CMS lacks procedures to ensure a systematic assessment of whether the collective set of measures it decides to develop or use will help CMS achieve each of the objectives, which could help determine the extent to which each of the objectives is being effectively addressed. The limited assessments across measures that CMS officials said they perform do not consider whether each of CMS’s objectives is being addressed. For example, one of CMS’s eight quality measurement strategic objectives directs CMS to address 19 high-impact measure areas. CMS officials told us that, for each quality program, they look at whether measures generally address the high-impact measure areas, but gaps in these areas remain to be filled. In 2018, there were no measures used in CMS quality programs that addressed the high-impact area “equity of care” and 13 of 17 Medicare quality programs had no measures that addressed the “community engagement” area. Measure developers did not submit measures to CMS that addressed these areas, and CMS did not identify specific initiatives to address them. CMS officials told us, however, that CMS supports discussions of key methodological considerations for collecting and analyzing measure data that could help enable future development of these measures. Last, CMS lacks procedures for documenting the consistent application of those systematic assessments. Federal internal control standards indicate the importance of documenting decisions to support achieving agency objectives. Specifically, CMS does not document, either in its public reporting or internal tracking system, how each measure it decides to use is expected to promote each of its eight quality measurement strategic objectives. For decisions on developing new measures, the agency records less information. For example, CMS does not maintain a consolidated list of decisions to initiate the development of new quality measures across the various Medicare quality programs. CMS officials also told us that they generally do not maintain documentation of discussions on how or why they selected one measure for development over another. If CMS develops procedures to consider the effect of each of its quality measurement decisions on each of its quality measurement strategic objectives, then documentation of these procedures would help to show that they are implemented consistently. Federal standards for internal control state that management should design and implement internal control activities, such as tools and documentation of decisions, to support the agency in achieving its objectives. Without procedures that ensure that its quality measures fully address its strategic objectives, CMS increases the risk that the measures it decides to develop and use will not help the agency achieve its quality measurement strategic objectives as effectively as possible. CMS Has Not Established Performance Indicators to Determine Its Overall Progress in Achieving Its Quality Measurement Strategic Objectives CMS has not developed and implemented performance indicators that would be needed to determine if it is making progress in meeting its quality measurement strategic objectives. Establishing these indicators and using them to evaluate its progress towards meeting each of its quality measurement strategic objectives would enable CMS to determine whether its quality measurement efforts are sufficient or whether changes in these efforts are needed. According to federal internal control standards, after agencies establish objectives, they should establish a set of performance indicators and use them to assess their effectiveness in achieving their objectives and identify improvements in their work, as needed. However, CMS has not established performance indicators for its strategic objectives that would provide a basis for determining its progress towards achieving these objectives. Such performance indicators would relate to each of CMS’s quality measurement strategic objectives and provide information on interim progress toward achieving these objectives. For example, CMS could establish one or more indicators of its progress toward addressing the 19 high-impact measure areas that safeguard public health, and an indicator of providers’ reporting burden for quality measurement to see if it showed an overall reduction. CMS officials told us that they assess the impact of the agency’s quality measurement activities by reviewing changes over time in health care providers’ reported performance on selected quality measures. However, these measures are for providers’ quality of care, and are not indicators designed to determine the agency’s progress in achieving its eight strategic objectives for quality measurement. Specifically, CMS has completed the National Impact Assessment of Quality Measures report every 3 years since 2012. These reports focus on trends in the performance of health care providers on a number of specific quality measures. Such analyses do not evaluate CMS’s performance in developing and choosing to use measures that promote its quality measurement strategic objectives. CMS has convened the Meaningful Measurement and Improvement Affinity Group, a workgroup of CMS officials involved in quality measurement. This workgroup’s stated mission is to champion the Meaningful Measures Initiative and facilitate its implementation across the agency. CMS officials told us that the workgroup has begun to discuss potential ways to evaluate the agency’s progress in achieving the eight strategic objectives laid out in the Meaningful Measures Initiative. However, the information CMS officials provided on the workgroup’s activities, as of March 2019, indicated that the group had not yet determined how to gauge such progress, such as by establishing performance indicators. Conclusions CMS plays a leading role in the process of developing new quality measures and selecting measures for use in its various quality programs in Medicare. These programs in turn affect the quality of care the program’s beneficiaries receive. However, CMS lacks complete information on the amount of resources it has obligated for its quality measurement activities and how its allocation of those resources relates to its quality measurement strategic objectives. The agency also lacks procedures to ensure that the decisions it makes to develop and use measures for its quality programs are consistent with those objectives. Finally, CMS has not developed and implemented performance indicators to evaluate its progress towards achieving these objectives. Taken together, these issues limit CMS’s ability to determine whether its allocation of resources and quality measurement decisions are optimal or whether changes are needed in its approach. Recommendations for Executive Action We are making the following three recommendations to CMS: The Administrator of CMS should, to the extent feasible, maintain more complete information on both the total amount of funding allocated for quality measurement activities and the extent to which this funding supports each of its quality measurement strategic objectives. (Recommendation 1) The Administrator of CMS should develop and implement procedures to systematically assess the measures it is considering developing, using, or removing in terms of their impact on achieving CMS’s strategic objectives and document its compliance with those procedures. (Recommendation 2) The Administrator of CMS should develop and use a set of performance indicators to evaluate the agency’s progress towards achieving its quality measurement strategic objectives. (Recommendation 3) Agency Comments We provided a draft of this report to HHS for review and comment. In its written comments, which are reproduced in appendix V, HHS concurred with our recommendations. Regarding our first recommendation, HHS stated that it has undertaken a review of its fiscal accountability processes for its quality improvement activities and is implementing more granular tracking of funding specific to quality measurement to the extent it is feasible. Regarding our second recommendation, HHS stated that it will determine what steps may be needed to further document how its measure decisions impact the achievement of CMS’s quality measurement strategic objectives. HHS’s comments did not address the need to develop and implement procedures for systematically assessing measures against the strategic objectives, as we recommended. Regarding our third recommendation, HHS stated it would consider how best to evaluate its progress in meeting its quality measurement strategic objectives. In addition, HHS provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Health and Human Services, the Administrator of the Centers for Medicare & Medicaid Services, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff members have any questions about this report, please contact me at (202) 512-7114 or farbj@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI. Appendix I: CMS Quality Priorities and Meaningful Measure Areas As part of its Meaningful Measures Initiative, the Centers for Medicare & Medicaid Services (CMS) identified 19 meaningful measure areas to specify its priorities under its quality measurement strategic objective to address high-impact measure areas that safeguard public health. The 19 areas are linked to six broader health care quality priorities previously identified in the 2011 National Strategy for Quality Improvement in Health Care. See table 4. Appendix II: CMS Appropriations for Medicare Quality Measurement Activities, Fiscal Years 2009-2018 The Centers for Medicare & Medicaid Services (CMS) has identified five separate appropriations that for various fiscal years have funded the activities assigned to the consensus-based entity (currently the National Quality Forum), along with certain other quality measurement activities, as described in sections 1890 and 1890A of the Social Security Act. See table 5. Three additional appropriations focus on specific Medicare quality measurement activities, such as post-acute care measures. See table 6. Appendix III: Description of Quality Measures CMS Selected for Its Annual Measures under Consideration List, 2014-2018 Tables 7 to 12 below present descriptive information that the Centers for Medicare & Medicaid Services (CMS) collects through its issue tracking system on the measures submitted to CMS by measures developers for potential use in CMS’s Medicare quality programs. Appendix IV: CMS-Contracted Organizations That Perform Quality Measurement Activities and Efforts to Encourage Coordination The Centers for Medicare & Medicaid Services (CMS) has used the majority of its Medicare quality measurement funding for activities conducted by outside organizations under contract with CMS. Between fiscal years 2009 through 2018, the amount of obligations to contracted organizations increased from $10 million to nearly $55 million. See table 13. The total amount of funds obligated to each contractor in fiscal years 2009 through 2018 to perform Medicare quality measurement activities varied, ranging from $1,000 to $139,397,410. For fiscal years 2009 through 2018, 91 percent of funds obligated to contracted organizations for Medicare quality measurement activities went to 12 of 59 contracted organizations. See table 14. CMS has undertaken efforts to coordinate the Medicare quality measurement activities performed by its contractors. For example, CMS works with a CMS contractor, Battelle, to facilitate monthly webinars with its Measure & Instrument Development and Support (MIDS) contractors. The purpose of the webinars is to provide contractors with a forum to discuss each other’s quality measurement activities and to exchange ideas. For more information about CMS’s formal efforts to coordinate the quality measurement activities of its contractors, see table 15. Appendix V: Comments from the Department of Health and Human Services Appendix VI: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Will Simerl, Assistant Director; Eric Peterson, Analyst-in-Charge, Jonathan Adams, George Bogart, Krister Friday, Cathy Hamann, Katie Mack, and Dan Ries made key contributions to this report. Also contributing were Vikki Porter and Ethiene Salgado-Rodriguez. Related GAO Products Health Care Quality: HHS Should Set Priorities and Comprehensively Plan Its Efforts to Better Align Health Quality Measures. GAO-17-5 (Washington, D.C.: October 13, 2016). Patient Protection and Affordable Care Act: Procedures for Reporting Certain Financial Management Information Should Be Improved. GAO-14-697 (Washington, D.C.: September 22, 2014). Budget Issues: Key Questions to Consider When Evaluating Balances in Federal Accounts. GAO-13-798 (Washington, D.C.: September 30, 2013). Health Care Quality Measurement: HHS Should Address Contractor Performance and Plan for Needed Measures. GAO-12-136 (Washington, D.C.: January 13, 2012). Program Evaluation: Improving the Flow of Information to the Congress. GAO/PEMD-95-1 (Washington, D.C.: January 30, 1995).


Introduction In 2018, approximately 38.1 million people, or 11.8% of the population, had incomes below the official definition of poverty in the United States. The poverty rate (the percentage that were in poverty) fell from 12.3% in 2017, while the number of persons in poverty declined from 39.6 million. In this report, the numbers and percentages of those in poverty are based on the Census Bureau's estimates. While this official measure is often regarded as a statistical yardstick rather than a complete description of what people and families need to live, it does offer a measure of economic hardship faced by the low-income population: the poverty measure compares family income against a dollar amount called a poverty threshold , a level below which the family is considered to be poor. The Census Bureau releases these poverty estimates every September for the prior calendar year. Most of the comparisons discussed in this report are year-to-year comparisons. This report only considers a number or percentage to have changed from the previous year, or to be different from another number or percentage, if the difference has been tested to be statistically significant at the 90% confidence level. However, in addition to the most recent year's data, this report presents a historical perspective as well as information on poverty for demographic groups (by family structure, age, race and Hispanic origin, and work status) and by state. Over the past several decades, criticisms of the official poverty measure have led to the development of an alternative research measure called the Supplemental Poverty Measure (SPM), which the Census Bureau also computes and releases. Statistics comparing the official measure with the SPM are provided at the conclusion of this brief. The SPM includes the effects of taxes and in-kind benefits (such as housing, energy, and food assistance) on poverty, while the official measure does not. Because some types of tax credits are used to assist the poor, as are other forms of assistance, the SPM may be of interest to policymakers. However, the official measure provides a comparison of the poor population over a longer time period, including some years before many current antipoverty assistance programs had been developed. In developing poverty-related legislation and conducting oversight on programs that aid the low-income population, policymakers may be interested in these historical trends. How the Official Poverty Measure Is Computed The Census Bureau determines a person's poverty status by comparing his or her resources against a measure of need. For the official measure, resources is defined as total family income before taxes, and the measure of "need" is a dollar amount called a poverty threshold. There are 48 poverty thresholds that vary by family size and composition. If a person lives with other people to whom he or she is related by birth, marriage, or adoption, the money income from all family members is used to determine his or her poverty status. If a person does not live with any family members, his or her own income is used. Only money income before taxes is used in calculating the official poverty measure, meaning this measure does not treat in-kind benefits such as the Supplemental Nutritional Assistance Program (SNAP, formerly known as food stamps), housing subsidies, or employer-provided benefits as income. The poverty threshold dollar amounts vary by the size of the family (from one person not living in a family, to nine or more family members living together) and the ages of the family members (how many of the members are children under 18 and whether or not the family head is 65 years of age or older). Collectively, these poverty thresholds are often referred to as the poverty line . As a rough guide, the poverty line in 2018 can be thought of as $25,701 for a family of four, $19,985 for a family of three, $16,247 for a family of two, or $12,784 for an individual not living in a family, though the official measure is actually much more detailed. The threshold dollar amounts are updated annually for inflation using the Consumer Price Index. Notably, the same thresholds are applied throughout the country: no adjustment is made for geographic variations in living expenses. The official poverty measure used in this report is the federal government's definition of poverty for statistical purposes, such as comparing the number or percentage of people in poverty over time. A related definition of poverty, the poverty guidelines published by the Department of Health and Human Services (HHS), is used for administrative purposes such as eligibility criteria for assistance programs and will not be discussed in this report. Historical Perspective Figure 1 shows a historical perspective of the number and percentage of the population below the poverty line. The number in poverty and the poverty rates are shown from the earliest year available (1959), through the most recent year available (2018). Because the total U.S. population has grown over time, poverty rates are useful for historical comparisons because they control for population growth. Poverty rates fell through the 1960s. Since then, they have generally risen and fallen according to the economic cycle, though during the most recent two expansions poverty rates did not fall measurably until four to six years into the expansion. Historically notable lows occurred in 1973 (11.1%) and 2000 (11.3) . Poverty rate peaks occurred in 1983 (15.2%), 1993 (15.1%), and 2010 (15.1%). Poverty rates tend to rise during and after recessions, as opposed to leading economic indicators such as new housing construction, whose changes often precede changes in the performance of the overall economy. The poverty rate's lag is explainable in part by the way it is measured: it uses income from the entire calendar year. Notably, the poverty rate in 2018 registered a fourth consecutive annual decrease since the most recent recession, though it remained higher than the rate in 2000, the most recent low point. Poverty for Demographic Groups10 The drop in the U.S. poverty rate (from 12.3% in 2017 to 11.8% in 2018) affected some demographic groups more than others, notably people in female-householder families, children and the population aged 18 to 64, and the non-Hispanic white population. Details for selected demographic groups are described below. Family Structure Because poverty status is determined at the family level by comparing resources against a measure of need, vulnerability to poverty may differ among families of different compositions. In this section, poverty data by family structure are presented using the official poverty measure, with "families" defined as persons related by birth, marriage, or adoption to the householder (the person in whose name the home is owned or rented). In the " Supplemental Poverty Measure " section of this report, a different definition will be used. Families with a female householder and no spouse present (female-householder families) have historically had higher poverty rates than both married-couple families and families with a male householder and no spouse present (male-householder families). This remained true in 2018: female-householder families experienced a poverty rate of 24.9%, compared with 4.7% for married-couple families and 12.7% for male-householder families. Unlike the other two family types, however, female-householder families experienced a decline in their poverty rate of 1.3%. Their 2018 rate of 24.9%, down from 26.2% in 2017, appeared to be among the lowest poverty rates for female-householder families on record. Among individuals not living in families, the poverty rate was 20.2% in 2018, not distinguishable from the previous year. Age When examining poverty by age, three main groups are noteworthy for distinct reasons: under 18, 18 to 64, and 65 and older. People under age 18 are typically dependent on other family members for income, particularly young children below their state's legal working age. People aged 18 to 64 are generally thought of as the working-age population and typically have wages and salaries as their greatest source of income. People age 65 and older, referred to as the aged population, are often eligible for retirement, and those who do retire typically experience a change in their primary source of income. Children and the working-age population experienced decreases in poverty. Among children, 11.9 million, or 16.2%, were poor, down from 12.8 million or 17.4% in 2017. Among the working-age population, 21.1 million, or 10.7%, were in poverty, down from 21.9 million or 11.1% in 2017. The aged population did not register any significant changes in its number in poverty or its poverty rate from 2017 to 2018: 5.1 million, or 9.7%, were poor. From a historical standpoint, the poverty rate for those age 65 and over used to be the highest of the three groups. In 1966, people age 65 and over had a poverty rate of 28.5%, compared with 17.6% for those under 18 and 10.5% for working-age adults. By 1974, the poverty rate for people age 65 and over had fallen to 14.6%, compared with 15.4% for people under 18 and 8.3% for working-age adults. Since then, people under 18 have had the highest poverty rate of the three age groups, as shown in Figure 3 . Race and Hispanic Origin13 Poverty rates vary by race and Hispanic origin, as shown in Figure 4 . In surveys, Hispanic origin is asked separately from race; accordingly, people identifying as Hispanic may be of any race. The poverty rate fell for non-Hispanic whites (from 8.5% in 2017 to 8.1% in 2018). Among blacks (20.8%), Asians (10.1%), and Hispanics (17.6%), the poverty rate did not register any statistically significant change from 2017. Work Status While having a job reduced the likelihood of being in poverty, it did not guarantee that a person or his or her family would avoid poverty. Among the 18 to 64-year-old population living in poverty, 77.2% had jobs in 2018. Poverty rates among workers in this age group were 5.1% for all workers, 2.3% for full-time year-round workers, and 12.7% for part-time or part-year workers, none of which were measurably changed from the previous year. Similarly, no significant change was detected among those who did not work at least one week in 2018 (29.7% were poor). Because poverty is a family-based measure, the change in one member's work status can affect the poverty status of his or her entire family. Among all 18 to 64-year-olds who did not have jobs in 2018, 58.9% lived in families in which someone else did have a job. Among poor 18 to 64-year-olds without jobs, 18.5% lived in families where someone else worked. Poverty Rates by State19 Poverty is not equally prevalent in all parts of the country. The map in Figure 5 shows states with relatively high poverty rates across parts of the Appalachians, the Deep South, and the Southwest, with the poverty rate in Mississippi (19.7%) among the highest in the nation, not statistically different from the rate in New Mexico (19.5%). The poverty rate in New Hampshire (7.6%) was lowest. When comparing poverty rates geographically, it is important to remember that the official poverty thresholds are not adjusted for geographic variations in the cost of livingâthe same thresholds are used nationwide. As such, an area with a lower cost of living accompanied by lower wages will appear to have a higher poverty rate than an area with a higher cost of living and higher wages, even if individuals' purchasing power were exactly the same in both areas. Puerto Rico and 14 states experienced poverty rate declines from 2017 to 2018: one in the Midwest (Illinois), three in the Northeast (Massachusetts, New Jersey, and New York); five in the South (Florida, Georgia, Louisiana, North Carolina, and West Virginia); and five in the West (Arizona, California, Colorado, Oregon, and Washington). Connecticut was the only state to experience an increase, and 35 states, as well as the District of Columbia, did not register a statistically significant change. Supplemental Poverty Measure Criticisms of the official measure have led to the development of the Supplemental Poverty Measure (SPM). Described below are the development of the official measure, its limitations, attempts to remedy those limitations, the research efforts that eventually led to the SPM's first release in November 2011, and a comparison of poverty rates in 2018 based on the SPM and the official measure. How the Official Poverty Measure Was Developed The poverty thresholds were originally developed in the early 1960s by Mollie Orshansky of the Social Security Administration. Rather than attempt to compute a family budget by using prices for all essential items that low-income families need to live, Orshansky focused on food costs. Unlike other goods and services such as housing or transportation, which did not have a generally agreed-upon level of adequacy, minimum standards for nutrition were known and widely accepted. According to a 1955 U.S. Department of Agriculture (USDA) food consumption survey, the average amount of their income that families spent on food was roughly one-third. Therefore, using the cost of a minimum food budget and multiplying that figure by three yielded a figure for total family income. That computation was possible because USDA had already published recommended food budgets as a way to address the nutritional needs of families experiencing economic stress. Some additional adjustments were made to derive poverty thresholds for two-person families and individuals not living in families to reflect the relatively higher fixed costs of smaller households. Motivation for a Supplemental Measure While the official poverty measure has been used for over 50 years as the source of official statistics on poverty in the United States, it has received criticism over the years for several reasons. First, it does not take into account benefits from most of the largest programs that aid the low-income population. For instance, it uses money income before taxesâmeaning that it does not necessarily measure the income available for individuals to spend, which for most people is after-tax income. Therefore, any effects of tax credits designed to assist persons with low income are not captured by the official measure. The focus on money income also does not account for in-kind benefit programs designed to help the poor, such as SNAP or housing assistance. The official measure has also been criticized for the way it characterizes families' and individuals' needs in the poverty thresholds. That is, the method used to compute the dollar amounts used in the thresholds, which were originally based on food expenditures in the 1950s and food costs in the 1960s, does not accurately reflect current needs and available goods and services. Moreover, the official measure does not take account of the sharing of expenses and income among household members not related by birth, marriage, or adoption. And, as mentioned earlier, the official thresholds do not take account of geographic variations in the cost of living. In 1995, a panel from the National Academy of Sciences issued a report, Measuring Poverty: A New Approach, which recommended improvements to the poverty measure. Among the suggested improvements were to have the poverty thresholds reflect the costs of food, clothing, shelter, utilities, and a little bit extra to allow for miscellaneous needs; to broaden the definition of "family;" to include geographic adjustments as part of the measure's computation; to include the out-of-pocket costs of medical expenses in the measure's computation; and to subtract work-related expenses from income. An overarching goal of the recommendations was to make the poverty measure more closely aligned with the real-life needs and available resources of the low-income population, as well as the changes that have taken place over time in their circumstances, owing to changes in the nation's economy, society, and public policies (see Table 1 ). After over a decade and a half of research to implement and refine the methodology suggested by the panel, conducted both from within the Census Bureau as well as from other federal agencies and the academic community, the Census Bureau issued the first report using the Supplemental Poverty Measure (SPM) in November 2011. Official and Supplemental Poverty Findings for 201827 Compared with the official measure, the SPM takes into account greater detail of individuals' and families' living arrangements and provides a more up-to-date accounting of the costs and resources available to them. Because the SPM recognizes greater detail in relationships among household members and geographically adjusts housing costs, it provides an updated rendering, compared with the official measure, of the circumstances in which the poor live. In that context, some point out that the SPM's measurement of taxes, transfers, and expenses may offer policymakers a clearer view of how government policies affect the poor population today. However, the SPM was developed as a research measure, and the Office of Management and Budget set the expectation that it would be revised periodically to incorporate improved measurement methods and newer sources of data as they became available; it was not developed for administrative purposes. Conversely, the official measure's consistency over a longer time span makes it easier for policymakers and researchers to make historical comparisons. Under the SPM, the profile of the poverty population is slightly different than under the official measure. The SPM was 1 percentage point higher in 2018 than the official poverty rate (12.8% compared with 11.8%; see Figure 6 ). More people aged 18 to 64 are in poverty under the SPM (12.2% compared with 10.7% under the 2018 official measure), as are people age 65 and over (13.6%, compared with 9.7% under the official measure). The poverty rate for people under age 18 was lower under the SPM (13.7% in 2018) than under the official measure (16.2%, with foster children included). Again, the SPM uses a different definition of resources than the official measure: the SPM includes in-kind benefits which generally help families with children; subtracts out work-related expenses, which are often incurred by the working-age population; and subtracts medical out-of-pocket expenses, which are incurred frequently by people age 65 and older. With the geographically adjusted thresholds, the poverty rate in 2018 was lower under the SPM than under the official measure for the Midwest (9.2% compared with 10.4%), while it was higher than the official measure for the Northeast (12.2% compared with 10.3%), the West (14.4% compared with 11.2%), and the South (13.9% compared with 13.6%).