Background Organizational Roles and Responsibilities for Managing the MHS The MHS is a complex organization in which responsibility for health care delivery is shared among the military departments—the Army, the Navy, and the Air Force—and the Defense Health Agency (DHA), with oversight from the Office of the Secretary of Defense and advice from the Joint Staff. As such, several leaders have responsibility for DOD’s medical workforces, their readiness, and the MTFs to which many of them are assigned. Specifically: The Under Secretary of Defense for Personnel and Readiness is the principal staff assistant and advisor to the Secretary and Deputy Secretary of Defense for health-related matters and, in that capacity, develops policies, plans, and programs for health and medical affairs. The Secretaries of each military department are responsible for organizing, training, and equipping military forces as directed by the Secretary of Defense as well as responsibilities related to ensuring the readiness of military personnel, and providing military personnel and other authorized resources in support of the combatant commanders and the DHA. The Surgeon General of each respective military department serves as the principal advisor to the Secretary of the military department concerned on all health and medical matters of the military department. The Assistant Secretary of Defense for Health Affairs (ASD(HA)) serves as the principal advisor for all DOD health-related policies, programs, and activities. He or she has the authority to develop policies, conduct analyses, provide advice, and make recommendations to the Secretary of Defense and others; issue guidance; and provide oversight on matters pertaining to the MHS. Further, the ASD(HA) prepares and submits a DOD unified medical program budget which includes, among other things, the Defense Health Program budget to provide resources for MTFs and the TRICARE Health Program. The Director of the DHA manages, among other things, the execution of policies issued by the ASD(HA) and manages and executes the Defense Health Program appropriation. The Director of the DHA is also responsible for the TRICARE Health Program. In December 2016, Congress expanded the role of the DHA by directing the transfer of responsibility for the administration of each MTF from the military departments to the DHA. By no later than September 30, 2021, the Director of the DHA will be responsible for the administration of each MTF. Specifically, the Director of the DHA will be responsible for budgetary matters, information technology, health care administration and management, administrative policy and procedure, and military medical construction, among other things. As of October 2019, the DHA had assumed administration and management responsibilities for all MTFs within the United States. MHS Workforces and the Role of MTFs in Supporting Military Readiness In fiscal year 2019, DOD’s Defense Health Program-funded workforce numbered over 174,000 personnel, comprising active-duty servicemembers from each military department (the Army, the Navy, and the Air Force), federal civilian employees of DOD, and private-sector contractors. These personnel included health-care providers, such as physicians (both primary and specialty care providers), nurses, and enlisted specialists who assist with medical procedures, and administrative and support personnel. MTFs vary in size and capabilities from small clinics, to ambulatory surgery centers, hospitals, and medical centers. Clinics generally provide primary-care services, which may include pediatrics at some locations. Other health-care services at clinics range from urgent care, women’s health, occupational health, and behavioral health, to orthopedics and other specialty services depending on location and population demand, according to MHS officials. Some clinics treat only active-duty servicemembers. Other clinics, along with hospitals and medical centers, also serve other eligible beneficiaries, including military family members, retirees, and some civilian employees of DOD. DOD’s hospitals provide emergency medicine, inpatient care and other specialty care services depending on population demand, according to MHS officials. For example, they generally offer surgical capabilities and labor and delivery services. According to DOD Instruction 6000.19, the primary purpose of MTFs is to support the readiness of the military services. In addition, the guidance states that the size, type, and location of MTFs must further this readiness objective. Further, each MTF must spend most of its resources supporting wartime skills development and maintenance for military medical personnel, or the medical evaluation and treatment of servicemembers. To that end, MTFs serve as training and readiness platforms for active-duty medical providers in two respects. First, many MTFs host graduate medical and dental education programs for physicians and dentists, and other training and education programs for medical providers. Graduate medical education (GME) programs train physician specialties through internships, residencies, and fellowships, thereby helping maintain the necessary pipeline of physicians to staff the MTFs and to deploy in support of military operations. The MTFs host non-GME training and education programs for other medical personnel, such as physician assistants, nurses, and enlisted technicians, which help them attain and maintain their skills. Second, day-to-day patient care at MTFs helps maintain the clinical skills and readiness of medical providers. The military departments track clinical readiness for providers using a series of checklists for deployable medical specialties. In addition, since 2018, DOD has piloted a clinical readiness metric for select physician specialties that provide combat casualty care. To meet the metric, a physician must attain a minimum threshold of points that indicate the complexity, diversity, and volume of patient care they provided. Finally, the MTFs also maintain data on physicians’ clinical workloads to measure their productivity against benchmarks and thereby approximate their clinical readiness. These clinical workload data are recorded as work Relative Value Units (wRVU), a metric of the level of professional time, skill, training, and intensity to provide a given clinical service. TRICARE Networks and Health Plans Under TRICARE, DOD maintains a purchased-care system of civilian providers to augment MTF capabilities. In each TRICARE region (East and West), DOD contracts with private-sector companies—referred to as managed-care support contractors—to develop and maintain networks of civilian providers and perform other customer service functions, such as processing claims, enrolling beneficiaries, and assisting beneficiaries with finding providers. The Director of the DHA awards and oversees the managed-care support contracts. TRICARE’s non-Medicare-eligible beneficiaries generally obtain coverage through two health plan options—TRICARE Prime (a managed-care option) and TRICARE Select (a self-managed, preferred provider option). All active-duty servicemembers are required to enroll in the Prime option, while other TRICARE beneficiaries may choose it. Prime enrollees receive most of their care from MTFs and also may receive purchased care from network civilian providers. Prime has the lowest out- of-pocket costs for beneficiaries, as care provided at MTFs does not have a copayment. TRICARE Prime has five access standards that set requirements for (1) travel time to provider sites, (2) appointment wait time, (3) availability and accessibility of emergency services, (4) composition of network specialists, and (5) office wait time. TRICARE Select beneficiaries are able to obtain health care from network and non-network providers. They can also receive care from MTFs, but they have a lower priority for receiving care than TRICARE Prime beneficiaries and are seen on a space-available basis. MTF Statutory Requirements and MHS Governance to Implement Reforms Section 703(a) of the NDAA for Fiscal Year 2017 added section 1073d to title 10, United States Code, which set forth various requirements for MTFs. To support the medical readiness of the armed forces and the readiness of medical personnel, the Secretary of Defense is required to maintain three types of MTFs—medical centers, hospitals, and ambulatory care centers (or clinics). All of these MTFs are required to provide specific health services required to maintain medical readiness. Hospitals are to be located in areas where civilian health care facilities are unable to support the health care needs of members of the armed forces and covered beneficiaries. Both hospitals and clinics are to provide limited specialty care that is cost-effective or is not available at civilian health care facilities in the area. In 2017, DOD appointed a Reform Leader for Health Care Management. Among other responsibilities, the Reform Leader led a work group to address Section 703 (hereafter, we refer to this as the 703 Work Group). The 703 Work Group included representatives from the Office of the ASD(HA), DHA, Joint Staff, the military services, and the TRICARE Health Plan. Together, the 703 Work Group members led DOD’s efforts to address section 703(c) of the NDAA for Fiscal Year 2017 by updating its 2016 Report on Military Health System Modernization (“the Modernization Study”) to address the future restructuring of MTFs pursuant to 10 U.S.C. § 1073d; determine the scope of its review of MTFs in the United States (i.e., identify which MTFs to evaluate for the Plan, as opposed to those to evaluate at a later date); develop MTF-specific recommendations for whether to restructure an MTF and in what ways to do so by developing and applying a methodology to assess each MTF in accordance with 10 U.S.C. § 1073d; and draft the final section 703(d) Plan to Congress delineating the restructuring actions it determined. In making determinations for selected MTFs, the 703 Work Group drafted a “Use Case” for each MTF summarizing potential restructuring actions and their analytical basis. The Work Group presented each MTF “Use Case” for review to a team of senior DOD leaders, including the Under Secretary of Defense for Personnel and Readiness, the ASD(HA), the undersecretaries of the military departments, and military service leaders, among others. When the senior leaders agreed on the restructuring actions for the MTFs, the 703 Work Group presented those determinations to the Secretary of Defense for approval. Other MHS Reforms and Prior GAO Work In recent years, DOD leaders have taken steps to refocus the MTFs as platforms for sustaining high-quality combat casualty care and the operational readiness of active-duty medical providers while increasing efficiency, in part by responding to congressional mandates. In 2016, for example, DOD submitted the Modernization Study to Congress in response to section 713 of the Carl Levin and Howard P. “Buck” McKeon National Defense Authorization Act (NDAA) for Fiscal Year 2015. Its main goals were to increase medical force readiness to support military operations and achieve cost savings. The Modernization Study included an MTF analysis of 24 hospitals to determine whether they should maintain inpatient capabilities or birthing centers. It recommended changes for 10 of the 24 hospitals, including closing inpatient services in whole or part at eight of them. In September 2016, we reported that the Modernization Study’s recommendations positioned DOD to improve the effectiveness and efficiency of the MHS, but there were shortcomings in its methodology. To strengthen any future assessments of MTF changes, we recommended that DOD describe steps taken to assess the reliability of supporting data. DOD concurred with the recommendation and has taken some steps to implement it. In response to other provisions in the NDAA for Fiscal Year 2017, DOD has made reforms aimed at improving the MHS focus on readiness. Our prior work has made recommendations to address gaps in those reforms. For example, in February 2019 we reported that DOD had not determined the required size and composition of its operational medical and dental forces who support the wartime mission or submitted a complete report to Congress, as required by section 721 of the NDAA. We recommended that DOD establish joint planning assumptions and a method for assessing efficiencies and risk, use them to determine its operational medical and dental requirements, and report the requirements to Congress. DOD concurred but had not implemented the recommendations as of May 2020. We also reported in February 2019 that DOD had begun initiatives to maintain the wartime readiness of medical providers in response to section 725 of the NDAA for Fiscal Year 2017. However, DOD’s methodology was limited with respect to a key initiative—the use of a metric to assess medical providers’ clinical readiness. We made three recommendations to improve DOD’s application of the metric. DOD concurred but had not implemented them as of May 2020. According to MHS leaders, efforts to identify the required number of operational medical personnel and the level of readiness they must maintain (pursuant to sections 721 and 725) were foundational steps toward section 703 of the NDAA for Fiscal Year 2017. A list of other related products is also included at the end of this report. DOD’s Methodology for Determining MTF Restructuring Actions Prioritized Cross- Cutting Statutory Elements, but Included Some Incomplete and Inaccurate Information The DOD 703 Work Group’s methodology for determining MTF restructuring actions was thoroughly documented and prioritized cross- cutting statutory elements, including support for military readiness, adequate nearby civilian health care, and cost-effectiveness. However, the group based key parts of its methodology on some incomplete and inaccurate information. The 703 Work Group Evaluated MTFs’ Support for Military Readiness, Nearby Civilian Health Care, and Cost- Effectiveness, and Documented This Approach and Its Results In reviewing the 703 Work Group’s methodology for determining MTF restructuring actions, we found that the group prioritized cross-cutting elements from 10 U.S.C. § 1073d to guide its approach. Its methodology to evaluate each selected MTF consisted of data analyses and interviews with officials from the MTF and its host installation. The 703 Work Group based its MTF evaluation determinations on, by order of priority, the (1) support each MTF provides to servicemembers’ medical readiness and the readiness of military medical providers, (2) adequacy of civilian health care facilities and providers to support the health care needs of servicemembers and other beneficiaries through purchased care near where each MTF is located, and (3) the cost-effectiveness of direct care services at the MTF relative to purchased care in the area. In addition to thoroughly documenting this methodology for evaluating the MTFs, the 703 Work Group documented the basis for the resulting conclusions. Servicemembers’ and military medical providers’ readiness. According to 703 Work Group leaders, as a first step in developing a methodology for evaluating MTFs for restructuring actions, they decided on a strategy they believed would prioritize MTF support to servicemembers’ medical readiness and the readiness of military medical personnel. To that end, the Work Group established minimum criteria to determine an MTF’s level of support to readiness. In most cases, the Work Group determined that MTFs should maintain certain minimum capabilities for servicemembers’ individual medical readiness, including primary care and, on a case-by-case basis, specialty services such as behavioral health and physical therapy. In addition, through site visits and interviews with MTF and installation personnel, the Work Group determined that certain MTFs should maintain urgent and emergency care services if they support a large training component on an installation. The other element of the 703 Work Group’s strategy to prioritize readiness was to evaluate the contribution of each MTF toward the clinical readiness of military medical providers and recommend restructuring actions on the basis of attaining minimum standards therein. In particular, our review of methodology documents revealed the Work Group prioritized MTFs’ support to the readiness of combat casualty care physicians. In doing so, the Work Group analyzed clinical workloads (e.g., wRVUs) and readiness metrics to identify which MTFs supported the physicians’ attainment of minimum thresholds. Finally, the Work Group determined that MTFs that host a GME program for training a combat casualty care or other physician specialty, or a graduate dental education program, should preserve the inpatient services required to continue the program. Adequacy of nearby civilian health care. A secondary criterion the 703 Work Group applied in its methodology for evaluating MTFs for restructuring was its determination of whether civilian health care facilities and providers in proximity to a given MTF (i.e., TRICARE network providers as well as non-network providers) were adequate to absorb an increased demand for certain health care services from the MTF—that is, whether DOD could use purchased care from civilian providers to replace care divested from a restructured MTF. According to Work Group officials, their strategy in applying this criterion was to reduce or eliminate health care services from MTFs if those capabilities (1) were not needed for readiness purposes and (2) could be adequately replaced with civilian facilities and providers through purchased care. In making this determination, the 703 Work Group conducted two assessments for each evaluated MTF. These assessments applied different criteria and assumptions to determine the adequacy of civilian health care, as shown in table 1. In addition, the Work Group supplemented the assessments by interviewing MTF and installation personnel to gain their perspectives about the availability of civilian care nearby. Cost-effectiveness of direct-care services at MTFs. Last in order of priority was the 703 Work Group’s determination of whether the MTF- delivered health care is cost-effective relative to nearby purchased care. Specifically, the Work Group assessed whether the cost per unit of health care delivered at each evaluated MTF was less than, equal to, or greater than the unit cost of purchased care. According to Work Group officials, these assessments supplemented the criteria on readiness and civilian health care adequacy by lending a resource-informed perspective to the overall methodology. A determination that an MTF’s unit costs exceeded those of nearby purchased care would confirm to the Work Group that it should consider replacing health care services from that MTF with purchased care, provided that the group had first determined that the corresponding capabilities (1) were not needed to support readiness, and (2) could be adequately replaced with purchased care from civilian health care providers nearby. Table 2 below illustrates an example of the 703 Work Group’s calculation of cost-effectiveness at one MTF it evaluated—the 2nd Medical Group outpatient clinic at Barksdale Air Force Base in Louisiana. In this case, the MTF’s unit cost in fiscal year 2017 was more than two times the cost of purchased care. These results confirmed the Work Group’s recommendation that some of the MTF’s capabilities should be replaced with purchased care—specifically, the health care services that it provided to non-active-duty servicemembers. The 703 Work Group Based Parts of Its Methodology on Incomplete and Inaccurate Information Notwithstanding the positive aspects of DOD’s methodology for evaluating MTFs previously discussed, our review found that information the 703 Work Group considered was sometimes limited in completeness, accuracy, or both. Specifically, the Work Group (1) conducted limited assessments of MTFs’ readiness support to military primary care and nonphysician medical providers, and did not include, as part of its methodology, (2) complete and accurate data about the quality of and access to purchased care from civilian providers, or (3) alternative assumptions that could affect the perceived cost-effectiveness of MTF- provided direct care. Limited Assessments of MTFs’ Readiness Support for Primary Care and Nonphysician Medical Providers The 703 Work Group conducted limited assessments of MTFs’ support for the readiness of military primary care physicians and nonphysician medical providers, including nurses, physician assistants, and enlisted medical and surgical specialists, which constitute a substantial portion of DOD’s medical forces. As discussed previously, the Work Group prioritized assessments of MTFs’ support to combat casualty care physicians’ readiness. For military primary care providers, the Work Group determined whether a minimum amount of patient care workload (i.e., RVUs) was available at each MTF to support productivity goals. This was due in part to the fact that DOD has not developed a clinical readiness metric for primary care and nonphysician providers as it has for combat casualty care providers, according to Work Group officials. Unlike a clinical readiness metric, a productivity goal does not account for the types of workload needed for readiness. According to MTF medical providers, they could meet their productivity goal as their MTF restructures, but doing so would not ensure that they addressed diverse and complex medical issues needed to maintain their clinical skills. MHS senior leaders and MTF officials, including providers, expressed concern that opportunities to treat diseases and nonbattle injuries will be limited in MTFs that restructure to serve only active-duty servicemembers. We also found that the 703 Work Group did not assess the support that certain training and education programs provide to the readiness of medical personnel at evaluated MTFs. The Work Group surveyed each MTF within its scope to identify any graduate medical and dental education and non-GME training and education programs the facility hosts. The Work Group determined that nonprimary care GME and graduate dental education programs are essential to maintain at MTFs, but did not evaluate the readiness benefits of primary care GME and non- GME training to MTFs. We found that half of the MTFs identified for restructuring as active-duty clinics or for closure host one or more non- GME training program for nurses, nurse practitioners, and enlisted medical personnel, among others. Four MTFs that were deferred or identified for reduction in capabilities or closure either host or support a GME program for primary care physicians. MTF officials we interviewed expressed concerns about the effects on military providers’ readiness from reducing or displacing the programs. However, DOD’s Plan states that any effects on GME and non-GME training programs will be addressed later in a next phase of executing MTF restructuring transitions, as discussed later in this report. Incomplete and Inaccurate Information on Quality of and Access to Civilian Health Care Providers near MTFs Although DOD assessed the availability of civilian health care providers and facilities in proximity to MTFs, as described above, our review of DOD’s assessments found that information gathered and applied in the course of its methodology was sometimes incomplete and inaccurate. Specifically, the information we reviewed did not consistently account for the quality of available civilian health care providers in proximity to MTFs and the extent to which those providers meet access-to-care standards, as described in detail below. As a result, DOD’s assessments may have included providers of lower quality health care and those who do not meet DOD’s access-to-care standards. Including such providers in its assessments means that DOD’s conclusions could be overestimated regarding the adequacy of civilian health care providers in proximity to some MTFs. Quality of available care near MTFs. The 703 Work Group’s assessments did not consistently account for the quality of available providers located in proximity to each MTF. Although the TRICARE Health Plan assessments documented and considered patient satisfaction scores and quality ratings for hospitals from the Centers for Medicare & Medicaid Services, its assessments of individual providers did not contain information about quality of care. The independent contractor’s assessments did not include any information about the quality of available providers it identified. Instead, DOD generally assumed that all identified providers were of sufficient quality. Officials from the 703 Work Group stated that they sometimes discussed the quality of available civilian health care during their site visits and interviews with MTF officials. However, our review found that quality of care was not consistently documented or considered for decision-making purposes. For example, in our review of 11 selected MTFs, we found that the Work Group documented and considered the quality of available civilian health care in proximity to one of the 11 MTFs—Bayne-Jones Army Community Hospital at Fort Polk, Louisiana. In this instance, the Work Group’s information about the variable quality of civilian health care near Fort Polk led to their determination that available care was not yet adequate to restructure the MTF. Other MTF officials discussed with us concerns they had about the quality of purchased care from some civilian providers. Similarly, a recent study found that TRICARE-insured families were less likely to report accessible or responsive care compared to civilian peers, whether commercially or publicly insured or uninsured. We have previously reported on concerns about DOD’s information about the quality of purchased care. In September 2018, we reported that the MHS does not monitor and report on quality measures for individual civilian providers, although it does so for purchased care networks as a whole. In contrast, the MHS maintains numerous measures for its MTFs to track, assess, and report the quality of care and related patient outcomes. Access to an accurate and adequate number of current civilian health care providers in the TRICARE networks. DOD’s assessments of available civilian health care surrounding MTFs did not consistently apply complete and accurate information about patients’ access to care in terms of the number of available TRICARE providers in proximity to MTFs. DOD’s assessments relied on the directories of network providers (primary and specialty care) that are maintained by each of the regional TRICARE contractors. In November 2019, we reported on problems with the accuracy of these provider directories. Specifically, we reported that as of June 2019, the TRICARE West region contractor’s directory of network providers was 76 percent accurate and the East region’s was 64 percent accurate, according to DHA officials. However, we found that the TRICARE Health Plan verified the accuracy of the directory entries of network providers in proximity to only one of 77 MTFs—the Army’s Farrelly Health Clinic at Fort Riley, Kansas. In this instance, the list of available health care providers in proximity to Farrelly clinic was overstated by 26 percent because of duplicate listings and practices that had closed, among other factors. Likewise, MTF officials we interviewed stated that the TRICARE network directories in their area contained inaccurate information, such as outdated provider listings, and overstated the number of providers who were accepting new TRICARE patients. Access to providers within standards for patients’ drive time. DOD’s independent contractor assessments of available civilian health care providers (both TRICARE network and non-network providers) used some inaccurate information about those providers, especially their locations. For 11 selected MTFs, we found that about 56 percent of primary care providers and 42 percent of specialty care providers that an independent contractor identified in its assessment exceeded DOD’s drive-time standards for TRICARE Prime patients by varying degrees, as shown in figure 1. A certain portion of the providers listed for each of the 11 selected MTFs were outside the drive-time standards, based on our analysis. In addition, for each of the 11 selected MTFs, there was one or more inaccuracies in the provider listing, such as providers that were no longer in practice, duplicate providers, or those that were mischaracterized as a medical provider. MTF officials we interviewed also expressed concerns that the assessments did not account for traffic, including bridges and tunnels that create traffic chokepoints. In other words, they believed that even providers that appeared to be within drive- time standards based on mileage could actually exceed the standard depending on their location and time of day. Appendix III illustrates the results of our analysis in detail. DOD guidance states that beneficiaries should have a choice of health care providers that is sufficient to ensure access to appropriate, high- quality health care. In addition, Standards for Internal Control in the Federal Government require the use of quality information that is appropriate, current, complete, accurate, accessible, and timely to inform decisions. Such standards also require that an agency’s management define objectives clearly to enable the identification of risk and risk tolerances, to include defining objectives in specific and measurable terms to allow for the assessment of performance toward achieving objectives. Applied to DOD’s analysis of civilian health care available in proximity to MTFs, such information would include (1) health care quality and (2) accurate and complete access-to-care data for civilian providers identified in its assessments. DOD’s assessments were missing complete and accurate information about the adequacy of purchased care through civilian health care providers because 703 Work Group officials stated that their analyses were detailed enough for the purposes of decision-making about restructuring. Furthermore, they stated that they plan to “test” the purchased-care networks of civilian providers during the transition of MTFs to their restructured end states. Officials stated they believe such a test will reveal that the supply of providers will increase over time to meet an increased demand for care from DOD beneficiaries. However, recent research has reported concerns about growing nationwide shortages of physicians, including primary care providers—a type of civilian health care provider that will be in high demand from DOD beneficiaries as MTFs restructure. For example, a 2019 study projected physician demand will continue to grow faster than supply, leading to a projected nationwide shortfall of between 46,900 and 121,900 physicians by 2032. DOD officials stated they expect to monitor health care quality and patients’ access during the implementation of MTF transitions. While this will be a positive step, a better understanding of the quality of civilian health care providers and patients’ access to an adequate supply of such providers within drive-time standards could help DOD in its implementation planning for MTF transitions and its tests of network capabilities by illustrating areas of highest risk. Until DOD consistently captures and assesses information about the quality of available civilian health care and the extent to which such care has met and will continue to meet patients’ access standards, DOD leaders may not fully understand risks to the achievement of their objectives in restructuring future MTFs. Cost-Effectiveness Assessments Based on a Single Set of Assumptions DOD applied a single set of assumptions in comparing the cost- effectiveness of direct care delivered at MTFs to that of purchased care, as previously discussed. On the basis of our analysis of the assumptions and related data elements, and interviews with DOD officials, we found that the assumptions do not account for uncertainties that could affect conclusions about an MTF’s cost-effectiveness. Specifically, DOD made assumptions about the costs of military personnel salaries, the workload performed at MTFs, and the reimbursement rates for TRICARE that individually and collectively likely result in the underestimation of the cost- effectiveness of MTFs, as described in more detail below. Including the full cost of military medical personnel does not account for their value outside of MTFs in support of military operations. DOD included the full cost of active-duty medical personnel salaries when calculating the unit-level cost of MTF health care. This approach assumed that military personnel spend all their time in MTFs. However, military personnel who staff MTFs sometimes spend half or more of their time contributing to other military work activities, according to MHS officials. These personnel are essential for military operations outside the MTFs. Accordingly, DOD referred to its medical personnel as a “fixed cost” in the Modernization Study. In its interim report to Congress for section 721 of the NDAA for Fiscal Year 2017, DOD determined that about 111,000 active-duty personnel are essential to support its war plans as part of the operational medical force. By including the full cost of military personnel salaries in calculations of the unit-level cost of MTF-provided care, DOD has likely underestimated the cost-effectiveness of MTFs given the dual purpose of active-duty medical personnel who staff MTFs but spend time on other military duties and deploy to support operations. According to MTF officials, some portion of the cost of military personnel salaries could be considered an approximation of the “cost of medical force readiness” for the wartime mission, though an imperfect one. Units of health care may underreport workload performed at MTFs. DOD calculated the cost of delivering a single unit (e.g., wRVU), of care at its MTFs. Doing so likely underestimates the cost- effectiveness of MTFs given concerns that wRVUs may be underreported. MTF officials at all 11 locations and 703 Work Group members agreed that wRVUs are likely underreported within MTFs for various reasons. For example, some MTF services are not recorded in wRVUs, such as telehealth consultations, which comprise a growing share of patient encounters, according to MTF officials. In addition, in February 2019 we reported that source data for DOD’s clinical readiness metric for physicians—the same data MTFs use to record wRVUs—had not passed DOD audits for at least 3 years. Likewise, in April 2016, we reported concerns that providers’ workload at MTFs was not being accurately recorded. TRICARE reimbursement rates for purchased care will likely need to increase. In comparing the cost-effectiveness of direct care at MTFs to purchased care from civilian providers in the TRICARE networks, DOD applied current TRICARE reimbursement rates in its calculations. MTF and 703 Work Group officials, along with senior MHS leaders, agreed that DOD may need to pay higher reimbursement rates in the future to attract new, quality network providers as its reliance on purchased care for beneficiaries increases in proportion to the decrease in access to health care services at many MTFs. In addition, MTF officials and MHS leaders stated that utilization of some purchased-care services from civilian providers may be higher than utilization of like services at MTFs because civilian providers are not incentivized to manage health services and costs the way the MHS does. This means that the cost of purchased care could increase by more than expected if utilization rates increase. For example, a research study completed in 2017 found that an estimated 21 percent of purchased medical care in the United States is attributed to unnecessary costs associated with overtreatment. In 2010, the Institute of Medicine reported that unnecessary services are the largest contributor to waste in the U.S. health care system, and could have accounted for about $210 billion in excess spending in 2009. By applying a single set of assumptions as described above, DOD’s assessment of the cost-effectiveness of MTFs was not consistent with a key practice in economic analysis. Our assessment methodology for economic analysis states that a sensitivity analysis is an essential element of a high-quality analysis of cost-effectiveness. Likewise, a DOD instruction on economic analysis states that analyses of investment alternatives should include, among other things, a sensitivity analysis, accounting for uncertainties by testing the sensitivity of the economic analysis results against various factors. A sensitivity analysis examines the effects that changes to key assumptions have on the analytic outcome and are helpful to understanding risk. To demonstrate the effect of a single set of assumptions versus an analytical approach that explored other assumptions, we adjusted some of the assumptions for illustrative purposes. Our analysis found that for two of seven MTFs we evaluated in detail, changing DOD’s assumptions in only one respect—by subtracting military personnel salaries—would have materially affected DOD’s assessment about whether direct care at the MTF was more cost-effective than purchased care. Further, if military personnel salaries are excluded from the assessments and TRICARE reimbursement rates increase by 5 percent, three of the seven MTFs would be more cost-effective than purchased care. For illustrative purposes, figure 2 shows how alternative assumptions could change both the data (i.e., costs and wRVUs) and the results of the assessments as to whether an MTF is more or less cost effective than purchased care. According to officials from the 703 Work Group, they did not apply alternative assumptions to analyze cost-effectiveness because readiness and the adequacy of civilian health care were more important in their methodology, and they generally assumed that purchased care is less costly. However, DOD could still maintain its prioritization sequence while augmenting its cost-effectiveness analyses with a sensitivity analysis to help provide more complete information for decision-making and, in the future, for executing MTF transitions. Without doing so, DOD leaders may further jeopardize their understanding of risks to the achievement of their objectives in restructuring future MTFs. DOD Is Not Yet Well Positioned to Execute MTF Restructuring Transitions Through its section 703(d) Plan to Congress, DOD has identified actions that will be needed to facilitate MTF restructuring. These actions include 17 recommendations for enterprise-wide changes across MTFs, and various MTF-specific steps to mitigate risks at a local level. However, DOD’s Plan also poses challenges for the military departments and the DHA related to medical provider readiness and MTF staffing. DOD does not have a process for monitoring MTF restructuring transitions to address these challenges. DOD Has Identified Collective and Specific Actions to Facilitate MTF Restructuring Through its Plan, DOD has taken preliminary steps toward transition planning by identifying actions needed to facilitate the restructuring and MTF transitions. Specifically, in the Plan DOD (1) recommended certain actions across the collective enterprise of MTFs to facilitate their restructure to reduced health-care delivery capabilities, and (2) identified risks and potential mitigation strategies specific to each MTF identified for restructuring. According to the Plan and 703 Work Group officials, DOD will begin to plan these transitions in detail after a congressional review period is completed in May 2020. Enterprise-wide actions for transition planning for restructured MTFs. In its plan, DOD recommended 17 enterprise-wide actions to facilitate MTF restructuring transitions. The Plan noted that the actions apply across various installations and MTFs, and were not specific to any certain region, military service, or population size. According to 703 Work Group officials, these actions will be critical to the successful transition of MTFs to their restructured end states. We found that the actions described in the Plan are interdependent and have implications for military readiness, the adequacy of civilian health care in proximity to MTFs, and the cost-effectiveness of MTF health care, which are discussed throughout our report. Moreover, we found that the recommendations require actions and coordination from multiple organizations and stakeholders. For example, the Plan recommends structuring health care operations to support patients from the military’s Exceptional Family Member Program in relevant markets. The military departments are responsible for oversight of this Program, and their coordination with the DHA, MTF officials, and with military commands will be needed to ensure those patients’ medical needs are met. This and the other 16 actions are listed below in table 3, along with the stakeholders who may be needed to implement them. MTF-specific actions for transition planning. A second step the 703 Work Group has taken toward preliminary transition planning is to identify, for each MTF, certain salient risks and potential mitigation strategies. The Work Group documented these risks and mitigations in the “Use Case” for each MTF, which are included in appendices to the Plan. The “Use Cases” summarize, among other things, the recommended restructuring actions and the related analytical basis. Specifically, the “Use Cases” list risks and related mitigation strategies, noting that the lists summarize observations from the Work Group’s analyses but are not exhaustive. For some risks, the “Use Cases” noted that a mitigation strategy should be determined later. For example, a risk associated with patients’ changes in expectations—from getting health care at the MTF to getting care outside the base (from a civilian provider)— will need to be monitored and managed. Other risks and mitigation strategies identified in the MTF “Use Cases” are specific to an MTF’s concerns based on local considerations, such as the health care services they deliver, the type of active-duty population they serve, and their knowledge of the nearby civilian health care providers. For example, at Langley Air Force Base, where DOD is recommending that the hospital transition to an ambulatory surgery center (which would not have inpatient care or an emergency room), the “Use Case” for the MTF notes that the elimination of inpatient capabilities would decrease the MTF’s support to readiness. This means that future numbers and types of patients and health care services delivered at Langley’s MTF, once it becomes an ambulatory surgery center, may not sustain the clinical readiness requirements of the active-duty medical personnel assigned to work there—requirements that they must meet for deployments. Accordingly, the “Use Case” notes that a related mitigation strategy would create partnerships across area hospitals where Langley’s medical personnel may be able to supplement their MTF workload and maintain their readiness. As another example, the “Use Case” for Fort Polk’s MTF in Louisiana— where DOD is recommending the MTF maintain inpatient care in the short term but monitor the expansion of local hospitals to determine when inpatient services can be replaced with purchased care—noted several specific risks related to patients’ access to care given the remote, rural location. One risk pertains to labor and deliveries, and the “Use Case” noted that the two nearest off-base hospitals in the TRICARE network have had problems with fiscal solvency and there is an insufficient number of obstetricians. Accordingly, DOD’s mitigation plan is for the MTF to initially maintain pre- and postnatal care services to expectant mothers, while (1) monitoring the TRICARE network hospitals and (2) ensuring the MTF’s obstetricians have privileges at those hospitals for labor and delivery until the number of network obstetricians is sufficient. DOD Does Not Have a Process for Monitoring Restructuring Transitions to Address Challenges Challenges for Military Departments and DHA Related to Medical Providers’ Readiness and MTF Staffing Despite DOD’s preliminary steps toward transition planning, including the mitigation strategies for risks previously discussed, its Plan poses other challenges for the military departments and the DHA in executing MTF restructuring. In particular, our review highlighted two interrelated challenges with respect to medical providers’ clinical readiness and MTF staffing levels. Military departments’ efforts to maintain the clinical readiness of primary care and nonphysician medical providers. As discussed earlier in this report, MTF officials stated that MTFs that restructure will not likely present the diverse and complex medical conditions needed to sustain the readiness of military primary care and nonphysician providers. A senior enlisted leader at one MTF observed that the staff would “have to be creative” to find the right mix of opportunities for enlisted personnel to gain the clinical experience they need to be ready to deploy as the clinic transitions to seeing only active-duty patients. MHS leaders we interviewed agreed with these concerns. To address MTF and MHS leaders’ concerns, 703 Work Group officials stated that the MHS would need to develop better metrics for primary care and nonphysician providers’ clinical readiness requirements, as the MHS has done for combat casualty care physicians. The officials also stated that another mitigation plan will be to allow MTFs that become active-duty clinics to diversify the patient population available to providers by treating some family members and retirees. MTF officials we spoke with were encouraged that continuing to treat some family members and retirees could help address the provider readiness shortfalls they believe are inherent to becoming an active-duty clinic. However, they and senior MHS leaders were concerned about the prospect of differentiating among such beneficiaries in terms of who may be eligible for MTF care at an active-duty clinic. To that end, officials stated that having the DHA clarify its roles and responsibilities in executing this flexibility will be a helpful step. To address challenges in maintaining the clinical readiness of medical providers assigned to MTFs that restructure, 703 Work Group officials stated that existing MHS partnerships with civilian hospitals and the Department of Veterans Affairs should be sufficient for MTF providers along with other available mechanisms, such as temporary duty at other MTFs. However, MHS leaders stated that existing civilian partnerships, in particular, may not have sufficient capacity to take on additional military medical personnel. As a result, the leaders believe they may need to expand partnerships to accommodate the expected increase in demand from military providers for on-the-job readiness training as MTF capabilities decrease during restructuring transitions. Furthermore, MTF officials we interviewed had mixed opinions about the readiness benefits they derived from their experiences with civilian hospital partnerships and training at other MTFs. DHA and the military departments’ ability to fully staff the MTFs. According to MTF officials, sending their medical providers to work outside their assigned MTF in support of clinical readiness, though temporary, creates another challenge by reducing providers’ availability to the MTFs. As more providers require such experience due to MTF capabilities’ decrease from restructuring, MTF officials we interviewed noted that staffing gaps could complicate their ability to execute the transitions and ensure the continuity of care for patients. Furthermore, MTF officials stated that active-duty medical personnel reductions that occurred in fiscal year 2019 have also created shortfalls in staffing that could pose challenges for them in executing the MTFs’ transitions. According to these officials, this is because they expect their administrative workload to increase while transitions are ongoing, while clinical workload for patient care would not decrease soon enough to mitigate any shortfalls in providers. DOD’s Plan states that the DHA should collaborate with the military departments to establish standard staffing models to facilitate MTF transitions, and transition plans must specify reductions in personnel and resources for the future state of the MTFs. However, the continuation of a phased transfer of MTF administration and management to the DHA from the military departments may present challenges to the DHA’s ability to concurrently accomplish new tasks related to restructuring the MTFs and facilitating their transitions. Likewise and more broadly, we reported in November 2018 that the transfer of MTF administration and management to the DHA may present challenges to the management of military personnel given that the military departments are responsible for medical personnel readiness, not the DHA, while DHA assumes responsibility for staffing the MTFs. No Process for Monitoring MTF Restructuring Transitions DOD has not established a process for monitoring MTF restructuring transitions to address the aforementioned challenges. Yet, the MHS plans to move forward with restructuring actions beginning in June 2020. While officials expect that transitions of certain smaller clinics to their restructured end state may be relatively simple, they acknowledged that other MTF transitions could be complex and take several years. According to the ASD(HA) and Work Group officials, DOD will readjust its plans by reversing or slowing an MTF transition, if needed, to address any challenges that arise with ensuring patients’ ability to access health care—one of the restructuring objectives. DOD’s Plan does not discuss conditions that would warrant slowing or reversing an MTF’s restructure, or how that would be determined. According to senior MHS leaders and MTF officials, the potential need to reverse or slow transitions will make monitoring the transitions important, and they are awaiting such decisions, along with associated roles and responsibilities from the DHA. However, the Plan does not establish a process for monitoring MTF restructuring transitions, as this was not within the scope of efforts, according to 703 Work Group officials. Rather, officials stated that decisions about monitoring should occur in a next phase of execution for MTF transitions after completion of the Plan. Accordingly, after DOD submitted its Plan to Congress in February 2020, the ASD(HA) issued a memorandum tasking the Director of the DHA to implement the changes specified in the Plan (i.e., the MTF restructuring actions) and providing high-level guidance. For example, the memorandum states that: MTF transitions are not authorized to start before May 19, 2020 (i.e., 90 days after the Plan was provided to Congress) but should be completed no later than October 1, 2025; transition planning may begin at the DHA Director’s discretion (but not later than the beginning of fiscal year 2021) and should include all impacts from ongoing personnel reductions and realignments; and detailed transition plans should include clear mechanisms for stakeholder tracking of activities and progress, and be arranged in a manner that addresses the needs of multiple stakeholders from the local to the national levels. Regarding the transition plans, the memorandum requested that that the DHA Director provide the ASD(HA) with a point of contact within 5 days, and a timeline, milestones, initial resource requirements, and task organization for the effort within 2 weeks—i.e., by February 26 and March 6, 2020, respectively. The DHA missed these milestones, having not yet provided the requested information, although an official from the Office of the ASD(HA) stated that, as of March 2020, the DHA response was being drafted. MHS reform and the DHA’s progress in achieving goals are longstanding challenges on which we have previously reported. In April 2012, before the DHA was established, we reported that DOD did not consistently employ key management practices in implementing initiatives to change its MHS governance structure. We recommended that the ASD(HA) and the Surgeons General implement a monitoring process across DOD’s portfolio of initiatives for overseeing progress and identify accountable officials and their roles and responsibilities for all of its initiatives. DOD implemented this recommendation by assigning each initiative a working group, an initiative leader, and executive sponsor to help ensure that the initiative remained on schedule, on budget, and achieved performance goals. After DOD established the DHA, we reported in November 2013 and later in September 2015 on its progress. In both reports, we identified deficiencies and made recommendations to provide decision makers with more complete information on the implementation, management, and oversight of the DHA. DOD concurred with the 10 related recommendations and implemented all but one. We reported in March 2004 that a process for monitoring progress is key to successful results-oriented management. However, DOD does not have such a process for the MTF restructuring transitions, in part because MHS officials stated they would first need to establish detailed roles and responsibilities for executing the transitions generally. Beyond the DHA Director’s role of transition leader, other roles and responsibilities have not been established, such as what involvement MTF officials will have in monitoring and tracking progress or challenges, and how the military departments will share responsibilities with the DHA. The Senior Military Medical Advisory Council could sufficiently monitor the transitions at a high level, according to the DHA Director. Other MHS leaders we spoke with believed that involvement from additional military department and Office of the Secretary of Defense leaders could also be needed. As we reported in October 2005, agreement on roles and responsibilities is a key step to successful collaboration when working across organizational boundaries, such as the military services. Committed leadership by those involved in the collaborative effort, from all levels of the organization, is needed to overcome the many barriers to working across organizational boundaries. Our prior work has also shown that a dedicated team vested with necessary authority and resources to help set priorities, make timely decisions, and move quickly to implement decisions is critical for a successful transformation. DOD also has not defined objectives in a measurable way with related thresholds and goals to enable monitoring of progress and challenges. For example, as previously discussed, DOD’s three general priorities, or objectives, for restructuring MTFs include ensuring (1) the medical readiness of servicemembers and readiness of medical providers, (2) that civilian health care facilities and providers adequately support the health care needs of beneficiaries near each MTF, and (3) the cost-effectiveness of MTF and purchased care. However, DOD has not decided how to define and measure any of those objectives. Furthermore, DOD has not established thresholds or goals in relation to the objectives. By first establishing clear roles and responsibilities for executing and monitoring restructuring transitions, DOD can be better positioned to navigate and overcome organizational boundaries between the DHA, which manages the MTFs, and the military departments that provide staff. In doing so, DOD could also be better positioned to address challenges in executing transitions, such as those that arise with mitigating providers’ clinical readiness challenges and MTF staffing gaps during transitions. Then, by defining measurable objectives, goals, and thresholds for tracking the progress of MTF transitions—such as the clinical readiness of providers, quality and accessibility of quality health care, and cost- effectiveness—DOD could better ensure its objectives are being met and help facilitate timely adjustments to the transitions, as needed. Conclusions As MHS leaders have acknowledged, correctly aligning MTF infrastructure to the size of the armed forces, the medical forces, and their desired readiness levels is essential to balancing mission requirements within available resources. DOD’s substantial work over the past 2 years on its Plan for MTF restructuring is a positive step toward meeting statutory requirements and prioritizing MTF readiness outcomes in a resource-informed manner. Notwithstanding the work DOD has undertaken in making a series of analytically-based determinations for restructuring in its Plan, our review highlighted several gaps in DOD’s methodology. Until DOD takes action to address these gaps by using more complete and accurate information about civilian health care quality, access, and cost-effectiveness, DOD leaders may not fully understand risks to the achievement of their objectives in restructuring future MTFs. DOD officials agree that some MTF restructuring actions may be more challenging than others. These challenges could be exacerbated by concurrent MHS reform efforts, including the transition of MTF administration and management to the DHA. However, by establishing clear roles and responsibilities for executing and monitoring the transitions, DOD can be better positioned to overcome the difficulties in navigating organizational boundaries between the DHA and the military departments, and make timely adjustments to their transition plans, as needed. In addition, by defining measurable objectives, thresholds, and goals for restructuring transitions, and applying them to evaluate progress and challenges, DOD can be better positioned to execute the transition of its MTFs and ensure that the objectives are being met. Recommendations for Executive Action We are making the following six recommendations to DOD: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Health Affairs, in coordination with the Surgeons General of the military departments and the Director of the DHA, consistently collect complete and accurate information about the quality of available civilian health care in proximity to its MTFs (such as ratings from the Centers for Medicare and Medicaid Services and perceptions from MTF officials who regularly coordinate with civilian providers, among other means) and assess that information to inform recommendations for future MTF restructuring decisions. (Recommendation 1) The Secretary of Defense should ensure that the Assistant Secretary of Defense for Health Affairs, in coordination with the Surgeons General of the military departments and the Director of the DHA, consistently collect complete and accurate information about the extent to which current health care providers within the TRICARE networks meet access-to-care standards, and assess that information to inform recommendations on future MTF restructuring decisions. (Recommendation 2) The Secretary of Defense should ensure that the Assistant Secretary of Defense for Health Affairs, in coordination with the Surgeons General of the military departments and the Director of the DHA, consistently collect complete and accurate information about the extent to which non-network civilian health care providers that could be incorporated into the TRICARE network meet access-to-care standards in terms of drive time, and assess that information to inform recommendations on future MTF restructuring decisions. (Recommendation 3) The Secretary of Defense should ensure that the Assistant Secretary of Defense for Health Affairs, in coordination with the Surgeons General of the military departments and the Director of the DHA, conducts a sensitivity analysis of the relative cost-effectiveness of MTF-provided care compared to civilian-provided care under varying assumptions, and document that information for decision makers to inform recommendations on future MTF restructuring decisions. Varying conditions could include different types of health care services, reducing the cost of military personnel salaries, and increasing estimated MTF wRVUs and civilian reimbursement rates. (Recommendation 4) The Secretary of Defense should ensure that the Assistant Secretary of Defense for Health Affairs, in coordination with the Surgeons General of the military departments and the Director of the DHA, establishes clear roles and responsibilities for executing and monitoring transitions for MTFs identified for restructuring. (Recommendation 5) The Secretary of Defense should ensure that the Assistant Secretary of Defense for Health Affairs, in coordination with the Surgeons General of the military departments and the Director of the DHA, defines measurable objectives for MTF restructuring transitions, establishes thresholds and goals for each objective, and applies them to evaluate progress and challenges. For example, measurable objectives, thresholds, and goals, should include an evaluation of medical providers’ clinical readiness, civilian health care provider adequacy, and the cost-effectiveness of MTF and purchased care. (Recommendation 6) Agency Comments and Our Evaluation We provided a draft of this report to DOD for review and comment. In its written comments, reproduced in appendix IV, DOD concurred with two of our recommendations and partially concurred with four recommendations. DOD also provided technical comments on the draft report, which we incorporated as appropriate. DOD concurred with our recommendations to establish roles and responsibilities for executing and monitoring MTF restructuring transitions (recommendation 5), and to define measurable objectives with thresholds and goals and apply them to evaluate progress and challenges for the transitions (recommendation 6). In its response, the department described actions it is taking and plans to take to implement both recommendations. DOD partially concurred with our first recommendation—to collect complete and accurate information about the quality of available civilian health care in proximity to its MTFs and assess that information to inform recommendations for future MTF restructuring. DOD stated that complete and accurate information on the quality of available care would require substantial resources to accomplish on a routine basis. To that end, DOD stated that until standardized quality data becomes readily available, it intends to collect this level of information as needed to support actions at a particular MTF. As noted in our report, we have previously reported that standardized information about hospital and outpatient care quality is available through the Centers for Medicare & Medicaid Services and has been widely adopted by major private insurers. As the restructuring of the MTFs continues and DOD relies to a greater extent on civilian-provided care, it will be important for the department to monitor the quality of care it purchases on behalf of beneficiaries. Thus, we continue to believe that DOD should make it a priority to collect and assess such information. DOD partially concurred with our second recommendation—to consistently collect complete and accurate information about the extent to which current health care providers within the TRICARE networks meet access-to-care standards, and assess that information to inform recommendations on future MTF restructuring decisions. In its response, DOD stated that each month, TRICARE contractors report, by specialty, average wait times from referral placement to patient appointment. Further, DOD stated that it is piloting centralized booking of MTF and network appointments, which, if successful, will result in more complete, accurate, and timely network access information. In cases where access standards are not being met, DOD explained that it works to mitigate the access shortfall either through MTF or expanded network resources. We agree that TRICARE’s monthly reports on patient wait times for appointments are a helpful tool for DOD in monitoring access to care, and that the pilot for centralized appointment booking is also a promising step. As we noted in our report, however, DOD’s analyses of the adequacy of civilian health care in proximity to MTFs were based on network provider directories that are of questionable accuracy and can overstate the number of available providers. MTF officials we interviewed stated that TRICARE directories in their area overstated the number of providers accepting new TRICARE patients. Even if the provider directory issues have not led to access-to-care challenges in the past in terms of patients’ wait times to appointments, such issues could cause challenges in the future with increasing numbers of DOD patients needing TRICARE network care. Accordingly, we continue to believe that it will be important for DOD to collect complete and accurate information about the extent to which current health care providers within the TRICARE networks meet access-to-care standards as DOD moves forward with its restructuring plans. DOD partially concurred with our third recommendation—to consistently collect complete and accurate information about the extent to which non- network civilian health care providers that could be incorporated into the TRICARE network meet access-to-care standards in terms of drive time, and assess that information to inform recommendations on future MTF restructuring decisions. DOD stated that drive times for non-network providers were assessed in the development of the recommendations for its Plan. DOD added that the approach used in the Plan included assessing drive times and distances from the beneficiaries’ homes, rather than the MTF, yielding a more accurate assessment of access, availability, and convenience. However, our review of DOD’s methodology workpapers showed that its analyses measured the distance between providers and a single location point that corresponds with the center of the zip code boundary in which a majority of beneficiaries reside. While a perfect methodology and information for projecting actual drive times may not be possible to achieve, the alternative method in our report illustrates that a portion of the providers DOD identified for potential network expansion would exceed access-to-care standards for some of its beneficiaries—especially those who live or work near an MTF (such as those who live on a military installation). According to DOD’s comments on the report, future restructuring efforts will be informed by the section 703 approach, and DOD will adjust the approach as needed by future analysis and conditions. As DOD moves forward with restructuring efforts in the future, we continue to believe that more accurately measuring the distance between providers’ locations and beneficiaries’ residences would improve the quality of DOD’s information about access-to-care. Accordingly, we continue to believe that DOD should fully implement our recommendation. DOD partially concurred with our fourth recommendation—to conduct a sensitivity analysis of the relative cost-effectiveness of MTF-provided care compared to civilian-provided care under varying assumptions, and to document that information for decision makers to inform recommendations on future MTF restructuring decisions. In response, DOD stated that there is value in the use of sensitivity or scenario analysis to inform decisions where a range of possibilities exist and a clear analytical question can be formed as a guide to both the analysts and decision-makers. However, DOD stated that it does not support the generic use of this analysis suggested by the recommendation. We disagree that our recommendation suggests the “generic use” of a sensitivity analysis. A sensitivity analysis is appropriate for evaluating restructuring opportunities for MTFs for two reasons. First, the evaluation of each MTF presents decision makers with a range of possibilities—from reducing or expanding the capabilities of an MTF, to closing it entirely or maintaining the status quo. Second, a sensitivity analysis would address uncertainties in DOD’s analytic assumptions about costs and workload for each MTF, which our report identified. As our own sensitivity analysis— conducted using minimal resources and available DOD data— demonstrated, changing the assumptions also changes the resulting conclusions about whether MTF or civilian care is less expensive. Therefore, we continue to believe that analyzing the relative cost- effectiveness of MTF-provided care compared to civilian-provided care under varying assumptions would provide more complete information for decision-making and executing MTF transitions. We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, the Under Secretary of Defense for Personnel and Readiness, the Assistant Secretary of Defense for Health Affairs, the Director of the Defense Health Agency, and the Secretaries of the Army, the Navy, and the Air Force. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V. Appendix I: Objectives, Scope, and Methodology In February 2020, the Department of Defense (DOD) submitted its implementation plan (“the Plan”) to Congress in response to section 703(d) of the National Defense Authorization Act (NDAA) for Fiscal Year 2017. This report addresses the extent to which (1) DOD’s methodology for determining military medical treatment facility (MTF) restructuring actions in the Plan prioritized cross-cutting elements from 10 U.S.C. § 1073d and considered complete information, and (2) DOD has positioned itself to execute transition planning for restructuring its MTFs. For both objectives, we selected a nongeneralizable sample of MTFs and applied a case study approach to review DOD’s methodology for determining restructuring actions and actions that may be needed for transition planning. From DOD’s initial list of 73 MTFs included in its scope, we selected 11 of them as case studies to represent a variety of characteristics, including a mix of hospitals and clinics from each military department, different recommendations for how they should be restructured, different conclusions about network adequacy, and urban and rural areas located in proximity to one another in terms of driving distance. Appendix II identifies the names and locations of each MTF within the scope of the DOD Plan. The 11 MTFs we selected are listed in table 4. For objective one, we reviewed DOD’s draft and final Plan and related documentation of the methodologies used to assess all 77 MTFs within its scope. We compared this information with cross-cutting elements for MTFs from 10 U.S.C. § 1073d. These elements include the (1) support an MTF provides to servicemembers’ medical readiness and the readiness of medical personnel, (2) adequacy of civilian health care facilities and providers in the proximity of the MTF to support the health care needs of servicemembers and other beneficiaries through purchased care, and (3) cost-effectiveness of direct care services at MTFs versus purchased care in the nearby civilian provider networks. We discussed the methodological approaches for assessing MTFs for possible restructuring actions, including assumptions, data sources, and any limitations, with representatives from relevant organizations across DOD, including the Office of the Assistant Secretary of Defense for Health Affairs; DOD’s Section 703 Work Group; Defense Health Agency; Office of the Assistant Secretary of the Army for Manpower and Reserve Affairs; U.S. Army Medical Command; Navy Bureau of Medicine; Air Force Medical Readiness Agency; and officials from 11 selected MTFs and from their host installations. Specifically, in analyzing DOD’s methodology for assessing MTFs’ support to readiness, we reviewed information DOD used to estimate MTFs’ readiness value in terms of support to servicemembers’ medical readiness and to medical force readiness. We compared this information with findings from our prior work regarding DOD’s methodology for assessing clinical readiness. We also reviewed records of interviews that DOD officials held with MTF, installation, and command officials during their visits to MTFs, noting the readiness-related effects and concerns that were documented. For DOD’s methodology for assessing available civilian health care services in proximity to each MTF, we reviewed reports on the results of DOD’s assessments to identify their findings, recommendations, and assumptions. For the civilian health care providers that DOD identified in proximity to each of 11 MTFs we selected, we verified the address of each listed provider by searching for each provider’s website and making phone calls to verify addresses and specialty types, and whether the practice was open or closed. We then used R software and data from openstreetmap.org to calculate to calculate the driving distance between the provider and the MTF, comparing the distance with DOD’s access-to- care standards. We evaluated the extent to which the assessment reports considered information about quality of health care services and access-to-care standards, comparing the information with DOD guidance for patients’ access to quality and timely health care services, and with federal internal control standards on the use of quality information to inform decision-making. Regarding DOD’s methodology for evaluating cost-effectiveness, we reviewed DOD’s work papers and interviewed officials about the calculations and source data they used. We compared DOD’s methodology with our assessment methodology for economic analysis and with DOD guidance for economic analysis. We also obtained the fiscal years 2017 and 2018 data DOD used to calculate the cost- effectiveness of MTF-provided direct care relative to civilian-provided purchased care. Using these data, we performed a sensitivity analysis by recalculating relative cost-effectiveness under different assumptions. Specifically, for seven of our 11 case study MTFs, we recalculated their cost-effectiveness relative to purchased care by (1) omitting military personnel salaries, given that DOD has characterized these as a fixed cost, (2) increasing the work Relative Value Units to adjust for potential underreporting of those data, and (3) increasing the reimbursement rate of purchased care to account for future increases that are likely necessary, according to DOD officials. We also assessed the reliability of each data source for DOD’s and our calculations of cost-effectiveness by administering questionnaires about the data to those who have quality control responsibilities, interviewing responsible DOD officials, reviewing the data for outliers and missing values, and reviewing our prior reports about the data. We determined that DOD’s data on the costs of MTF care and civilian health care were sufficiently reliable for the purpose of calculating the total costs of health care services. However, DOD’s data on units of health care delivered in fiscal year 2018 were of undetermined reliability for the purpose of calculating a unit-level health care cost. To provide additional information on DOD’s methodology and supplement our understanding of available data, we conducted a literature review of research articles. We conducted a search of the literature on military health system clinical readiness, trends in physician supply and demand across the United States, and cost analyses of military health care published from 2014 through 2019 to identify articles on key challenges and methodological alternatives. To identify relevant articles, we searched a variety of databases with the assistance of a research librarian, limiting our review to papers that were included in peer-reviewed publications, as well as government reports, trade and industry articles, and publications by associations, nonprofits, or think tanks. We then reviewed the results and excluded any that were technical in nature or did not have wide applicability across MTFs or to health care analyses. For objective two, we reviewed DOD’s draft and final Plan, including detailed appendices on the MTFs within the scope of the plan, noting any aspects of transition planning described or recommended, and the agencies and organizations that would be responsible for managing those transition aspects. We also interviewed MTF officials from the selected case study locations regarding steps they had begun taking and actions they believed would be needed to facilitate a restructuring of the facility. In addition, we reviewed our prior, related work on MHS reform and the establishment of the Defense Health Agency to identify related themes and challenges. We corroborated our understanding of transition planning steps described in the Plan by interviewing 703 Work Group officials, the Director of the Defense Health Agency, and the Surgeons General of the Army and the Air Force to better understand what roles and responsibilities and monitoring mechanisms they had considered. We compared this information from the Plan and from our interviews with practices identified in our prior work on results-oriented government. Specifically, our prior work has found that agreement on roles and responsibilities and committed leadership by those involved are key steps to successful collaboration when working across organizational boundaries. A dedicated team vested with necessary authority and resources to help set priorities, make timely decisions, and move quickly to implement decisions, along with a process for monitoring progress, are also key to success. Finally, to assess the extent to which DOD’s section 703(d) Plan addressed all requirements of section 703(d), we compared the Plan with the elements from the statute. Examples of those elements included, for each MTF, whether it will be restructured, whether its functions will be expanded or consolidated, and the related costs. Some of the elements required that multiple items be addressed. We considered an element “addressed” if it included all of the items listed in the NDAA; “partially addressed” if it included some, but not all, of the items; and “not addressed” if it did not include any of the items. We conducted this performance audit from February 2019 to May 2020 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Locations of Military Medical Treatment Facilities within the Scope of DOD's Section 703(d) Plan on Restructuring A work group of representatives from across the Department of Defense (DOD) led efforts to address section 703(d) of the National Defense Authorization Act for Fiscal Year 2017—the restructure or realignment of military medical treatment facilities (MTF). The 703 Work Group developed a methodology to address section 703(d) and determined the scope of its review of MTFs in the United States by identifying which of those to evaluate for the mandated implementation plan (the “Plan”). Figures 3 through 5 below identify the name and location of each of the 77 MTFs within the scope of DOD’s Plan, which it submitted to Congress in February 2020. Appendix III: Distances from Military Medical Treatment Facilities to Civilian Providers Identified in DOD Assessments The Department of Defense’s (DOD) 703 Work Group based its military medical treatment facility (MTF) restructuring determinations for its implementation plan to Congress, in part, on its assessments of the adequacy of civilian health care facilities and providers to support the health care needs of DOD beneficiaries near each MTF. In one component of the assessments, DOD identified civilian health care facilities and providers in proximity to each of its 77 evaluated MTFs. For each provider DOD identified in proximity to 11 MTFs—which we selected from the 77 MTFs DOD evaluated—we verified the provider’s address, specialty type, and whether the practice was open or closed. We then calculated the driving distance between each MTF and the respective listed providers. Figures 6 through 16 below show that a certain portion of the providers listed for each of the 11 selected MTFs were outside DOD’s access-to-care standards for travel time to provider sites for TRICARE Prime patients, based on our analysis. In addition, for each of the 11 selected MTFs, there was one or more inaccuracies in the provider listing, such providers that were no longer in practice, duplicate providers, or those that were mischaracterized as a medical provider. Appendix IV: Comments from the Department of Defense Appendix V: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Lori Atkinson (Assistant Director), Melissa Blanco (Analyst in Charge), John Beauchamp, Timothy Carr, Alexandra Gonzalez, Hannah Hubbard, David Jones, Amie Lesser, John Mingus, Jr., Oliver Richard, Terry Richardson, Guiovany (Geo) Venegas, and Lillian M. Yob made key contributions to this report. Related GAO Products Defense Health Care: Opportunities to Improve Future TRICARE Managed Care Support Contract Transitions. GAO-20-39. Washington, D.C.: November 21, 2019. Defense Health Care: DOD’s Proposed Plan for Oversight of Graduate Medical Education Programs. GAO-19-338. Washington, D.C.: March 28, 2019. Defense Health Care: Actions Needed to Determine the Required Size and Readiness of Operational Medical and Dental Forces. GAO-19-206. Washington, D.C.: February 21, 2019 Defense Health Care: Additional Assessments Needed to Better Ensure an Efficient Total Workforce. GAO-19-102. Washington, D.C.: November 27, 2018. Defense Health Care: DOD Should Demonstrate How Its Plan to Transfer the Administration of Military Treatment Facilities Will Improve Efficiency. GAO-19-53. Washington, D.C.: October 30, 2018. Defense Health Care: Expanded Use of Quality Measures Could Enhance Oversight of Provider Performance. GAO-18-574. Washington, D.C.: September 17, 2018. Defense Health Reform: Steps Taken to Plan the Transfer of the Administration of the Military Treatment Facilities to the Defense Health Agency, but Work Remains to Finalize the Plan. GAO-17-791R. Washington, D.C.: September 29, 2017. Defense Health Care Reform: DOD Needs Further Analysis of the Size, Readiness, and Efficiency of the Medical Force. GAO-16-820. Washington, D.C.: September 21, 2016. Defense Health Care Reform: Actions Needed to Help Ensure Defense Health Agency Maintains Implementation Progress. GAO-15-759. Washington, D.C.: September 10, 2015. Military Health System: Sustained Senior Leadership Needed to Fully Develop Plans for Achieving Cost Savings. GAO-14-396T. Washington, D.C.: February 26, 2014. Defense Health Care Reform: Additional Implementation Details Would Increase Transparency of DOD’s Plans and Enhance Accountability. GAO-14-49. Washington, D.C.: November 6, 2013. Defense Health Care Reform: Applying Key Management Practices Should Help Achieve Efficiencies within the Military Health System. GAO-12-224. Washington, D.C.: April 12, 2012. Military Personnel: Enhanced Collaboration and Process Improvements Needed for Determining Military Treatment Facility Medical Personnel Requirements. GAO-10-696. Washington, D.C.: July 29, 2010.