Background VHA’s Allocation of Funds In February of each year, the President submits the budget request for VHA health care, which includes requested funding for the upcoming fiscal year as well as an advance appropriation request to Congress. VHA allocates funds by the beginning of each fiscal year—October 1—to VISNs and medical centers based on the amount VA received in the advance appropriation. Once appropriations are enacted for the upcoming fiscal year, VHA updates the allocated funding levels for VISNs and medical centers. For example, VHA’s appropriations for fiscal year 2018 included advance appropriations for fiscal year 2019. VHA allocated funds by October 1, 2018 (the beginning of fiscal year 2019), based on the advance appropriation for fiscal year 2019 and updated the funding levels once appropriations for fiscal year 2019 were enacted. VHA allocates general purpose funds to its 18 VISNs through the Veterans Equitable Resource Allocation (VERA) model. It uses a separate model, called the Medical Center Allocation System (MCAS), to allocate each VISN’s general purpose funds—as determined by the VERA model—to the medical centers within each network. VHA guidance permits VISNs to make adjustments to the general purpose funding levels determined by MCAS for each medical center. VHA uses other methods to allocate specific purpose funds to VHA program offices that manage various health care programs, such as those for community care, prosthetics, and homelessness. The program offices, in turn, typically allocate funds for these programs directly to medical centers. (See fig. 1.) Once funds are allocated and distributed to VISNs and medical centers, these funds may be redistributed in accordance with law across VA’s health care system. These redistributions can help address unfunded needs or surpluses that may arise. For example, according to officials, a medical center may need additional funds to provide care for veterans when natural disasters occur. VHA’s General Purpose and Specific Purpose Funding Levels From fiscal year 2015 to fiscal year 2019, general purpose funds increased by 33 percent- from $37 to $49 billion—while specific purpose funds increased by 24 percent—from $19 to $23 billion. (See fig. 2.) In fiscal year 2019, community care accounted for $10.5 billion—46 percent—of the $23 billion allocated in specific purpose funds. Patient care services, homelessness programs, non-recurring maintenance, and medical residency programs also accounted for large portions of specific purpose funds. VA’s Appropriation Accounts and Community Care As of fiscal year 2017, VA primarily receives appropriated funds for all health care it provides or purchases through four appropriation accounts. The amount of funds in each appropriation account is determined by VA’s annual appropriation. VHA allocates both general and specific purpose funds from these appropriation accounts. These accounts include the following: Medical Services: health care services provided to eligible veterans and other beneficiaries in VA facilities and non-VA facilities, among other things. Medical Community Care: health care services that VA authorizes for veterans and other beneficiaries to receive from community providers. Medical Support and Compliance: the administration of the medical, hospital, nursing home, domiciliary, supply, and research activities authorized under VA’s health care system, among other things. Medical Facilities: the operation and maintenance of VHA’s capital infrastructure, such as the costs associated with nonrecurring maintenance, leases, utilities, facility repair, laundry services, and groundskeeping, among other things. Separate from VA’s health care appropriation accounts, the Veterans Access, Choice, and Accountability Act of 2014 established the Veterans Choice Fund and provided $10 billion in funds for the Veterans Choice Program (Choice Program), which was implemented in early fiscal year 2015 and authorized until funds were exhausted or through August 7, 2017, whichever occurred first. The Choice Program allowed veterans to elect to receive care from community providers when the services needed were not available at a VA medical center, were not available within VHA’s wait-time goals, or when veterans did not reside near a VA medical facility with a full-time primary care provider. Eligible veterans could also elect to receive care in the community if they met other eligibility criteria as well. VA received additional authority and funds to maintain the Choice Program through June 6, 2019, when it sunsetted, and the new Veterans Community Care Program (VCCP) went into effect. The VCCP was established by the VA MISSION Act and consolidated the Choice Program along with several other community care programs. The VCCP is primarily funded through specific purpose funds in the Medical Community Care appropriation account. The VCCP is similar to the former Choice Program in allowing veterans to elect to receive care from community providers when certain eligibility criteria are met, including criteria relating to the availability and accessibility of the services at VHA. Under the VCCP, VHA adopted designated access standards for VCCP eligibility determinations that are broader than the eligibility criteria that existed under the Choice Program. VHA Allocates Funds Based on Patient Workload but Does Not Use the Most Up- to-date Data and Certain Adjustments to Funding Levels May Lead to Inefficiencies VHA Allocates General Purpose and Community Care Funds Based on Patient Workload VHA’s VERA model uses a national, formula-driven approach that considers the number and type of veterans served and the complexity of care provided—collectively referred to as patient workload—as well as certain geographic factors, such as local labor costs, to determine the amount of general purpose funds each VISN should receive. VHA uses VERA to establish funding levels for each VISN in the following areas: patient care, equipment, education support, and research support, the largest of which is patient care. After determining the amount of funds VISNs should retain for VISN-level initiatives, administrative purposes, and reserves, VHA uses its MCAS model to distribute the remainder of each VISN’s general purpose funds to medical centers within the VISN. MCAS is based on a workload measure developed by VHA, called patient-weighted work (PWW) that accounts for medical center-level factors such as patient volume, case- mix, and specialized services. According to VHA officials, PWW establishes an equitable measure of workload among medical centers that vary significantly in their geographic location, and types and costs of services provided. Furthermore, PWW lessens the impact of cost differences between medical centers, by recognizing the varying costs and levels of resource intensity associated with providing care for each patient at each medical center. For example, PWW would result in more funds being allocated to a medical center that provides more complex care, such as open heart surgery, than a workload measure based solely on a count of each individual patient, which would not account for the additional costs associated with more complex care. Similar to MCAS, VHA’s Office of Community Care uses a patient workload-based model to allocate community care funds—which are specific purpose funds—to medical centers, based on each medical center’s community care patient workload in prior years. To determine the community care funding needs for each medical center, VHA calculates the PWW associated with community care. VHA determines the total amount of funds available for community care based on the amounts appropriated to the Medical Community Care appropriation account and the amount available for community care in the Veterans Choice Fund. VHA distributes the funds to each medical center in proportion to each medical center’s PWW. VHA officials told us that VHA is considering making changes to the methodology for allocating community care funds under the new VCCP, but as of July 2019, updates to the methodology had not been developed or implemented. The other four program offices we reviewed developed other methodologies for allocating other specific purpose funds. In general, these methodologies involve coordination between VISNs and their medical centers on needs for these funds and allocating available funds based on identified needs. For example, the Office of Patient Care Services which manages prosthetics and hepatitis C drugs allocates available funds based on identified needs by each of the medical centers. Appendix 1 provides an overview of the methodologies used by these four program offices to allocate special purpose funds. VHA’s Allocation Models for General Purpose and Community Care Funds Do Not Reflect the Most Up-To-Date Patient Workload Data Available To allocate funds for an upcoming fiscal year, VHA’s allocation models rely on actual patient workload data from prior fiscal years, but not the most recently completed fiscal year. VHA’s VERA model relies on actual patient workload data from two to six years prior to the upcoming fiscal year, in addition to future workload projections. For example, to allocate funds for fiscal year 2019, the VERA model relied on actual workload data from fiscal years 2013 and 2017, in addition to some future projected workload, but did not take into account actual workload data from fiscal year 2018. VHA’s MCAS and community care models rely on actual patient workload data from two years prior to the upcoming fiscal year. For example, the 2019 MCAS and community care models were based on actual workload data from fiscal year 2017, but did not take into account actual workload data from fiscal year 2018. According to VHA officials, patient workload data from the most recently completed fiscal year are not yet available when VHA runs the preliminary VERA, MCAS, and community care models for each fiscal year in August. However, these officials told us that these data would be available in December of each year and therefore could be incorporated into the final model run after VHA receives its enacted appropriation amount for the upcoming fiscal year. These officials told us that doing so would result in little to no delay in when the final model run and the final distribution of funds takes place, which occurs after the appropriation act is enacted. Specifically, according to officials from VHA’s Office of Finance, if the full fiscal year appropriation is enacted prior to the start of the fiscal year on October 1, VHA will be able to perform the final model runs by mid- November. As a result, incorporating data from the most recent fiscal year would result in a one month delay in the final model run. Should the enactment of a full year appropriation be delayed, the timing of the final model would not be impacted by using data from the most recently completed fiscal year. (See fig. 3.) A VHA Office of Finance official told us that VHA had not previously considered using patient workload data from the most recently completed fiscal year because VHA did not believe that using updated data would have a significant impact on the model. However, the official told us that the implementation of the VCCP in June 2019 may result in more significant year-to-year workload changes due to veterans increasing their use of VHA health care services. As a result, the official told us that using more up to date information would be more useful in informing allocation levels. Federal standards for internal control related to information calls for management to use quality information to achieve the entity’s objectives. Quality information is appropriate, current, complete, accurate, accessible, and provided on a timely basis. Because the VERA, MCAS, and community care models do not use the most up-to-date patient workload data available, the models may not reflect the most recent workload trends affecting medical centers. This may result in funding levels determined by the models that may not be commensurate with medical centers’ actual patient workload. For example, VHA data we reviewed show that some medical centers experienced workload changes in fiscal year 2018—changes that were not captured by the models for fiscal year 2019 allocations. Specifically, from fiscal years 2017 through 2018, while PWW for care provided by VA medical centers grew over 1 percent VHA-wide, 34 medical centers experienced growth of over 3 percent, and 9 experienced a decline of over 3 percent. Similarly, the PWW for care in the community grew over 6 percent VHA-wide from fiscal year 2017 to fiscal year 2018, with 97 medical centers experiencing growth of over 3 percent and 25 experiencing a decline in community care of over 3 percent over this time period. Additionally, officials we interviewed at six VISNs told us that the models have not accounted for recent workload growth their medical centers were experiencing due to an increase in the number of veterans they serve, the addition of new services, or changes in the medical centers’ reliance on community care. Two of these VISNs analyzed recent workload trends at the medical centers within their VISN and allocated additional funds to those medical centers with recent growth not accounted for by MCAS. If VHA were to incorporate the most recent available workload data into its allocation models, the need for such funding changes would likely be reduced. VHA Does Not Adequately Monitor Adjustments Made to Medical Centers’ Allocated Funding Levels, and Certain Adjustments May Lead to Inefficiencies As part of the allocation process, VISNs may make adjustments to the amounts of general purpose funds calculated by MCAS and allocated to medical centers. VHA guidance requires VISNs to provide a written explanation to VHA for any adjustments. However, we found that VHA does not adequately monitor these adjustments and that some of these adjustments may lead to inefficiencies. VHA Does Not Adequately Monitor Allocation Adjustments We found that VHA Office of Finance officials did not adequately review the fiscal year 2019 MCAS adjustments to ensure that adjustments were documented and fully explained. Specifically, VHA did not provide evidence that they sought an explanation for MCAS adjustments made by 2 VISNs that provided no written explanation for their adjustments, even though these explanations are required by VHA guidance. VHA Office of Finance officials said they use informal methods via email to learn about the adjustments and follow-up as needed, but could not provide documentation that follow-up and review had occurred. Additionally, VISN officials we interviewed from all 18 VISNs stated that they had not received questions or other feedback from the VHA Office of Finance on the adjustments they made, even if they had not documented and explained the adjustments. Furthermore, even if VISNs follow the requirement and submit written explanations for the adjustments, they may not provide the type of information VHA needs to adequately monitor the adjustments. This is because VHA guidance does not require VISNs to provide information on how they determined how much and for what reasons they are making the adjustments. For example, we found that 6 VISNs provided limited explanations for their fiscal year 2019 MCAS adjustments, such as stating that they had decided to reallocate funds among medical centers to ensure “continuity of operations,” which is insufficient information to allow VHA to determine if the adjustments were appropriate. Federal standards for internal control related to monitoring state that management should establish and operate monitoring activities to monitor the internal control system and evaluate the results. These monitoring activities could include establishing a formal process to document VHA’s review of VISN adjustments to medical center allocations. Additionally, monitoring activities could include requiring VISNs to provide information on how they determined how much and for what reasons they are making the adjustments and then reviewing such information. As VHA evaluates the adjustments, documenting the results of its monitoring and having the information needed to help determine the appropriateness of the adjustments will help VHA identify areas for improvement in the allocation process. Without adequate monitoring, VHA cannot reasonably ensure these adjustments are justified and align with VA’s strategic plan, which calls for the efficient allocation of funds. Certain Adjustments May Lead to Inefficiencies Based on interviews with VISN officials, we found that, in fiscal year 2019, seven VISNs adjusted the allocations determined by MCAS to ensure that every medical center within their VISN received either the same level of funding or a minimum funding increase of up to 2 percent relative to the prior year. According to VISN officials, funds were often shifted from medical centers that had received relatively large increases in funds due to growing workload to medical centers that had received a decrease or relatively flat funds compared to the prior year due to either declining or relatively flat workload. According to VISN officials, declining workload may be the result of medical centers serving fewer patients or patients obtaining care from community providers rather than VA providers. When asked about these adjustments, officials at the seven VISNs stated that they were necessary to ensure that affected medical centers could continue to cover the costs for the services they offer and the staff they employ, including providing federally mandated annual salary increases for those staff. Officials from four of these VISNs stated that it is difficult for medical centers to absorb a funding cut or only a small increase in funding from one year to the next due to rising costs they face. While VISNs are allowed to make adjustments to medical centers’ allocated general purpose funds, these adjustments may lead to inefficiencies because medical centers are not required to improve efficiency—such as, adjust the level of services they offer—to account for their decreases in workload. Additionally, officials from VHA’s Allocation Resource Center within the Office of Finance, which is responsible for developing and executing VHA’s allocation models, told us that because allocations made through MCAS are based on medical center workload, VISNs should avoid reallocating funds so that all medical centers receive a minimal increase. These officials said that doing so results in medical centers failing to adjust the level of services to meet workload needs. However, we found that for medical centers with declining workload, VHA guidance on allocation of funds does not require VISNs—in conjunction with these medical centers—to develop and submit approaches to improve the efficiency of medical center operations. Such improvements in efficiency would help lower overall costs. As we have previously stated, VHA’s strategic plan calls for the efficient allocation of funds. In addition, federal internal control standards related to control activities state that management should design control activities to achieve agency objectives. Such an activity could include having guidance on the allocation and adjustment of funds that promotes the efficient use of funds for delivering health care services to veterans. Without requiring VISNs—in conjunction with medical centers—to develop and submit an approach to change how medical centers with decreasing workload should operate, VHA increases the risk that these adjustments will not align with VA’s strategic plan. VHA Monitors the Use of Allocated Funds, but Does Not Adequately Monitor Redistribution of Funds VHA Monitors the Use of Allocated General Purpose and Specific Purpose Funds Using Multiple Mechanisms Once funds are allocated and distributed to VISNs and medical centers, VHA uses multiple mechanisms to monitor the balance of general purpose and specific purpose funds. VA uses these mechanisms to ensure that VISNs and medical centers operate within their allocated funding levels and are in compliance with the Anti-Deficiency Act. VHA’s primary monitoring mechanism is through VA’s financial management system, which is used to track obligations and prevent VISNs and medical centers from obligating amounts that are greater than the funds they have available. VHA also employs additional mechanisms to monitor the use of general and specific purpose funds. These additional mechanisms are described below. General Purpose Funds VHA’s Office of Finance requires each VISN to prepare an annual operating plan after the initial allocation of general purpose funds for each fiscal year that reflects the total planned obligations for the medical centers they oversee. These operating plans describe the planned obligation of funds throughout the fiscal year for various budget categories, such as personnel, equipment, transportation, and supplies and materials. VHA requires planned obligations reported in operating plans to align with the funding levels available to each VISN, which include allocated funds as well as anticipated collections, reimbursements, and funds carried over from previous years. VISNs are required to revise their operating plans during the fiscal year if major changes are made to their available funding levels, due to, for example, the enactment of a final appropriation bill, which results in final allocations. To monitor VISNs’ use of general purpose funds, VHA uses the operating plans to compare each VISN’s planned obligations with actual obligations on a monthly basis. VHA does not compare planned obligations with actual obligations for each medical center individually; instead, each of the 18 VISNs as well as the five medical centers we reviewed developed their own tools to monitor the use of funds. According to VHA officials, VHA requires each VISN to provide an explanation to VHA’s Office of Finance on a monthly basis about any variances of 5 percent or more above or below the amount between planned obligations in their operating plans and actual obligations. Based on VHA documents we reviewed, all 18 VISNs provided explanations for situations in which their actual obligations were equal to, higher, or lower than 5 percent from their planned obligations in fiscal years 2018 and 2019 and in some cases, also explained the actions they were planning to take to address the variance. VISNs reported several reasons for the variances. For example, some VISNs reported that their actual obligations exceeded planned obligations in some months because contracts or equipment purchases were executed earlier than anticipated in the year. Conversely, some VISNs reported that contracting delays led to actual obligations lagging behind planned obligations reported in their operating plans. An official from the VHA Office of Finance told us that they may contact VISN leadership—including the Director and Chief Financial Officer—if the variations are significant and additional actions needed to be taken. VHA Office of Finance officials also told us that they may review other reports if they become aware of an issue of significant interest to VHA leadership regarding a VISN’s or medical center’s obligations. Specific Purpose Funds Based on our review of VHA documents and interviews with program office officials, VHA program offices use various monitoring processes developed by each program office to monitor the use of specific purpose funds. Specifically, officials from the Office of Community Care told us that they monitor the use of community care funds by comparing actual obligations to planned obligations based on authorized community care. According to VHA officials, as of February 2019, VHA was in the process of developing an updated process to monitor the use of community care funds, which—starting in fiscal year 2019—were obligated at the time of claim payment rather than when care in the community was authorized. The other four VHA program offices we reviewed monitor the use of the funds they manage by generating a monthly or quarterly budget status report that compares each medical center’s actual obligations against their planned obligations. For example, officials from the Office of Patient Care Services told us that to monitor the use of funds for prosthetics, they conduct monthly reviews of obligations and ask the VISNs and medical centers to explain deviations between the actual and planned expenditures and provide an action plan. VHA Does Not Adequately Monitor the Redistribution of Funds Throughout the VA Health Care System, Which May Promote Inefficiencies After funds are allocated and distributed to VISNs and medical centers, VHA can redistribute funds across the VA health care system in accordance with law. VHA officials told us these redistributions are done to address unfunded needs and surpluses as they are identified and occur throughout the year. However, we found that VHA does not adequately monitor redistributions. We identified the following instances in which allocated funds are redistributed: VHA officials told us that VHA’s Office of Finance redistributes any surplus general purpose and specific purpose funds to medical centers based on VHA priorities and to address needs identified by VISNs. These officials said that these redistributions typically occur after the middle of the fiscal year. As of June 2019, according to VHA officials, one VISN had identified unfunded needs to VHA, but the VISN was working on addressing the funding needs using its own internal resources. Officials from another VISN told us that the VISN anticipated unfunded needs, but had not made a request to VHA for additional funds as of the end of May 2019. VISNs may also exchange funds with other VISNs. For example, if a VISN has excess medical facilities funds but a shortage of medical services funds, the VISN may exchange these funds with another VISN that has excess medical services funds but a shortage of medical facilities funds. According to VHA officials, VISNs must inform VHA about these exchanges of funds, but are not required to provide an explanation and do not require VHA approval for the exchanges. VHA officials told us that VISNs also have the flexibility to redistribute funds throughout the year from medical centers within their VISN that are experiencing a funding surplus to ones with unfunded needs. However, VISNs are not required to inform VHA about these redistributions and are not required to provide an explanation or get approval from VHA. For example, officials at one medical center told us that in recent years, its VISN redistributed an average of $15 million per year above allocated funding levels to this medical center to address unfunded needs. While the redistribution of funds throughout the year gives VHA flexibility to move funds where they are needed, VHA’s Office of Finance does not adequately monitor these redistributions. Specifically, VHA’s Office of Finance does not require VISNs to identify the reasons why they redistribute funds between VISNs and medical centers, and a VHA Office of Finance official told us VHA does not examine the amount of funds that are redistributed. For example, VHA’s Office of Finance could not provide us the total amount of redistributions that occurred throughout fiscal year 2018. As a result, VHA’s Office of Finance does not know why VISNs redistributed funds and the extent to which redistributions resulted in a deviation from VHA’s workload-based allocation levels. Monitoring the redistributions would provide VHA with information on the effectiveness of the allocation models and how they might be improved. VHA’s actions are inconsistent with federal internal control standards related to monitoring, which state that management should establish and operate monitoring activities to monitor the internal control system and evaluate the results. Without requiring VISNs to provide this information and without requiring VHA to document the results of its review of the redistributions, VHA cannot ensure that these redistributions align with VHA’s workload-based allocation of funds. As a result, VISNs and medical centers may not be efficiently operating within available funding levels, which include allocated funds, collections, reimbursements, and carry over from previous years. Conclusions VA’s strategic plan calls for the efficient use of funds for delivering health care services to veterans. Accordingly, it is critical that VHA closely monitor and account for how its funds are allocated to VA medical centers and redistributed throughout the year to help ensure the most efficient use of funds. Especially as the number of veterans eligible to receive care from a community provider potentially expands, it will be important for VHA to ensure allocated funding levels accurately reflect individual medical center funding needs. However, VHA has opportunities to strengthen its processes for allocating and monitoring funds distributed across its health care system. VHA could improve how it allocates funds to its VISNs and medical centers if it were to use the most up-to-date workload data available as part of its allocation models. This would allow VHA to account for significant changes in workload from year-to-year. VHA could also improve how it monitors VISN adjustments to medical center allocation levels as well as redistributions that may occur after medical centers receive their allocations. While these adjustments and redistributions afford flexibility and may be appropriate in certain circumstances, VHA does not have the information it needs to monitor these changes to ensure that they are appropriate and consistent with department goals. Specifically, VHA does not require VISNs and medical centers to provide information on how they determined the amount and reasons for adjustments, nor does VHA require VISNs—in conjunction with medical centers— to develop and submit an approach to improve efficiency at medical centers with declining workload, such as adjusting the level of services offered. Additionally, VHA does not require VISNs to identify the reasons why they redistribute funds between VISNs and medical centers after allocations have been made, and VHA does not document its review of these redistributions. As a result, VHA lacks reasonable assurance that adjustments and redistributions align with its strategic goals for efficient use of funds to best serve the needs of veterans across its healthcare system. Recommendations for Executive Action We are making the following five recommendations to VHA: The VA Under Secretary of Health should use workload data from the most recently completed fiscal year as part of the models that inform VISNs’ and medical centers’ general purpose funding needs, when doing so would not significantly delay the allocation of funds. (Recommendation 1) The VA Under Secretary of Health should establish a formal process to document VHA’s review of VISNs’ adjustments to medical center allocation levels. (Recommendation 2) The VA Under Secretary of Health should revise VHA’s existing guidance to require VISNs to provide information on how they determined how much and for what reasons they made adjustments to medical center allocation levels. (Recommendation 3) The VA Under Secretary of Health should revise its existing guidance to require VISNs—in conjunction with medical centers—to develop and submit approaches to improve efficiency at medical centers with declining workload that received adjusted funding levels. These approaches could include adjusting the level of services offered. (Recommendation 4) The VA Under Secretary of Health should require VISNs to provide explanations on the amount of funds redistributed between VISNs and medical centers and VHA to document its review of these redistributions. (Recommendation 5) Agency Comments We provided a draft of this report to VA for review and comment. In its written comments, reprinted in appendix II, VA concurred with four recommendations and concurred in principle with one recommendation. VA also described steps that it plans to take to implement the recommendations. In addition, VA provided a technical comment, which we incorporated as appropriate. Specifically, VA concurred with the first recommendation, stating that it will re-run the VERA model to allocate funds based on prior year workload data if an enacted budget is passed after the start of the second quarter of the current fiscal year. VA also concurred with the second and third recommendations, stating that it will update guidance to establish a formal process to document the review of VISN adjustments to medical center allocation levels and will require VISNs to provide information on how they determined adjustments prior to processing the adjustments. VA concurred in principle with the fourth recommendation, stating that VHA is conducting market assessments over a multi-year period to increase access and quality of care to veterans. VA said that after completing the market assessments and reviewing information from other VHA efforts, it may consider adjusting the level of services along with other alternatives. VA also concurred with the fifth recommendation, stating it will require review of redistributions between VISNs to ensure adequate explanations are included. According to VA, the department will also run a monthly report identifying redistributions between medical centers in a VISN that exceed 1.5 percent of the VISN’s funding allocation. We are sending copies of this report to the Secretary of Veterans Affairs, appropriate congressional committees, and other interested parties. This report is also available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or silass@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Appendix I: Methodologies Used by Four Selected Program Offices to Allocate Specific Purpose Funds In addition to the Office of Community Care, which managed $10.5 billion, or 46 percent, of specific purpose funds in fiscal year 2019, we contacted four other Veterans Health Administration (VHA) program offices that managed the largest amounts of specific purpose funds in fiscal year 2019—these included funds for patient care services, homelessness programs, non-recurring maintenance, and medical residency programs. These four program offices managed at least $1 billion of funds and collectively managed about 36 percent of all specific purpose funds in fiscal year 2019. The four program offices developed methodologies for allocating specific purpose funds that involve coordinating with VISNs and their medical centers on the purposes for which the funds would be used and allocating available funds based on these needs. See table 1. Appendix II: Comments from the Department of Veterans Affairs Appendix III: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Rashmi Agarwal (Assistant Director), Michael Zose (Analyst-in-Charge), and Carmen Rivera-Lowitt made key contributions to this report. Also contributing were Krister Friday, Cathleen Hamann, Jacquelyn Hamilton, and Ethiene Salgado- Rodriguez. Related GAO Products VA Health Care: Estimating Resources Needed to Provide Community Care. GAO-19-478. Washington, D.C.: June 12, 2019. VA’s Health Care Budget: In Response to a Projected Funding Gap in Fiscal Year 2015, VA Has Made Efforts to Better Manage Future Budgets. GAO-16-584. Washington, D.C.: June 3, 2016. Veterans’ Health Care Budget: Improvements Made, but Additional Actions Needed to Address Problems Related to Estimates Supporting President’s Request. GAO-13-715. Washington, D.C.: August 8, 2013. Veterans’ Health Care: Improvements Needed to Ensure That Budget Estimates Are Reliable and That Spending for Facility Maintenance Is Consistent with Priorities. GAO-13-220. Washington, D.C.: February 22, 2013. Veterans’ Health Care Budget: Better Labeling of Services and More Detailed Information Could Improve the Congressional Budget Justification. GAO-12-908. Washington, D.C.: September 18, 2012. Veterans’ Health Care Budget: Transparency and Reliability of Some Estimates Supporting President’s Request Could Be Improved. GAO-12-689. Washington, D.C.: June 11, 2012. VA Health Care: Estimates of Available Budget Resources Compared with Actual Amounts. GAO-12-383R. Washington, D.C.: March 30, 2012. VA Health Care: Methodology for Estimating and Process for Tracking Savings Need Improvement. GAO-12-305. Washington, D.C.: February 27, 2012. Veterans’ Health Care Budget Estimate: Changes Were Made in Developing the President’s Budget Request for Fiscal Years 2012 and 2013. GAO-11-622. Washington, D.C.: June 14, 2011. VA Health Care: Need for More Transparency in New Resource Allocation Process and for Written Policies on Monitoring Resources. GAO-11-426. Washington, D.C.: April 29, 2011. Veterans’ Health Care: VA Uses a Projection Model to Develop Most of Its Health Care Budget Estimate to Inform the President’s Budget Request. GAO-11-205. Washington, D.C.: January 31, 2011. VA Health Care: Challenges in Budget Formulation and Issues Surrounding the Proposal for Advance Appropriations. GAO-09-664T. Washington, D.C.: April 29, 2009. VA Health Care: Challenges in Budget Formulation and Execution. GAO-09-459T. Washington, D.C.: March 12, 2009. VA Health Care: Long-Term Care Strategic Planning and Budgeting Need Improvement. GAO-09-145. Washington, D.C.: January 23, 2009.


Background VHA’s Family Caregiver Program is designed to provide support and services to family caregivers of post-9/11 veterans who have a serious injury that was incurred or aggravated in the line of duty. The program provides approved primary family caregivers with a monthly financial stipend as well as training and other support services, such as counseling and respite care. The Family Caregiver Program has a series of eligibility requirements that must be satisfied in order for family caregivers to be approved. To meet the program’s initial eligibility criteria, the veteran seeking caregiver assistance must have a serious injury that was incurred or aggravated in the line of duty on or after September 11, 2001. According to the program’s regulations, a serious injury is any injury, including traumatic brain injury (TBI), psychological trauma, or other mental disorder, that has been incurred or aggravated in the line of duty and renders the veteran or servicemember in need of personal care services. The veteran must be in need of personal care services for a minimum of 6 continuous months based on any one of the following clinical eligibility criteria: (1) an inability to perform one or more activities of daily living, such as bathing, dressing, or eating; (2) a need for supervision or protection based on symptoms or residuals of neurological or other impairment or injury such as TBI, post-traumatic stress disorder, or other mental health disorders; (3) the existence of a psychological trauma or a mental disorder that has been scored by a licensed mental health professional, with a Global Assessment of Functioning score of 30 or less, continuously during the 90-day period immediately preceding the date on which VHA initially received the application; or (4) the veteran has been rated 100 percent service connected disabled for a qualifying serious injury and has been awarded special monthly compensation that includes an aid and attendance allowance. To be considered competent to care for the veteran, family caregivers must meet certain requirements including (1) having the ability to communicate and follow details of the treatment plan and instructions related to the care of the veteran; (2) not determined by VA to have abused or neglected the veteran; (3) being at least 18 years of age; and (4) either being a family member—such as a spouse, son or daughter, parent, step-family member, or extended family member— or an unrelated person who lives or will live full-time with the veteran. Family caregivers must also complete required training before being approved for the program. Family Caregiver Program Organizational Structure VHA’s Caregiver Support Program office is responsible for developing policy and providing guidance and oversight for the Family Caregiver Program. It also directly administers the program’s stipend, provides support services such as a telephone hotline and website, and arranges coverage through the Civilian Health and Medical Program of the Department of Veterans Affairs (CHAMPVA) for eligible caregivers if they have no other coverage. Further, the office provides funding to VAMCs to cover certain program costs. These costs may include the salaries of the caregiver support coordinators (CSC), who implement and administer the Family Caregiver Program at the local VAMC level, and the costs VAMCs incur for having their clinical staff, such as nurses, conduct the program’s required in- home visits to approved caregivers and their veterans. CSCs are generally licensed social workers or registered nurses, and they have both clinical and administrative responsibilities. Their clinical responsibilities may include identifying and coordinating appropriate interventions for caregivers or referrals to other VA or non-VA programs, such as mental health treatment, respite care, or additional training and education. Their administrative responsibilities may include responding to inquiries about the program, overseeing the application process, entering information about applications and approved caregivers into IT systems, and facilitating the processing of appeals. As of May 2014, there were 233 CSCs assigned to 140 VAMCs or health care systems across the country. Additionally, each regional VISN office has a VISN CSC lead for the program, who provides guidance to CSCs and helps address their questions or concerns. GAO Has Previously Reported on the Family Caregiver Program IT System Limitations CAT, which was deployed in May 2011, is a web-based system that was designed to facilitate the exchange of information about approved caregivers between VAMCs and other VHA entities. Such entities include the Health Administration Center, which processes the caregiver stipend payments and administers CHAMPVA. In 2014, we reported that the Caregiver Support Program office was not able to easily retrieve data from CAT that would allow officials to better assess workload trends at individual VAMCs—such as the length of time applications are delayed or the timeliness of home visits—even though these data were already captured in the system. Caregiver Support Program officials only retrieved workload data on an ad hoc, as-needed basis, which limited their ability to assess the scope and extent of workload problems comprehensively at individual VAMCs and on a system-wide basis. Program officials also expressed concern about the reliability of the system’s data. As we noted in our report, program officials also identified the need for a more capable and flexible system that could interface with other departmental systems. The officials told us that they had taken initial steps to obtain another IT system to support the Family Caregiver Program; however, the officials were not sure how long it would take to implement the system. Accordingly, we recommended that VA expedite the process for identifying and implementing a system that would fully support the Family Caregiver Program. VA concurred with our recommendation and subsequently began taking actions in 2015 to implement a replacement system. These actions included taking steps toward implementing short-term improvements to CAT that were to be followed by the implementation of a long-term replacement system. The recommendation continues to remain open. Statute Directs VA to Implement an IT System to Support the Family Caregiver Program The John S. McCain III, Daniel K. Akaka, and Samuel R. Johnson VA Maintaining Internal Systems and Strengthening Integrated Outside Networks Act of 2018 (VA MISSION Act), which was enacted in June 2018, included provisions directing VA to implement an IT system to support the Family Caregiver Program and the incremental expansion of program eligibility. Specifically, the act required VA to implement an IT system to fully support the Family Caregiver Program by October 1, 2018. According to the act, the system is to allow for data assessment and comprehensive monitoring of the program. In particular, the system is to have, among other things, the ability to (1) retrieve data to monitor workload trends at the medical center and aggregate levels; (2) manage an increased number of caregivers as the program expands; and (3) integrate with other relevant IT systems at VHA. The act also stated that VA was to submit an initial report to Congress regarding the status of the planning, development, and deployment of this system within 90 days of enactment of the VA MISSION Act, and that the department is to submit a final report to Congress by October 1, 2019. The final report is to include a certification by the VA Secretary that the system has been implemented, along with a description of how the Secretary is using the system to monitor the workload of the program. VA Has Not Yet Implemented an IT System That Effectively Supports the Family Caregiver Program Although we previously recommended that VA expedite implementation of a replacement for CAT, and the MISSION Act directed the department to implement an IT system to support the Family Caregiver Program, VA has not yet been successful in its multiple efforts to implement such a system. Specifically, VA has faced a number of difficulties in developing and implementing short-term improvements as well as a long-term replacement system for CAT. In July 2015, VHA and the Office of Information and Technology (OIT) initiated a joint acquisition project, called CAT Rescue, to update CAT and improve the system’s data reliability. However, the department reported in January 2017 that this project had experienced delays and identified a large number of defects during system testing. VA terminated the project in April 2018 before any new system capabilities were implemented. A companion project to CAT Rescue that VA initiated in September 2015 was to develop the Caregivers Tool (CareT), a new system intended to be a long-term replacement for CAT. As envisioned, this system was to use the improved data from CAT Rescue while also adding new system capabilities. However, the user acceptance testing of CareT identified the need for the department to develop more system capabilities than originally planned. Further, the department determined that the time period needed to perform additional system development would have extended beyond the term of the development contract, which ended in April 2017. VA subsequently awarded a new CareT development contract in July 2017. However, after additional system development, the department determined during user acceptance testing that the system was not performing as expected and implementation of CareT was further delayed. In October 2018, the department reported to congressional committees that implementing a system to fully support the Family Caregiver Program by the VA MISSION Act deadline was not feasible. Subsequently, the department determined that CareT was not a viable solution and VHA and OIT terminated work on the system in February 2019. VHA and OIT began a third effort in March 2019 to acquire a replacement system that is to be based on an existing commercial product. According to OIT officials, the new IT solution, referred to as the Caregiver Record Management Application (CARMA), is intended to replace CAT. However, the department has not yet established a date for completing CARMA. Thus, VA’s efforts to implement an IT system that supports the Family Caregiver Program have been continuing with no end in sight. We have ongoing work to further evaluate the status and progress of the department’s efforts to implement a system to support the Family Caregiver Program consistent with the VA MISSION Act requirements. Figure 1 provides a timeline of the various IT projects that VA has undertaken to support the program. Critical Factors Underlying Successful IT Acquisitions Our prior work has determined that successfully overcoming IT acquisition challenges can best be achieved when critical success factors are applied. Specifically, we reported in 2011 on common factors critical to the success of IT acquisitions, based on seven agencies having each identified the acquisition that best achieved the agency’s respective cost, schedule, scope, and performance goals. These factors remain relevant today and can serve as a model of best practices that agencies can apply to enhance the likelihood that the acquisition of an IT system such as CARMA will be successfully achieved. Among the agencies’ seven IT investments, agency officials identified nine factors as having been critical to the success of three or more of the seven investments. These nine critical success factors are consistent with leading industry practices for IT acquisition. The factors are: Active engagement of program officials with stakeholders. Qualified and experienced program staff. Support of senior department and agency executives. Involvement of end users and stakeholders in the development of requirements. Participation of end users in testing system functionality prior to formal end user acceptance testing. Consistency and stability of government and contractor staff. Prioritization of requirements by program staff. Regular communication maintained between program officials and the prime contractor. Sufficient funding. Officials for all seven selected investments cited active engagement with program stakeholders—individuals or groups (including, in some cases, end users) with an interest in the success of the acquisition—as a critical factor to the success of those investments. Agency officials stated that stakeholders, among other things, reviewed contractor proposals during the procurement process, regularly attended program management office sponsored meetings, were working members of integrated project teams, and were notified of problems and concerns as soon as possible. Further, officials from two investments noted that actively engaging with stakeholders created transparency and trust, and increased the support from the stakeholders. Additionally, officials for six of the seven selected investments indicated that the knowledge and skills of the program staff were critical to the success of the program. This included knowledge of acquisitions and procurement processes, monitoring of contracts, large-scale organizational transformation, Agile software development concepts, and areas of program management such as earned value management and technical monitoring. Finally, officials for five of the seven selected investments identified having the end users test and validate the system components prior to formal end user acceptance testing for deployment as critical to the success of their program. Similar to this factor, leading guidance recommends testing selected products and product components throughout the program life cycle. Testing of functionality by end users prior to acceptance demonstrates, earlier rather than later in the program life cycle, that the functionality will fulfill its intended use. If problems are found during this testing, programs are typically positioned to make changes that would be less costly and disruptive than ones made later in the life cycle. In conclusion, VA has invested considerable time in multiple efforts toward improving and replacing its IT system to better serve the Family Caregiver Program. However, even with these efforts, the department has not yet implemented a system and the program is not prepared for expansion. Going forward, it is important that VA take steps to improve its efforts to implement a replacement IT system for the Family Caregiver Program. In this regard, the department could benefit from applying critical success factors we previously reported as leading to successful federal IT acquisitions. These factors can serve as a model of best practices that the department can apply to enhance the likelihood that its effort to replace the IT system for the Family Caregiver Program will be successful. Chairs Lee and Brownley, Ranking Members Banks and Dunn, and Members of the Subcommittees, this completes my prepared statement. I would be pleased to respond to any questions that you may have. GAO Contact and Staff Acknowledgments If you or your staffs have any questions about this testimony, please contact Carol C. Harris, Director, Information Technology Management Issues, at (202) 512-4456 or harriscc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony statement. GAO staff who made key contributions to this testimony are Mark Bird (Assistant Director), Rebecca Eyler, Jacqueline Mai, Monica Perez-Nelson, Scott Pettis, and Jennifer Stavros-Turner (Analyst in Charge). This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.


Background VR&E Eligibility and Process To be entitled to VR&E services and related benefits, veterans generally must (1) have at least a 20 percent service-connected disability rating from VA and (2) be in need of rehabilitation because of an employment handicap. Entitled veterans may generally receive up to 48 months of vocational rehabilitation services and up to an additional 18 months of employment services, which include counseling, and placement and postplacement services. If a veteran is entitled to receive VR&E services and found to be employable, a counselor is to work with the veteran to identify a suitable employment goal, and to incorporate that goal and the needed services and benefits to achieve it into a vocational rehabilitation and employment plan (hereafter “employment plan”). To develop an employment plan, the counselor and veteran review labor market information for jobs within the veteran’s identified abilities, aptitudes, and interests that will not aggravate his or her service-connected disability or disabilities. After assessing obstacles to employment, they agree on a written employment plan that describes the employment goal and the services needed to achieve it. Common services provided by VR&E are funding for higher education, career counseling, and short-term employment services like job search assistance. Counselors have the authority to approve a wide variety of educational programs and may approve employment plans that have an annual cost of up to $25,000. VR&E Organization Within VA’s Veterans Benefits Administration (VBA), the VR&E central office is responsible for overseeing the VR&E program, including training staff and monitoring their work to ensure high performance and consistency. Among other elements, VR&E’s quality assurance efforts entail reviewing a subset of case files on a monthly basis to ensure that the entitlement decisions, development of plans, and delivery of services are performed and documented in accordance with VA regulations, VR&E’s operations manual, and other directives. VR&E services are provided by field staff at 56 regional offices and about 300 satellite locations. The satellite locations include college campuses to help veterans successfully complete their training and find employment, as well as military sites to help servicemembers with disabilities as they begin their transition to veteran status and the civilian workplace. VR&E field positions include (1) VR&E officers who manage the program and its staff in each region; (2) vocational rehabilitation counselors who work directly with veterans to assess their entitlement, develop their employment plans, and manage their progress; and (3) staff to support the administration of the program. As of June 2019, 1,394 field staff members were administering the VR&E program, of which nearly 75 percent (1,026) were counselors. From September 2013 to June 2019, VR&E’s total caseload peaked in fiscal year 2016 with almost 135,000 participants (see fig. 1). Over the same period, the number of counselors changed little until 2019. In 2019, the number increased after VA hired an additional 88 counselors in response to a provision in an appropriations law suggesting that the agency aim to serve 125 veterans or fewer per full-time equivalent counselor. The increase in staffing helped reduce the average caseload of 130-141 cases per counselor during fiscal years 2013 through 2016 to 113 cases in June 2019 (see fig. 2). Counselors Generally Considered Common Factors When Developing Veterans’ Plans but Noted Inconsistent Application of Those Factors Likely Occurs Counselors in Our Review Generally Considered a Set of Common Factors When Developing Plans VR&E counselors consider a set of common factors, including the veteran’s disability or disabilities, interests, and local labor market conditions, when developing and approving veterans’ employment plans. Program regulations require an assessment of some of these factors when the veteran is initially evaluated. VR&E quality review data from fiscal years 2016 through 2018 suggest that counselors generally documented certain plan considerations during the evaluation. For example, in 98 percent of the 1,080 cases VA reviewed for accuracy in fiscal year 2018, counselors documented the veteran’s service needs based on their functional limitations. In 95 percent of cases, counselors documented that they assessed the veteran’s abilities, aptitudes, and interests. Lastly, in nearly 99 percent of cases, counselors documented that the veteran was involved in vocational exploration activities such as career searches and labor market research. During our more focused review of how counselors developed plans for a non-generalizable sample of 34 VR&E case files, we found that counselors generally documented a set of common factors. Consistent with program guidance stipulating that counselors are to consider a veteran’s service needs, abilities, aptitudes, and interests, we identified common consideration factors including one’s functional limitation from disability, prior education, aptitude results, and career interests. Table 1 presents these factors and the number of files in which the factors were documented. Our case file review found that 30 of the 34 counselors also documented the estimated cost of VR&E employment plans. According to testimony from a veteran service organization, many VR&E participants are dissuaded by their counselor from pursuing education at a top tier university because of cost. VA’s VR&E operations manual states that if more than one local training or educational facility will meet a veteran’s needs, counselors must justify their decision to select a school that is more expensive than the least costly one. Counselors are not required to document all of the educational facilities that would serve a veteran’s needs; therefore, we could not determine the extent to which counselors chose the lowest cost facility. Counselors we interviewed in each of the three regional offices we visited said that while mindful of cost, they strive to develop employment plans that best meet the needs of the veteran. For example, counselors at one regional office described a situation in which a higher priced school was chosen because the school offered smaller class sizes that better suited the veteran’s particular mental health conditions. Of the 34 files we reviewed, the annual plan cost exceeded $25,000 in 3 cases. Counselors we interviewed said that they considered the veteran’s career interests but weighed these interests against other factors, such as the veteran’s functional limitations and information about the local labor market. All 34 plans we reviewed aligned with the veteran’s stated career goals, though in some cases the veteran’s goals evolved after talking with the counselor about alternative occupations. In a few instances among these cases, the final plan’s career goal was notably different from the initial goal that the veteran had stated on the program intake form. Table 2 presents examples of how a plan can evolve as a result of career exploration activities and conversations between the veteran and their counselor. Counselors in Our Review Stated They Strive to Develop Individualized Plans but Acknowledged That Some Unintended Differences in Plans for Similarly-Situated Veterans Likely Occurs Counselors we interviewed described how veterans’ employment plans are individually designed to suit a veteran’s needs and, as a result, may differ from one another even when veterans have similar goals, characteristics, and circumstances. In some instances two veterans may appear to be similar, but may actually differ in some critical respect that results in appropriate variation across plans. A common difference among veterans is the geographical location where they are seeking employment. Counselors described how a veteran may be encouraged to explore an occupation with many job opportunities within a specific region, while a veteran with similar characteristics and interests living in a different area may be dissuaded from pursuing the same occupation for a lack of job opportunities in the area for that occupation. Local labor markets may also drive the need for a certain type of educational credential. For example, counselors said that some veterans will be competitive in certain labor markets with a bachelor’s degree, while others living in a different region with a more educated population may need a master’s degree. Likewise, they said that certain occupations, such as certified public accountants or school teachers, may require different forms of credentialing in different states. Other characteristics of individual veterans may also cause counselors to develop different plans for veterans who appear to have similar circumstances. One counselor we interviewed described a scenario in which one veteran who received a high score on an aptitude test for reading comprehension skills might obtain a certain employment plan while another veteran who received a much lower score would be steered toward a different plan. If the veterans were to compare their final plans, but were unaware of the differences in their aptitude test scores, they could perceive inconsistent treatment. Counselors also described how conversations they have with veterans as they work to develop employment plans can reveal other character traits, such as interpersonal skills, which can lead them to suggest different plans to two otherwise similar veterans. The counselors said that such conversations play an important role towards the development of successful plans. However, counselors we interviewed in each of the three regional offices we visited acknowledged that unintended variation likely occurs across plans developed for similarly-situated veterans. They explained that the reasons for such potential inconsistency can include (1) the prominent role professional judgment plays in the program and the potential for unintended bias, (2) counselors’ different VR&E experience levels, and (3) variations in regional offices’ policies. Judgment and bias. The counseling role is inherently subjective and requires counselors to use their professional judgment in each case. The VR&E operations manual describes counselors’ responsibilities in broad terms, stating that counselors are to guide and assist the veteran in making an informed decision on an appropriate plan based on the veteran’s abilities, aptitudes, and interests. According to counselors we interviewed, professional judgment enables them to develop a plan that is best suited for the veteran’s unique needs, although it also introduces the potential for personal bias and inconsistent plans for veterans with similar circumstances. For example, a counselor we interviewed cited a case in which he saw the need to develop a plan that allowed for a school that was closer to a veteran’s home over other, less costly options because of his sensitivity to the veteran’s childcare responsibilities. Another counselor in the same office may not have seen the need for that accommodation. Further, counselors we interviewed said that some of their colleagues may be more comfortable suggesting that a veteran reconsider his or her career goal given circumstances such as the veteran’s disabling conditions or the local labor market. They explained that while some counselors would be hesitant to make the veteran unhappy, and possibly angry, other counselors would be more inclined to work through the conflict. Counselors said that they try to mitigate inconsistency by asking their fellow counselors to weigh in on these sorts of judgments, either informally, or at periodic information-sharing meetings. Counselor experience. Although all counselors have at least a master’s degree in rehabilitation counseling or a related field, differences in counselors’ levels of VR&E experience may affect their approach to plan development. Counselors at two regional offices noted that the focus of the VR&E program has oscillated between education and employment, with employment being the current primary focus. They said, as a result, a counselor’s general approach to plan development could be influenced by the prevailing focus that existed at the time he or she was hired. Counselors also said that, in general, counselors with more experience will tend to approach plan development differently than a less seasoned counselor because they will apply lessons learned from serving many other veterans. For example, one counselor said that, based on years of prior experience and observation, he has developed a better understanding of which local educational programs offer veterans the best chance for success and which do not. He said while he is able to apply his institutional knowledge and experience to do what is best for veterans, a less experienced counselor may not have the same level of knowledge, which could lead to inconsistent plans for veterans with similar circumstances. According to counselors we interviewed, because of the recent hiring of new counselors to meet caseload targets, differences in VR&E experience among counselors may be more pronounced at this time. Regional office variation. Differences in administrative policies specific to individual regional offices may also contribute to inconsistent plan development. For example, according to program officials, to ensure the soundness of employment plans in the local labor market, some regional offices require management to approve plans involving a master’s degree, while others do not. Counselors in one region told us that requiring management approval might dissuade a counselor from developing a plan focused on a master’s degree because of the time the extra step would require. They acknowledged that this sort of approval policy could cause inconsistency across counselors’ plans and also cause a discrepancy in the number of master’s degree programs being approved at one regional office versus another. In general, the large number of variables involved in the development of employment plans may complicate the ability to determine the extent to which differences among counselors lead to inconsistent plans among veterans. Counselors we interviewed said that given the subjective nature of the program, such inconsistency is likely. However, counselors cautioned against making the plan development process overly structured and formulaic. In their view, a more restrictive approach would eliminate the flexibility that they need to generate plans that suit each veteran’s unique needs. VA Trains Counselors and Monitors Their Performance but Does Not Monitor the Consistency of Employment Plans VA Trains Counselors to Develop Sound Employment Plans for Veterans VA trains counselors on developing sound and complete employment plans for veterans. New counselors receive a series of training courses that are developed and deployed through VR&E’s central office, and then receive additional courses and mentorship that are delivered through the regional offices. Course topics for new counselors include understanding vocational impairments, developing a rehabilitation plan, and documenting a narrative of the plan. The formal training emphasizes that plans should be individualized to accommodate the veteran’s rehabilitation needs, abilities, aptitudes, and interests. Collectively, these trainings take up to 80 hours. As of 2019, experienced counselors—those on the job for at least a year—take up to 20 hours of refresher training each year determined according to how they score on an annual assessment. The assessment evaluates counselors’ technical competencies such as knowledge of relevant regulations, vocational assessment and evaluation, and case management. If a counselor scores low on a particular topic, related courses are identified for the counselor to complete. In designing training for counselors, VA followed principles for strategically developing training that are consistent with a related guide for federal managers. For instance, VA obtained and considered input from multiple sources—including field advisory committees, quality assurance reviewers, and internal site visit auditors—to identify needs for counselor training. For example, questions and input from the field about a policy clarification led to a training about veterans’ entitlement to VR&E services. In addition, VA built flexibility into its training curricula for counselors so they could receive training on emerging topics such as implementing new policies throughout the year as needed. VA also has evaluated its training efforts in multiple ways. For example, it has evaluated training courses by surveying counselors to get immediate feedback and by checking with attendees and their supervisors to gauge improvements in skills and knowledge. VA Checks If Plans Are Complete but Does Not Monitor Consistency among Counselors VA monitors employment plans to ensure that they are complete, but does not check for consistency among counselors for veterans with similar circumstances. Quality reviews occur nationally as well as locally at each regional office. The purpose of the national reviews is to monitor the quality of regional offices’ work such as plan development, whereas the purpose of the local reviews is to help evaluate the performance of individual counselors. Nationally, a centralized quality assurance team monitors the completeness of regional offices’ VR&E entitlement decisions, employment plans, and service delivery by reviewing a randomly-selected subset of case files from each regional office on a monthly basis. Among other criteria, reviewers check whether a veteran’s plan identified goals and objectives, included an employment focus, and incorporated the veteran’s need for various services. Locally, VR&E officers or their designees are to review plans using the same criteria. Officers are supposed to review at least three cases per counselor per quality category (e.g., accuracy of evaluation, planning and rehabilitation services) per quarter. Reviewers do not check for consistency among counselors for similarly-situated veterans, at either the national or local levels. VR&E officials we interviewed identified challenges to completing and monitoring local reviews, but VA is addressing these challenges. According to the VR&E officer in each of the three regional offices we visited, it is difficult to complete local reviews given system limitations and their other job responsibilities such as implementing case management initiatives. They said that it is likely that some officers are not completing the required reviews while others are conducting them with varying degrees of thoroughness. Historically, VA has not identified the specific cases VR&E officers are supposed to review locally. Consequently, VA could not determine if VR&E officers conducted the requisite number of reviews or whether officers were selecting cases for quality review uniformly and fairly. In June 2019, during the course of our review, VA began a pilot in five regional offices to centrally and systematically identify the cases officers are to review to gauge individual counselors’ performance. The new process and system are intended to help officers conduct and track local reviews as well as to help VA monitor the completion of local reviews. VA plans to expand this process to all regional offices in fiscal year 2020. Although VA trains counselors to develop complete employment plans and reviews the completeness of some plans, it does not monitor the consistency of plans among different counselors. The code of professional ethics for rehabilitation counselors calls for counselors to be fair in the treatment of all clients and to provide appropriate services to all. In addition, one of the objectives of VR&E’s central office is to provide training and guidance to ensure high performance and consistency among field staff. Several veteran service organizations have testified at congressional hearings that VR&E is marked by inconsistent treatment of similarly-situated veterans. For example, one testimony cited veterans who allegedly received different plan approvals, such as access to graduate level education, merely on the basis of their counselor. Unlike for VA staff members who work on disability claim decisions, VA does not compare the output of VR&E counselors by, for example, analyzing responses to identical hypothetical cases for training or monitoring purposes. As a result, in addition to missing a training opportunity for counselors about employment plan development, VA does not know the degree to which inconsistency among counselors occurs. For example, the agency does not know the extent to which counselors would agree to a particular veteran’s pursuit of a master’s degree through VR&E. Moreover, VA cannot respond in an informed way—and take mitigating steps if warranted—to criticisms of subjectivity in the program. VR&E officials explained that the agency has not yet conducted such a comparative analysis because of other priorities, but agreed that it could do so particularly through its training efforts. Conclusions VA uses several training and monitoring practices to help ensure that VR&E counselors develop employment plans that help veterans with disabilities obtain and sustain employment. In approving these plans, VR&E counselors use their judgment and discretion fostered in part by their formal education and professional experience in vocational rehabilitation. While our review of a non-generalizable sample of 34 cases found that counselors generally considered common factors in developing employment plans, counselors we interviewed nevertheless acknowledged that counselors may apply the factors differently because of their varying backgrounds and experience levels. The variability of counselors’ experiences and veterans’ circumstances may make it difficult to determine the full extent of any inconsistency. However, taking steps to examine the prevalence and type of any inconsistency among counselors who, for example, consider the same hypothetical case, would better position VA to mitigate any unfair differences in plans for veterans with similar circumstances. An understanding of how effectively and consistently counselors assist veterans will be even more important in the coming years as VA fully integrates the new counselors hired to decrease the average caseload. Recommendation for Executive Action The Secretary of VA should ensure that the Director of VR&E assesses the consistency of VR&E plans among counselors and takes mitigating steps if results warrant. For example, as part of its training efforts, VA could have counselors respond to identical hypothetical veteran cases and, if unfair inconsistencies in plans result, the agency could enhance training on plan development. (Recommendation 1) Agency Comments We provided a draft of this report to VA for comment, and its written comments are reproduced as appendix I in this report. VA concurred with our recommendation and said that VBA will develop a consistency study of VR&E plan development. It emphasized that no two veterans are the same. It also provided technical comments which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report will be available at no charge on the GAO website at https://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7215 or curdae@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. Appendix I: Comments from the Department of Veterans Affairs Appendix II: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the individual named above, Mark Glickman (Assistant Director), Joel Green (Analyst in Charge), and David Perkins made significant contribution to the report. In addition, Jennifer Cook, Holly Dye, Alex Galuten, Monica Savoy, Mimi Nguyen, Almeta Spencer, Jeff Tessin, Rosemary Torres Lerma, and Sonya Vartivarian made key contributions.