Background Federal Agencies and Programs that Assist Homeless Veterans VA, HUD, and DOL are the federal agencies that provide programs specifically aimed at assisting homeless veterans. They are among the 19 federal member agencies of USICH—an independent establishment within the Executive Branch charged with coordinating the federal response to homelessness and creating a national partnership at every level of government and with the private sector to reduce and end homelessness nationally. USICH, VA, and HUD have jointly established criteria and benchmarks to guide communities that are taking action toward being certified as having ended veteran homelessness. USICH stated that an end to homelessness does not mean that no one will ever experience a housing crisis again, but that every community will have a systematic response in place to prevent homelessness whenever possible and ensure that homelessness is a rare, brief, and non-recurring experience. VA, HUD, and USICH coordinate their efforts towards this goal through a working group, Solving Veterans Homelessness as One, which was established in 2012. We identified 16 federal programs that target services specifically to veterans who are homeless or are at risk of becoming homeless. As shown in table 1, the programs provide permanent and transitional housing, health care, employment assistance, and supportive services, such as assistance with rent, utilities, and moving costs. Twelve of the programs are administered solely by VA, two are jointly administered by HUD and VA, and two are administered by DOL. VA’s and HUD’s homelessness programs follow the principles of “Housing First,” which is intended to provide housing without any preconditions and barriers to entry such as sobriety, treatment, or service participation requirements. The largest of these programs were HUD-VASH, GPD, and SSVF. In fiscal year 2019, HUD-VASH reported over $1 billion in obligations; GPD reported obligations of over $234 million; and SSVF reported $380 million in obligations. Role of Local VAMCs, Service Providers, and Other Entities Homeless veterans can access program services in several ways, including through: VA Medical Centers (VAMCs). Program services can be provided to homeless and at-risk veterans at their local VAMCs. Service providers. Veterans may also obtain services through local, state, or nonprofit organizations in the community, some of which receive grants from federal agencies to fund program services. Public housing agencies (PHAs). Housing vouchers are administered to homeless veterans by PHAs, which are HUD-funded city, county, and state agencies. VAMCs, service providers, and PHAs may coordinate through Continuums of Care (CoC), which are composed of stakeholders in a geographical area that, among other things, coordinate to provide homeless services, apply for grants, set local priorities, and collect homelessness data for all homeless populations. Each year, HUD awards CoC program funding competitively to nonprofit organizations, states, and other local recipients. The CoC is responsible for its operation and developing and implementing its plan and strategies to prevent and end homelessness. Additionally, the CoC must choose an entity to operate the local information system used to collect client-level data and data on the provision of housing and services to homeless individuals and families and those at risk of homelessness (referred to as the Homeless Management Information System). The CoC also designates an entity that prepares and submits the CoC program application for HUD funding (referred to as the collaborative applicant). HUD requires each CoC to establish and operate a centralized or coordinated assessment system (referred to as Coordinated Entry). This system may include implementing a “no wrong door” approach in which a homeless family or individual can show up at any homeless housing and service provider in a geographic area and get screened for services using the same assessment tool (see figure 1). The goal of Coordinated Entry is to ensure that people experiencing a housing crisis within a CoC are quickly and consistently assessed and referred for services. HUD officials stated that Coordinated Entry is a process that was first developed by some CoCs based on best practices. In 2017, HUD adopted and codified requirements for all CoCs to participate in Coordinated Entry. That same year, VA published requirements for VAMCs to participate in Coordinated Entry. VA and Selected Service Providers Reported Facing Challenges Related to Meeting Veterans’ Needs, Limited VA Staffing, and Other Factors Meeting Veterans’ Needs and Other Factors Create Challenges, According to VA and Service Providers According to VAMC staff and service providers we interviewed, they faced challenges serving homeless veterans and those at risk of becoming homeless due, in part, to the additional level of service and support that some veterans need. For example: Substance use and mental illness. Substance use disorders and mental health issues such as post-traumatic stress disorder (PTSD) and depression, are among the most complex issues many homeless veterans face, according to USICH. In 2018, USICH reported that 28 percent of homeless veterans that receive VA-provided health care have been diagnosed with depression. Thirteen percent have been diagnosed with PTSD. Further, 19 percent struggle with alcohol abuse and 20 percent with drug abuse. VAMC staff and service providers told us that addressing the complex nature of these conditions is often a challenge for them. For example, one SSVF provider told us that it is challenging to find housing for veterans with mental health or substance use disorders; further, HVCES staff at one VAMC told us that employment programs for the general population may not be suitable for clients with these disorders. HUD-VASH staff at one VAMC told us that there are not enough mental health providers in the VA system. Overall, staff from three VAMCs from the GPD and HVCES programs, five GPD service providers, and three SSVF service providers cited challenges related to substance use and mental illness. Aging homeless veterans. In 2018, USICH reported that the number of homeless veterans who were 62 and older increased by 54.3 percent between 2009 and 2016. VA officials told us that this trend is expected to continue and that this population has increased, in part, because the services that VA offers are not targeted to aging veterans. According to VA officials, there is a similar aging trend in the general veteran population. HUD-VASH staff at three VAMCs, HVCES staff at one VAMC, and GPD staff at two VAMCs told us that aging veterans require a higher level of care than what existing programs may be able to fully address. Some of these veterans may suffer from ambulatory and cognitive issues and have difficulties living alone but cannot afford an assisted living arrangement. HUD-VASH staff at one VAMC we visited told us the VAMC was able to hire five occupational therapists to assist aging clients with their specialized needs. Further, VA officials told us that HUD-VASH is collaborating with VA’s Geriatrics and Extended Care programs office to explore how aging homeless veterans can be served through other programs, such as the Medical Foster Home and Community Residential Care programs. This collaboration would allow VA to provide funding for services while the HUD-VASH voucher would pay for housing costs. VA officials also told us that they are working to market HUD-VASH to developers and funders to increase the development of project-based HUD-VASH housing. This would give the program a dedicated housing stock and better serve subpopulations of veterans, such as veterans who are elderly or suffer from mental illness. Resistance to program participation. According to HCHV staff at two VAMCs, HVCES staff at three VAMCs, two PHAs that administer the HUD-VASH voucher program, and five service providers (two GPD, two HVRP, and one SSVF), a key challenge in addressing the needs of homeless veterans is their resistance to participating in a program, particularly if it places restrictions or requirements on them. This issue makes it challenging for outreach and treatment teams to deliver services. In addition to the challenges cited above, veterans must meet certain eligibility requirements to participate in homeless assistance programs which if not met can present challenges to VAMC staff and service providers when providing veterans with services. For example, veterans must meet certain criminal history requirements to be eligible for HUD- VASH. HVCES staff from one VAMC, DCHV staff from two VAMCs, HUD-VASH staff from another VAMC, and three service providers (one GPD and two SSVF) also told us that it is challenging to find housing and employment for homeless veterans with legal or criminal problems and landlords may be resistant to working with them. One PHA that works with HUD-VASH and one SSVF service provider told us that a veteran’s ineligibility for VA health care services also presents a challenge to them. This is because a number of VA homeless programs require a veteran to be eligible for VA health care benefits as a condition to enrollment. Generally, veterans are eligible to receive VA health care benefits if they served in the active military, naval, or air service and were discharged under conditions other than dishonorable. Therefore, according to VA officials, veterans with dishonorable discharges cannot access VA homeless assistance programs and veterans with other-than- honorable discharges may have limited access to them. In addition to meeting the discharge status requirement, a person must also meet the definition of “veteran” to be eligible for VA health services. However, the definition of “veteran” depends on factors including length of service and if the individual served on active duty or was part of the National Guard or the Reserves. Therefore, even if an individual has the appropriate discharge status to be eligible, they may not meet other eligibility requirements for VA health benefits. One service provider told us that they have to work on alternative solutions to help veterans that do not meet eligibility requirements. In 2017, VA in partnership with HUD implemented a flexibility within the HUD-VASH program—the HUD-VASH Continuum—which according to VA officials, will permit PHAs to make up to 15 percent of their total HUD- VASH allocation available to veterans who are ineligible for VA health care services, with some exceptions. According to VA officials, this expands the availability of permanent supportive housing to service members who are not eligible for VA health care. In addition, the House and Senate are considering bills to expand HUD-VASH eligibility. DOL has also implemented statutory changes to HVRP eligibility requirements to provide veterans with better access to job training programs. The program’s eligibility requirements have been broadened to include veterans participating in the HUD-VASH, Tribal HUD-VASH, and SSVF programs, and other veterans that were not previously eligible. Broader Challenges, Such as Limited VA Staffing and Affordable Housing, Affected Assistance, According to VA and Service Providers VAMC staff and services providers cited broader challenges—not specific to veterans or the assistance programs themselves—as impacting their ability to provide assistance to homeless veterans. Those challenges include VA staffing issues and external factors, such as the lack of affordable housing and limited transportation options. VA staffing shortages. VA officials, HVCES staff at three VAMCs, and HUD-VASH and DCHV staff at two VAMCs told us they faced difficulties with recruitment and retention, which have led to persistent understaffing. For example, staff at four VAMCs for the HUD-VASH, DCHV, and HVCES programs told us that the hiring and onboarding process can often take a long time, and by the time an offer is finalized, qualified applicants have moved on to other jobs. DCHV staff at two VAMCs and HUD-VASH staff at three VAMCs cited understaffed human resources offices and a taxing approval process as contributing factors. HUD-VASH staff from one VAMC told us that it is difficult to fill some positions because the outreach work requires extensive travel within large geographic areas. Further, they told us that in high-cost areas, the VA’s local pay scale is not high enough to attract new recruits for case manager positions. HUD-VASH staff at one VAMC indicated that they have not been fully staffed for several years. Limited staffing may limit the number of veterans who can be served, according to VA officials. For example, DCHV program staff at one VAMC told us that they had to close 83 beds because there is not enough staff to keep them operational. One PHA working with the HUD-VASH program told us that VA’s staffing challenges create a bottleneck of services to clients while staff at one VAMC working with SSVF told us that high turnover of program staff is disruptive for clients. Overall, staffing shortages were cited as a challenge by VAMC staff for several programs: HUD-VASH, DCHV, HCHV, GPD, and HVCES. VAMC staff we interviewed have taken some steps to limit the impact of staffing issues. For example, at one VAMC, staff from the HCHV and GPD programs have conducted cross-training so they can back each other up when staffing shortages occur. Two other VAMCs have brought in staff from other locations to help with the workload or have developed an action plan to address employee burnout. Our past work has highlighted VA’s staffing challenges, including recruiting and retaining clinical staff. For example, we previously reported that difficulties in recruiting and retaining skilled health care providers and human resource staff at VAMCs make it difficult to meet the health care needs of more than 9 million veterans. We have also previously reported that, in addition to high attrition and increased workload, human capital shortfalls can lead to burnout among the staff whose job it is to implement these programs. Housing cost and availability. Limited and high cost housing exacerbate the other challenges VAMC staff and service providers identified. For example, HUD-VASH staff at one VAMC told us that even with subsidies, it is difficult for veterans to obtain housing because HUD-VASH vouchers may not be sufficient to cover rent. HUD-VASH staff from one VAMC and one PHA we interviewed told us that because housing costs are rising, and housing in metro areas remains limited, expensive, and competitive, veterans have fewer housing options available to them. Limited housing was cited as a challenge by VAMC staff and service providers for HUD-VASH and SSVF programs in all types of locations—urban, suburban, and rural areas. Finding and recruiting landlords is a significant challenge in getting veterans housed, according to HUD officials, HCHV staff at one VAMC, HUD-VASH staff from two VAMCs, two PHAs, and five SSVF service providers. According to HUD-VASH staff from one VAMC and SSVF staff from another VAMC, the demand for housing exceeds supply and landlords have few incentives to accept homeless veterans. HUD-VASH staff from one VAMC, one PHA, and one SSVF service provider also told us that some landlords perceive veterans to be risky because some have criminal records or substance use disorders, and may be reticent to work with them out of fear of incurring damages to their property. Some service providers have taken steps to create incentives for landlords to participate in VA’s programs. For example, HUD-VASH staff at one VAMC told us that local providers may partner to cover moving fees for veterans and encourage landlords to accept veterans’ housing vouchers. One PHA has put together landlord forums and is working to build relationships with landlords in their communities. Another PHA has held housing tours and fairs to bring landlords and clients together. VA has also implemented program changes to help address the lack of affordable housing. For example, the new Shallow Subsidies initiative that became effective in September 2019, allows SSVF service providers to provide very low-income veteran families a rental subsidy for a two-year period without requiring recertification. The two-year period ensures no reduction in subsidy even if a recipient’s income situation improves within that time frame and the family is no longer considered very low income. This provides a strong incentive for employment gains because the assistance is not dependent on income during this two-year period. Launched in 2018, VA’s Rapid Resolution initiative is another solution that is designed to prevent or resolve homelessness by providing immediate assistance when a veteran enters an emergency shelter system—such as by offering landlord mediation and conflict resolution, or connecting the veterans to support networks in other places. According to VA officials, Rapid Resolution is being implemented through VA’s SSVF program, in coordination with HUD and USICH. Limited resources (other than staffing). HUD-VASH staff at one VAMC told us they are short on equipment like laptops, office supplies, and office space. Additionally, HUD-VASH staff at another VAMC told us they do not have access to government cars for work- related travel. One HVRP service provider told us that case managers do not have enough vehicles to travel long distances or to remote locations to meet clients. To make up for shortages in resources, one SSVF service provider told us that it develops partnerships with local programs to meet the needs of the client. HUD-VASH staff at one VAMC indicated that they have communicated issues of insufficient resources to their leadership, but the issues have not been addressed to date. Resource limitations were cited as a challenge by HUD-VASH staff from two VAMCs, GPD staff from three VAMCs, DCHV staff at two VAMCs, HCHV staff at two VAMCs, and four service providers (two GPD and two HVRP). Transportation limitations. According to VAMC staff and service providers, the lack of transportation for veterans is a significant challenge for some programs. For example, according to DCHV and HVCES staff at one VAMC, HUD-VASH staff at another VAMC, and one GPD and two HVRP service providers, some veterans may not have vehicles or may live in areas with limited public transportation systems. This makes it difficult for the veterans to access resources, go to job interviews, or secure transportation for jobs. Some service providers told us they make alternative arrangements for their clients to help address these issues. For example, one HVRP service provider told us they might drive veterans to interviews or arrange for public transportation. DCHV staff at one VAMC and HVCES staff at another VAMC told us that they work with community partners to provide alternatives like shuttle services and bus passes. Additional challenges related to specific programs we reviewed are discussed in appendix II. Homeless Assistance Programs for Veterans Overlap in Services, but Address Different Needs Overlap Exists Among Some Program Services for Homeless Veterans We reviewed the services provided, eligibility requirements, and population served by the 16 programs that exclusively target homeless veterans to identify duplication and overlap. We determined that there is no duplication among the programs, but identified overlap across some program services. Specifically, we identified 18 main services that are commonly offered across the 16 programs and found that at least six of those services overlap across two or more programs (see figure 2). However, we also found these programs differed in meaningful ways, for example in terms of the different types of homeless veterans served or specialized services or focus. As we have previously reported, fragmentation, overlap, and duplication exists across the government, which can present benefits and challenges. Duplication occurs when two or more programs provide the same services to the same beneficiaries. Overlap occurs when two or more programs offer similar services to similar beneficiaries. As shown in figure 2, 15 of the 16 programs overlap in two or more of the following services that they offer. Case management is a process for managing a client’s care that includes assessing the needs of the veteran and evaluating health care options and services to ensure those needs are met while maintaining a primary focus on resolving the veteran’s homelessness through permanent housing. Eleven programs provide case management services: HUD-VASH, Tribal HUD-VASH, HCHV, HCRV, H-PACT, SSVF, GPD, DCHV, CWT-TR, HVRP and VJO. Supportive services might include providing meals, counseling, child care, housing assistance, transportation, and other services essential for achieving and maintaining independent living. Six programs provide supportive services: HUD-VASH, Tribal HUD-VASH, GPD, SSVF, HVRP, and Stand Down. Outreach involves directly contacting veterans in need of homeless services and connecting them with housing, health care, and supportive services. Six programs conduct outreach: HCHV, HVRP, SSVF, CRRCs, VJO, and HCRV. Referrals are the most common way for homeless veterans to find out about program services available to them. Referral services include conducting an assessment of the clients’ needs, connecting them to the appropriate programs, and following up with the clients as well as documenting all referral activities. Six programs provide referral services: HVRP, SSVF, CRRCs, NCCHV, HCRV, and Stand Down. Employment services include help with creating job opportunities for veterans, job searches, interviewing, and other employment assistance. Three programs provide employment-related services: HVCES, CWT-TR, and HVRP. Rental subsidies are offered to veterans through vouchers and grants, which help subsidize rental costs. Three programs offer rental subsidies: HUD-VASH, Tribal HUD-VASH, and SSVF. Although we identified overlap in these services across 15 of the 16 programs, the programs differ in meaningful ways. Specifically, some of these programs serve specific subpopulations of veterans and some provide a specialized service that other programs do not offer. For example, of the 11 programs that offer case management services, one program provides medical care (H-PACT), while others provide services in different areas such as transitional housing (GPD), housing subsidies (HUD-VASH), supportive services (SSVF), preparing veterans for employment (HVRP) and outreach (HCHV). Other programs that offer case management services serve unique subpopulations of homeless veterans such as those with mental health or substance use issues (CWT-TR and DCHV), American Indians and Alaskan Natives living in or near reservations or other Indian areas (Tribal HUD-VASH), or justice- involved veterans in local jails (VJO) and state and federal prisons (HCRV). According to a VA directive, more than one case manager may be involved in care planning and service delivery for veterans with complex needs. In addition, staff at the six VAMC locations we visited told us that clients may be co-enrolled in more than one program and can receive case management services from each of those programs. Figure 3 illustrates how case management may overlap across programs, but each program provides distinct services to the veteran. Similarly, of the three programs that provide employment services, HVCES focuses on establishing partnerships with employers to develop job opportunities for veterans and connect them with community services, while HVRP helps the veteran prepare to pursue and gain meaningful employment. The CWT-TR program, on the other hand, focuses on veterans with more complex issues such as substance use, mental health issues, and challenges in obtaining or sustaining employment that may accompany these conditions. Similar meaningful distinctions in subpopulations of beneficiaries and services exist across the programs that provide other types of services to homeless veterans that overlap— supportive services, outreach, referrals, and rental subsidies. Additional information on program differences, including information on program beneficiaries and services, can be found in appendix II. Overlap in Program Services Presents Potential Benefits and Challenges As we previously reported, in some cases it may be appropriate or beneficial for multiple agencies or entities to be involved in the same programmatic or policy area due to the complex nature or magnitude of the effort. Overlapping programs may also facilitate access to services because persons experiencing homelessness are not steered toward one specific point of entry and, in contrast, can access services through several entry points. However, when multiple programs overlap, there is also a risk of program administrators making inefficient use of available resources if they do not coordinate their efforts. For example, according to VA officials, overlap may result in operational costs if the overlapping services are not coordinated well. Table 2 describes some of the potential benefits and challenges of overlap in services for homeless veterans, as identified by agency officials, VAMC staff, and others we interviewed. Effective collaboration among agencies and service providers can help address some of these potential challenges and may help avoid the potential inefficiency that overlapping services may create. VAMC staff and service providers told us that they have taken steps to limit duplication where appropriate. Additionally, they told us that they collaborate and communicate with each other to avoid or mitigate overlap. VA has also issued guidance directed at enhancing coordination between its homeless programs and eliminating or reducing duplication of services, including the following: Veterans Health Administration (VHA) Directive 1110.04, Integrated Case Management Standards of Practice. This guidance states that case management services should be coordinated, collaborative, and veteran-centered throughout the VHA. It also directs case management teams to develop procedures and processes to support cost effective, high quality case management across the VAMC to eliminate duplication of services where appropriate. VHA Handbook 1162.09, Health Care for Homeless Veterans Program. Under the HCHV program, program coordinators are responsible for ensuring coordination of HCHV services with other homeless programs at the VAMC such as GPD, HUD-VASH, DCHV, VJO, HCRV, SSVF, and CRRCs. GPD’s Case Management Services Grant Program, Final Rule. This final rule stipulates that the case management grant may not be used for veterans receiving case management from certain other programs to ensure that there is no duplication of case management services. VHA Handbook 1162.01 (1), GPD. This guidance states that GPD liaisons are to ensure the coordination of care for homeless veterans in GPD-funded programs by following a plan that clearly delineates the roles of those responsible for the service provision to reduce duplication of services. VHA Handbook 1101.10 (1), Patient Aligned Care Team Handbook. This guidance directs staff to coordinate care in a manner that avoids unnecessary duplication. The following section of this report discusses how federal agencies collaborate more broadly on implementing federal homelessness assistance programs for veterans. Key Federal Efforts Incorporate Many, but Not All, Leading Practices on Collaboration We identified two key collaborative mechanisms that federal agencies use to help address veteran homelessness: (1) the Solving Veterans Homelessness as One (SVHO) working group, which coordinates VA, HUD, and USICH’s efforts at the national level, and (2) VA’s integration into Coordinated Entry, which seeks to ensure that homelessness services are coordinated at the local level. As shown in table 3 and as discussed in more detail below, both mechanisms follow leading practices for effective interagency collaboration we have identified in prior work, with some exceptions. Solving Veterans Homelessness as One According to USICH officials, in 2012, USICH convened the SVHO workgroup to coordinate with HUD and VA on key priorities and maximize efforts to end veteran homelessness. SVHO serves as an interagency decision-making body that plans and executes strategic actions through goal setting, policy gap identification, communication, and action. The SVHO working group fully followed all seven leading practices for effective interagency collaboration that we identified in prior work. A discussion of our assessment follows: Defining Outcomes and Monitoring Accountability. Ending veteran homelessness is one of the national goals listed in USICH’s Federal Strategic Plan to Prevent and End Homelessness. SVHO’s work is organized to support this goal. USICH reports SVHO’s efforts in its annual Performance and Accountability Reports. For example, USICH reported that in fiscal year 2019, SVHO’s efforts led to supplemental guidance and coaching to help sustain the efforts of communities that had been certified as having ended veteran homelessness. Bridging Organizational Cultures. To operate across agency boundaries, SVHO members hold regular meetings. During these meetings, SVHO members have updated one another on each agency’s efforts, discussed strategic objectives, shared program data, and coordinated on technical assistance for service providers. SVHO also held a strategic planning retreat to discuss SVHO’s priorities. Clarifying Leadership. SVHO has a Strategic Decision and Coordination Team that serves as the decision-making body and includes leadership from VA, HUD, and USICH. The team’s decisions are made by consensus, and the role for facilitating the team rotates every four months among the three agencies. The Strategic Decision and Coordination Team’s responsibilities, which include providing strategic guidance on cross-agency issues, providing joint oversight and decision-making, and facilitating the approval of decisions from the individual agencies are outlined in SVHO’s charter. Clarifying Roles and Responsibilities. The SVHO charter outlines the roles and responsibilities of the Strategic Decision and Coordination Team and the Support Team, whose responsibilities include responding to priority projects and elevating issues requiring decision and coordination to the Strategic Decision and Coordination Team. Including Relevant Participants. SVHO members (USICH, VA, and HUD) are the relevant participants because they are the agencies centrally involved in implementing veteran homelessness programs. Identifying Resources. USICH, VA, and HUD contribute staff resources to the working group. Representatives from each of the agencies attend regular SVHO meetings to ensure continuity, provide the necessary subject matter expertise, and make decisions. SVHO has also developed resources to facilitate the group’s meetings, such as agendas to guide discussions. Updating and Monitoring Written Guidance and Agreements. In March 2020, SVHO revised its charter to remove outdated information and to reflect the group’s current structure and operations. The revised charter describes the purpose of establishing SVHO as a formal structure for coordination and decision-making (to enable member agencies to execute joint activities necessary for the goal of preventing and ending veteran homelessness), SVHO’s structure (the group is comprised of a leadership team and support team with various responsibilities), and operating procedures (which involve holding regular meetings). USICH officials told us it was important to have an updated charter that solidified the commitments of the member agencies to the group. VA officials added that updating the charter would help serve as a reminder of the group’s purpose. VA’s Integration into Coordinated Entry Coordinated Entry is a process designed to help communities prioritize people who are most in need of assistance by standardizing the assessment process, defining community-wide prioritization policies, and coordinating referrals, among other things. HUD established minimal requirements for Coordinated Entry in a 2012 Continuum of Care Program Interim Rule. HUD officials said they established additional requirements in 2017 in coordination with other federal agencies, including VA. VA also issued a memo in 2017 stating that VAMCs must be actively engaged in their local Coordinated Entry. Efforts to integrate VAMCs into Coordinated Entry fully followed five of the seven leading practices on effective interagency collaboration and partially followed the other two (Bridging Organizational Cultures and Updating and Monitoring Written Guidance and Agreements). A discussion of our assessment follows: Defining Outcomes and Monitoring Accountability. VA established requirements for the VAMCs as they integrate into Coordinated Entry, which include active engagement with the CoC, involvement with case conferencing, and aligning standardized assessments. VA has a checklist that VAMCs use to assess their compliance with Coordinated Entry requirements. According to VA officials, they monitor VA integration into Coordinated Entry through self- assessment checklists that VAMCs are required to submit monthly through an internal VA system. VAMCs are also required to submit monthly operation plans to track their progress. Bridging Organizational Cultures. As we previously reported, collaborating agencies should establish ways to operate across agency boundaries and address their different cultures. VA requires VAMCs to actively engage with all coordinated entry systems within their catchment area. VA has provided some guidance to help VAMCs operate across organizational boundaries as they integrate into Coordinated Entry, but this guidance is broad in some areas. For example, it instructs VAMCs to collaborate with local CoC leadership to establish a clear process for making and receiving referrals and to share aggregate program data with each of their communities as needed. But the guidance does not describe steps that VAMCs can take to do so. In addition, two service providers and staff from two VAMCs told us that it can be challenging to work with multiple CoCs because each has their own processes. Additionally, staff from three VAMCs and one CoC entity told us that staff turnover creates challenges in their coordinated entry systems, including impeding relationship-building among partners. VA’s guidance acknowledges that VAMCs may be working with multiple CoCs, but the guidance does not provide any best practices to help address this issue, nor does it expressly address relationship-building in light of staff turnover. Clarifying Leadership. As previously discussed, VA oversees the integration of the VAMCs into Coordinated Entry. Additionally, USICH and HUD officials told us there was an interagency working group on Coordinated Entry, where several agencies, including USICH, VA, and HUD, convened to discuss, among other things, what was happening in the field and barriers to Coordinated Entry implementation across all homeless programs, including those for veterans. HUD officials told us they also worked closely with VA to fully integrate VAMCs into Coordinated Entry. Clarifying Roles and Responsibilities. VA issued guidance that defined VAMCs roles in Coordinated Entry. For example, one or more representatives must be involved in the community planning process and in case conferencing, with sufficient knowledge and decision-making power to actively engage in each activity. Including Relevant Participants. All homeless assistance organizations should be involved in Coordinated Entry, according to HUD guidance. Coordinated Entry includes CoCs, VAMCs, service providers, and public housing agencies, among others. Staff from one VAMC, one service provider, and one CoC entity that we spoke with described their coordinated entry systems as being inclusive of all relevant stakeholders, including veteran homeless service providers. Identifying Resources. VA funded 86 Coordinated Entry Specialist positions through the HCHV program, of which 81 had been filled, as of January 2020, according to VA officials. Staff from two VAMCs and two CoCs told us that these new positions play an important role in VAMCs’ integration into Coordinated Entry because they serve as a liaison between the CoCs and the VAMCs. Additionally, VA requires that VAMCs dedicate a portion of VA resources (such as HUD-VASH vouchers or VA Homeless Program Residential Treatment beds) for their inclusion into the greater pool of homeless service resources that are accessed by veterans through Coordinated Entry. Updating and Monitoring Written Guidance and Agreements. We previously reported that agencies can strengthen their commitment to working collaboratively by formally documenting their agreements, and that those written agreements are most effective when regularly monitored and updated. As discussed earlier, VA has issued some guidance to help VAMCs integrate into Coordinated Entry. VA has also held webinar trainings and issued some program-specific documents, such as an SSVF Coordinated Entry fact sheet and a “frequently asked questions” document for HUD-VASH. VA has also provided technical assistance by request, according to agency officials. However, as noted earlier, VA’s guidance is broad in some areas and neither provides best practices to help VAMCs working with multiple CoCs, nor expressly addresses relationship-building in light of staff turnover. VA officials told us they do not have plans to issue additional guidance on Coordinated Entry because they believe their current guidance provides sufficient direction. However, several interviewees (staff from three VAMCs, one service provider, and one PHA) told us they need additional guidance on Coordinated Entry, specifically about how to better collaborate among partners. For example, staff from one of the VAMC’s said that while they understood that implementing Coordinated Entry required some flexibility, it would be beneficial if VA provided common parameters that communities could follow. Further, some VA guidance (such as the “frequently asked questions” document for HUD-VASH) may not be accessible by all service providers for VA’s homeless programs because it is stored on the agency’s internal system (the Homeless Programs Hub) or provided via technical assistance only upon request. Staff from two VAMCs stated that VA could better disseminate guidance. Additionally, one service provider and one PHA told us it would be helpful for VA to share best practices on collaboration used in other parts of the country. By providing additional information on how VAMC staff and service providers can collaborate with local partners, such as best practices, and making available guidance readily accessible, VA can help ensure that VAMCs and service providers are able to more effectively collaborate with other local providers to serve homeless veterans. Selected Programs Reported Meeting Most Targets, but Some Aspects of Performance Measurement Could Be Strengthened National Data Show Selected Programs Met Most Targets According to VA officials, since 2011, VA has focused on three primary outcome measures for the homelessness assistance programs we selected for review: 1) placement into permanent housing, 2) employment, and 3) negative exits from programs. DOL developed four critical measures for HVRP, including the placement rate for total enrollment, which tracks the total number of program participants employed in one or more jobs. VA and DOL officials told us they review their performance measures annually and adjust them as needed. National level performance data for fiscal years 2015 to 2019 show that five of the seven selected programs we reviewed have generally met their performance targets (see table 4). However, two programs, HUD-VASH and DCHV, have not met some of their targets. Specifically, in four of the last five years, HUD-VASH did not meet its targets for “percent housed in HUD-VASH housing” and “percent housed within 90 days.” In the last two years (2018 and 2019), HUD-VASH did not meet its targets for “negative exits”; however, VA had decreased the target for those years (making it more difficult to meet). DCHV did not meet its targets for “exits to permanent housing” for the last three fiscal years, and “negative exits” for two of the last five fiscal years. According to VA officials, factors that have affected VAMCs abilities to meet HUD-VASH performance targets—some of which are challenges identified by local VAMC staff and providers that we have discussed previously—include an insufficient number of case management staff, which has led to fewer veteran admissions into HUD-VASH and a lack of safe and affordable housing for veterans. VA officials told us that DCHV program outcomes have been affected by factors including discharges to other transitional housing programs (which would not be included under an exit to permanent housing) and limited affordable housing resources. To help improve program outcomes for HUD-VASH, VA officials told us they are focusing on increasing HUD-VASH voucher utilization, such as by using vouchers for non-Veteran Housing Administration eligible homeless veterans through the HUD-VASH Continuum program and expanding project-based HUD-VASH efforts (discussed previously). To improve DCHV program outcomes, VA officials said they are holding in- depth discussions with DCHV staff to highlight lessons learned from those VAMCs that are meeting performance targets. Performance Measurement Reflected Most Leading Practices, but Data Reliability and Communication Could Be Strengthened The performance measures used for the selected programs we reviewed reflected most of the attributes of successful performance measures that we identified in prior work (see table 5). VA’s measures fully reflected all six of these attributes. DOL’s measures fully reflected five attributes and partially reflected one, the reliability attribute. Performance measures that include these attributes are effective in monitoring progress and determining how well programs are achieving their goals. A discussion of our assessment of VA’s and DOL’s performance measures follows: Clarity. VA’s and DOL’s policies clearly state the names and descriptions of the performance measures we reviewed. The names and descriptions are also consistent with the methodologies that were used to calculate them. Measureable Target. VA and DOL have established quantifiable, numerical targets for their performance measures, which allows them to compare expected and actual results. VA officials told us they developed the targets for their measures by first obtaining baseline data and then looking at historical and projected performance. HVRP service providers identify their own targets during the annual grant competition process, according to DOL officials. DOL officials told us they provide some parameters, such as the national targets, to help providers develop their individual targets. Objectivity. VA’s and DOL’s performance measurement policies describe what is expected to be measured (for example, the percent housed and percent employed). They also indicate which specific population (veterans) and under what timeframes (the relevant reporting period). Baseline and Trend Data. Nearly all the measures have baseline and trend data for the last five fiscal years. The exceptions are measures that have been recently discontinued. Having baseline and trend data allows VA and DOL to monitor changes in program performance. Linkage. DOL’s performance measures for HVRP align with one of DOL’s agency-wide strategic objectives to provide veterans with resources and tools to gain and maintain employment. DOL officials told us that information about the measures is communicated to grantees through local officials, who review a data dashboard created by DOL officials at headquarters. VA’s performance measures are aligned with VA’s agency-wide goal to end veteran homelessness, as outlined in VA’s most recent strategic plan. VA officials told us they communicate information about the performance measures to the VAMCs and service providers through scorecards. Reliability. Measures reflect this attribute when they produce the same result under similar conditions. Reliability is increased when verification and validation procedures exist, such as checking performance data for significant errors by formal evaluation or audit. VA’s performance measures fully reflected the reliability attribute; DOL’s measures partially reflected it. VA officials told us they ensure data quality through the use of validation processes, error messages, and notifications that appear in real-time as data are entered. Additionally, there are dedicated program offices that work with the VAMC’s and service providers to monitor and reconcile data. Finally, VA’s policies describe steps that should be taken to review and verify the quality of the data. DOL officials told us they review HVRP performance data quality at different levels in the agency (regional and national) and use a data validation tool to identify potential errors. However, DOL officials acknowledge limitations with data quality, namely the lack of an electronic system to compile the data and the potential for human error when entering data into spreadsheets. Further, HVRP service providers may be unclear about the data quality steps to take because DOL’s performance measurement policies provide limited information on data reliability procedures. DOL officials stated that they have conducted webinar training on the data validation tool, but acknowledge that no written policy exits for the data validation process. Without guidance from DOL on the quality control processes that should be applied to performance data, service providers may not understand how to improve data quality and DOL may not have reasonable assurance that these performance data are the most accurate and reliable available. While VA’s measures reflected all the selected attributes of successful performance measures, including communicating linkage, we identified other areas where communication about these measures is not clear. For example, staff from three of the VAMCs we interviewed and two service providers described communication issues related to performance measures for four programs (HUD-VASH, GPD, HVCES, and DCHV). These issues included concerns that VA does not understand the realities on the ground that prevent VAMC staff and service providers from meeting the measures (such as limited housing availability) and VAMC staff being unaware they could use performance scorecards to drill down and learn more about why their performance targets were not met. Additionally, some VAMC staff and service providers we interviewed do not fully understand the measures. For example, DCHV and HCHV staff we interviewed from four VAMCs and three GPD service providers told us they have felt penalized for transitioning veterans from a VA homeless assistance program to another program or to substance abuse or mental health treatment because VA’s performance measures count these transitions as “negative exits.” According to VA officials, however, there are only three instances where participant program exits are counted as negative: 1) when participants are asked to leave for failure to follow rules; 2) when participants leave for failure to comply with program requirements; and 3) when participants leave without telling program staff. VA officials told us they have implemented processes to obtain quarterly feedback from VAMCs and service providers—through operation or actions plans—about the measures, including feedback about not meeting performance targets. However, HUD-VASH staff from one VAMC said that they have reported their concerns about not having information on how to improve performance to VA leadership and GPD staff from another VAMC and two GPD service providers said they have reported their concerns about how negative exits are measured, but the concerns have not been addressed. Additionally, staff from another VAMC were unaware that VA had a way for them to provide formal feedback about the performance measures, suggesting that VA’s feedback process and avenues of communication may lack clarity. We previously reported that improving the communication of performance information among staff and stakeholders can enhance or facilitate the use of performance information by agency managers. Performance information can be used to identify gaps in performance, improve organizational processes, and improve performance. Clearer communication by VA’s Homeless Programs Office about performance measurement—what performance measures capture and how to obtain and provide feedback—would help VAMCs and service providers better understand how their program data are used to measure performance and therefore how to improve performance, which could also help VA better assess program outcomes. Agencies Have Conducted Some Program Evaluations VA, HUD, and DOL published some annual reports during the last five fiscal years that monitored the performance of some of the selected homelessness assistance programs for veterans we reviewed. In addition, they conducted a limited number of evaluations to assess their overall effectiveness or impact and conducted other studies that examined other aspects of the programs, such as characteristics of program participants. Program evaluations are systematic studies that use research methods to address specific questions about program performance. We identified two program evaluations conducted by or on behalf of HUD and VA that assessed the impact of HUD-VASH. Published in 2016, the Family Options study examined how the effects of three types of programs—permanent housing subsidies (such as HUD-VASH vouchers), community-based rapid rehousing, and project-based transitional housing—compared with one another and with the usual care available to homeless families. Findings from the Family Options study indicated that giving people experiencing homelessness priority access to deep permanent housing subsidies, such as housing choice vouchers, benefitted program participants by improving housing stability. However, as discussed in the study, heads of households that received permanent housing subsidies experienced a reduction in employment in comparison to participants in other programs. The permanent housing subsidy also cost more than the other programs. The second study was the HUD-VASH Exit study. Published in 2017, the study was part of an effort to improve program effectiveness. It assessed how and why veterans exit the HUD-VASH program, identified obstacles to their obtaining and maintaining housing with a HUD-VASH voucher, described the value of services, and identified barriers to successful collaboration between VA and HUD in administration of the program. Among other things, the study found that the program was successful, as demonstrated by high rates of retention in housing, and that relationships with community partners and the ability to connect veterans to community resources contributed to successful outcomes. While the several other studies and reports we identified did not assess the impact of programs, some did analyze program performance or outcomes (for example, the agencies’ annual performance plans and reports), and others assessed specific aspects of the programs (for example, factors associated with exiting homelessness programs and characteristics of program participants). VA officials noted that resource limitations constrain their ability to conduct impact evaluations. However, they stated that in the future, they plan to evaluate new programs and models, such as the SSVF’s rapid resolution program (discussed previously). DOL officials told us they have commissioned an impact evaluation for HVRP, which is scheduled to be completed in 2022. The study is assessing the effectiveness of the HVRP program on improving homeless veterans’ employment outcomes and will build knowledge about program models including variations. We found that HUD and DOL have developed plans outlining the evaluations they plan to conduct and the steps they used to create their plans, but VA did not. VA’s National Center on Homelessness Among Veterans, which conducts research and assesses the effectiveness of VA’s homelessness programs, has an evaluation agenda listed on its website that describes the Center’s planned studies, but not the steps taken to develop the agenda and prioritize what studies to conduct. HUD and DOL have also developed policies describing the steps the agencies take to ensure evaluation quality and rigor. VA’s National Center on Homelessness Among Veterans, on the other hand, does not have written policies on evaluation quality. VA officials told us they ensure the quality and rigor of the Center’s work by submitting study results for publication through a peer-reviewed standard scientific protocol, consistent with other VA research, but had not yet developed formal written policies as their processes are well known in the Center. However, the Foundations for Evidence-Based Policymaking Act of 2018—enacted in January 2019—will now require VA and other agencies to, among other things, designate an evaluation officer who is to establish and implement an agency evaluation policy and assess the coverage, quality, methods, consistency, effectiveness, independence, and balance of the portfolio of evaluations, policy research, and ongoing evaluation activities of the agency. The Act requires agencies to develop written annual evaluation plans—that discuss steps taken to develop the plan such as how studies were prioritized—to be submitted with their annual performance plan. In June and July 2019, the Office of Management and Budget released its initial guidance on implementing the Act, and additional guidance is forthcoming. The Act also includes provisions for GAO to conduct studies to review agency implementation efforts. Conclusions VA, HUD, and USICH have taken significant steps to ensure effective collaboration between the agencies and among local service providers when addressing veteran homelessness. However, VA can help local agency staff and service providers better collaborate by fully incorporating leading practices for interagency collaboration. More specific and accessible information on how to collaborate with partners through Coordinated Entry, including on key activities such as making referrals and sharing data, could position local VA staff and service providers to better aid homeless veterans with services at the local level. Opportunities also exist for the agencies to improve some performance measurement procedures. Documenting its data quality processes can help give DOL reasonable assurance that these performance data are the most accurate and reliable available. Additionally, providing clearer communication about performance measurement—what the performance measures capture and how to obtain and provide feedback—can help VA ensure that VAMCs and service providers have a better understanding of how their program data are used in measuring performance (and how to improve performance), which may also help VA better assess program outcomes. Recommendations for Executive Action We are making a total of three recommendations, two to VA and one to DOL. Specifically: VA’s Under Secretary for Health should provide additional information, such as best practices, about how VA medical centers and service providers participating in Coordinated Entry can collaborate with local partners on key activities (for example, making referrals and sharing data) and ensure that this information and other resources are accessible to VA medical center staff and service providers. (Recommendation 1) The Assistant Secretary for DOL’s Veterans’ Employment and Training Service should document its data quality validation processes for performance data for the Homeless Veterans’ Reintegration Program and disseminate these processes to service providers. (Recommendation 2) VA’s Under Secretary for Health should clearly communicate with local VA staff and service providers about how it measures performance and how to obtain and provide feedback about performance measures. (Recommendation 3) Agency Comments and Our Evaluation We provided a draft of this report to DOL, HUD, USICH and VA for review and comment. DOL and VA provided written comments, which are reproduced in appendixes III and IV, respectively. HUD and VA provided technical comments, which we incorporated as appropriate. A USICH official stated that USICH did not have concerns with the proposed recommendations and had no additional comments on the draft. In its comments, DOL neither agreed nor disagreed with our recommendation that it document and disseminate its data quality validation processes for performance data for HVRP (Recommendation 2). DOL stated that it agreed with the importance of data quality validation processes and noted that it uses a data validation tool (discussed earlier in our report). In addition, DOL provided new information in its comments on the draft report, stating that the agency released a user manual and training video for field staff and grantees on the validation tool and provided a hyperlink to additional information, including the user manual. While the user manual outlines the steps for downloading the validation tool and how to run validation tests, it does not describe what validation tests are run or the data quality reviews that DOL officials told us occurred at the regional and national level, as discussed earlier in our report. Therefore, we maintain our recommendation that DOL document all of its data quality validation processes for HVRP performance data and disseminate them to service providers to give the agency reasonable assurance that its performance data are the most accurate and reliable available. VA agreed with our recommendations in its written comments (Recommendations 1 and 3) and outlined actions it plans to take to address them, including: Providing additional information, such as successful strategies, about how VAMCs and service providers participating in Coordinated Entry can collaborate with local partners on key activities and enhancing communication through monthly calls on Coordinated Entry collaboration, including case conferencing, streamlined referral processes, and data sharing that will be recorded and accessible any time by staff. Clearly communicating with local VA staff and service providers about how it measures performance and how to obtain and provide feedback about performance measures. VA’s target completion date for these actions is May 2021. In addition, the draft report we originally sent the agencies included recommendations to VA, HUD, and USICH to revise their SVHO working group charter. However, the agencies informed GAO that they had issued a revised charter in late March and VA and HUD provided a copy of the final charter. Based on our review of the charter, we revised our discussion of the charter in the report and removed the recommendations. We are sending copies of this report to the appropriate congressional committees, the Secretary of the Department of Veterans Affairs, the Secretary of the Department of Housing and Urban Development, the Secretary of the Department of Labor, the Executive Director of the U.S. Interagency Council on Homelessness, and other interested parties. In addition, this report will be available at no charge on GAO’s website at https://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-8678 or cackleya@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V. Appendix I: Objectives, Scope, and Methodology This report focuses on federal programs that provide services to veterans that are experiencing homelessness or are at risk of being homeless and their dependents. Our report (1) describes the challenges agencies and service providers reported experiencing in implementing selected programs that assist homeless veterans; (2) assesses the extent, if any, of overlap and duplication among programs; (3) evaluates how well federal agencies collaborate to address veteran homelessness; and (4) reviews what is known about the performance of selected programs. We identified a total of 16 programs that specifically target homeless veterans by reviewing agency reports, guidance, and other documentation and past GAO and Congressional Research Service reports. From these 16 programs, we selected 7 that we focused on for our first objective on program challenges and our fourth objective on program performance: Housing and Urban Development-Veterans Affairs Supportive Housing (HUD-VASH); Grant and Per Diem (GPD); Supportive Services for Veteran Families (SSVF); Health Care for Homeless Veterans (HCHV); Domiciliary Care for Homeless Veterans (DCHV); Homeless Veteran Community Employment Services (HVCES); and the Homeless Veterans’ Reintegration Program (HVRP). We selected these programs based on size (largest programs based on funding and the number of veterans served) and services offered (a mix of programs addressing a variety of needs). The results of our review of these programs are not generalizable. For all objectives, we selected and interviewed representatives from the following national advocacy organizations for homeless veterans and other knowledgeable groups to obtain subject matter context: the National Alliance to End Homelessness; the National Coalition for the Homeless; the National Coalition for Homeless Veterans; and American Legion. We judgmentally selected these groups based on their knowledge about homeless veteran policy issues, their ability to share perspectives on a variety of homeless veterans’ subpopulations, and their knowledge about federal homelessness grants. We also interviewed officials from the Department of Veterans Affairs (VA), Department of Housing and Urban Development (HUD), Department of Labor (DOL), and the U.S. Interagency Council on Homelessness (USICH). Additionally, we conducted semi-structured interviews with staff from local VA medical centers (VAMCs) and service providers implementing the selected programs we reviewed; public housing agencies (PHAs) that administer HUD-VASH vouchers; and Continuum of Care (CoC) entities across different locations. Specifically, we interviewed staff from six VAMCs (staff for the HUD-VASH, GPD, SSVF, HCHV, HVCES, and DCHV programs); six CoC entities; six PHAs; and 23 service providers (eight GPD providers, seven SSVF providers, two HVRP providers, two providers that were HVRP, SSVF, and GPD grantees, two providers that were HVRP and GPD grantees, and two providers that were HVRP and SSVF grantees). The results of these interviews are not generalizable. The locations where we conducted these interviews were: Atlanta, Georgia; Kansas City, Missouri; Long Island, New York; Los Angeles, California; Helena, Bozeman, Fort Harrison, and Box Elder, Montana; and Seattle, Washington. We judgmentally selected this sample of sites based on several factors. To select those locations, we started with the 67 communities that were designated as Priority 1 communities by VA in 2015. We then judgmentally selected six of those communities based on the following factors: (1) to reflect a mix of communities with high concentrations of homeless veterans and communities certified as having ended veteran homelessness; (2) to reflect geographic diversity (a mix of urban, suburban, and rural locations); (3) proximity of CoCs and VAMCs (to ensure we could interview both local VAMC staff and service providers); and (4) the presence of our selected programs (to cover as many programs as possible). To identify challenges agencies and service providers reported experiencing in implementing selected programs, we interviewed agency officials, VAMCs, service providers, and PHAs. Specifically, we first asked them a general question about what challenges they face. We then analyzed their responses to develop a list of challenges. A second analyst then verified the steps taken to develop the list of challenges. We also reviewed agency reports, program documentation, and available information on trends on homeless veterans and the general homeless population. To determine the extent of duplication or overlap across programs, we reviewed agency guidance, program descriptions, and other documentation to obtain information on program services and beneficiaries for the 16 veteran homelessness programs we identified, using the process we described above. We then applied GAO guidance on duplication and overlap by comparing the programs using the following definitions: duplication occurs when two or more programs provide the same services to the same beneficiaries; overlap occurs when two or more programs offer similar services to similar beneficiaries. To identify potential benefits and challenges of overlap, we reviewed past GAO reports, and conducted interviews, as outlined above. To assess how federal agencies collaborate to address veteran homelessness, we first identified two collaborative mechanisms—the Solving Veterans Homeless as One (SVHO) working group and VA’s integration into Coordinated Entry—by reviewing agency reports, guidance, and other documentation and interviewing agency officials. We then assessed the collaborative efforts against leading interagency collaboration practices identified in prior GAO work. Specifically, we assessed the extent to which the SVHO working group and VA integration into Coordinated Entry used each leading practice using three categories. “Fully follows” indicates that actions related to a practice reflected most or all of the issues to consider related to the practice; “partially follows” indicates that actions related to a practice reflect some, but not all, the issues to consider related to the practice; and “does not follow” indicates that there have been no actions taken related to the issues to consider for the practice. One analyst reviewed the reports, guidance, and other agency documentation related to the collaborative efforts and made the initial assessment. A second analyst then reviewed this information to make their own determination about the assessment and reach consensus with the first analyst. To determine what is known about the performance of the selected programs we reviewed, we analyzed national performance data for fiscal years 2015 to 2019 from VA and DOL. To assess the reliability of those data, we reviewed the data for obvious errors or inaccuracies by comparing the data to publicly available data from VA’s and DOL’s annual performance reports (to the extent the data were published). We also interviewed VA and DOL officials with knowledge of the systems and methods used to produce these data. We determined that the data we included in the report were sufficiently reliable for purposes of describing program performance for the selected programs we reviewed. To assess if the performance measures the agencies used are effective in monitoring progress, we reviewed VA’s and DOL’s performance measurement guidance. We then compared the measures against selected leading practices we identified in past GAO work. Specifically, our prior work identified ten key attributes for successful performance measures. Measures that include these attributes are effective in monitoring progress and determining how well programs are achieving their goals. We selected six attributes relevant to our analysis. We excluded the remaining four attributes because they are used to assess agency-wide performance and therefore were not applicable to our program-specific analysis. We assessed the performance measures as “fully reflects” if all the performance measures for the selected programs reflected most or all of the definition of the relevant key attribute; “partially reflects” if the measures reflected some, but not all, of the definition of the relevant key attribute; and “does not reflect” if the measures did not reflect the definition of the relevant key attribute. One analyst reviewed the performance measures and guidance and made the initial assessment. A second analyst then reviewed this information to make their own determination about the assessment and reach consensus with the first analyst. To determine the extent to which VA, HUD, and DOL had evaluated selected programs, we conducted a literature search for studies conducted during the last five fiscal years. We also obtained program evaluations from VA, HUD, and DOL. Additionally, we reviewed the agencies’ evaluation policies and interviewed agency officials to obtain additional information about the agencies’ program evaluation efforts. We conducted this performance audit from January 2019 to May 2020 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Additional Program Information We identified 16 federal programs that target their services specifically to veterans who are homeless or are at risk of becoming homeless. These programs are funded through the Departments of Veterans Affairs (VA), Housing and Urban Development (HUD) and Labor (DOL). As shown in table 6, the programs provide permanent and transitional housing, health care, rehabilitation, employment assistance, and supportive services, such as assistance with rent, utility, or moving costs. Eligibility requirements vary by program. VA’s Grant and Per Diem (GPD) program awards grants to community- based agencies for transitional housing and case management for homeless veterans. In 2017, VA implemented changes to the program and, as seen in table 7, the program now has six housing models. Each model targets a different population of homeless veterans or focuses on different areas of service. Some VA medical centers (VAMCs), service providers, and public housing agencies (PHAs) we interviewed told us the homelessness programs for veterans we reviewed are working well. Others identified additional challenges that were specific to individual selected programs we reviewed, in particular the GPD program that underwent recent changes. For example, with respect to GPD’s new models, four service providers and staff from three VAMCs told us that the housing models and program guidelines are too restrictive and complex, which hinder the delivery of services. Staff from another VAMC told us that the new housing models are based on best practices but the implementation is challenging. For example, one of these models, Bridge Housing, generally limits the length of stay to 90 days which GPD staff from one VAMC and one provider told us is not enough time to meet the needs of some clients. However, VA officials said that veterans are not asked to leave Bridge Housing after 90 days if the housing plan has not been executed by this time. According to VA officials, GPD grantees can provide transitional housing and services to family members of a veteran, however, the program can only pay per diem for veterans, not their families. In addition, two GPD service providers told us that the bed reimbursement rate is inadequate to cover the cost of providing services to veterans, and GPD staff at one VAMC told us that the existing funding does not cover the full cost of the program. Despite these cited challenges, our review of national performance data shows that VA is generally meeting the performance targets for these six models. Finally, GPD staff at one VAMC told us that there is a shortage of shelters and beds in some areas, and as a result, they cannot accommodate all the homeless veterans that are referred to them. Appendix III: Comments from the Department of Labor Appendix IV: Comments from the Department of Veterans Affairs Appendix V: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Allison Abrams (Assistant Director), Erika Navarro (Analyst in Charge), Kimberly Bohnet, Emily Bond, Evelyn Calderon, Lilia Chaidez, Jill Lacey, and Jessica Sandler made key contributions to this report. Also contributing to this report were Ryan Cirillo, Ben Licht, Marc Molino, Sarah Veale, James Whitcomb, and Michael Zose.