Background Federal agencies conduct a variety of activities related to animal research. These activities include: funding intramural research conducted by agency personnel at federal facilities; funding extramural research conducted by universities, industrial firms, and other nonfederal entities through contracts, grants, or cooperative agreements; establishing guidelines for regulating products that may have been tested for safety or efficacy using animals; and overseeing the welfare of animals used for research. Table 1 shows key activities related to animal research or use that HHS, USDA and EPA and their component agencies and offices are involved in through their funding of intramural and extramural research and regulation of products. Some agencies within USDA and HHS have roles in overseeing animal welfare at federal and nonfederal research facilities. Under the Animal Welfare Act and its implementing regulations, USDA’s APHIS oversees federal and nonfederal research facilities to ensure the humane treatment of covered species of warm-blooded animals when they are used in research, teaching, testing, or experimentation. These species include dogs, cats, nonhuman primates, guinea pigs, hamsters, rabbits, horses used for research purposes, and (with certain exceptions) other warm- blooded animals. Requirements under the act related to consideration of alternatives to animal research include the following: Under the act, research facilities are to appoint an animal care and use committee to, at least semiannually, review the facility’s program for humane care and use of covered animals, inspect all facilities, and prepare reports of its evaluation. The committee is responsible for reviewing research proposals to determine whether the proposed activities are in accordance with the act. This includes a review of research proposals to determine whether researchers have (1) considered alternatives to procedures that may cause more than momentary or slight pain or distress to covered animals and (2) have provided a written narrative description of the methods and sources they used to determine that alternatives were not available. The committee can ask a researcher to explain why any alternatives found are not used in the researcher’s proposal or withhold approval of the proposal. Facilities that used or intended to use live covered animals in research are to submit a retrospective annual report about those animals to APHIS on or before December 1 of each calendar year. In particular, the annual reports are to include an assurance that each researcher considered alternatives to painful procedures. HHS’s NIH is responsible for establishing guidelines implementing certain provisions of the Health Research Extension Act of 1985. NIH’s responsibilities under the act include reviewing federal and nonfederal research facilities’ vertebrate animal care and use programs to determine whether they meet relevant standards and are thereby eligible to receive funding from HHS agencies covered by the act, including NIH. NIH implements the animal care provisions of the act through its Public Health Service Policy. The policy’s requirements related to research facilities’ consideration of alternatives include the following: Consistent with the act, NIH’s Public Health Service Policy directs facilities to provide for NIH’s approval a document that describes their vertebrate animal care and use program and that provides assurances that the research institution meets applicable standards. Such assurances must include a synopsis of training or instruction in research or testing methods that minimize the number of vertebrate animals required to obtain valid results and minimize animal distress and that the facility offers to scientists, animal technicians, and other personnel involved in animal care, treatment, or use. As a condition of receiving funding for animal research from HHS agencies, facilities must, for the most part, adhere to the eighth edition of the Guide for the Care and Use of Laboratory Animals (Guide). The Guide states that in preparing and reviewing research protocols, researchers and animal care and use committees should consider the availability or appropriateness of using less invasive procedures, other species, isolated organ preparation, cell or tissue culture, or computer simulation. The Guide does not limit this to research that is painful or distressful. NIH conducts site visits at selected research facilities to assess compliance with the act. Whereas the Animal Welfare Act applies to certain warm-blooded animals, the definition of animals used for the purposes of the Health Research Extension Act covers all vertebrates, including the mice, rats, and fish species commonly used in laboratory research. Other Relevant Legislation Other laws relevant to the consideration of alternatives include the following: The National Institutes of Health Revitalization Act of 1993 directs the Director of NIH to prepare a plan to conduct or support research into methods of biomedical research and experimentation that do not require the use of animals, that reduce the number of animals used in such research, and that produce less pain and distress in such animals. The act also directs NIH to prepare a plan for establishing the validity and reliability of the new methods it develops, encouraging the scientific community’s acceptance of these methods, and training scientists in using such methods. The act further directs NIH to periodically review this plan and, as appropriate, make revisions and include those revisions in a biennial report. In response to the act, in September 1994 NIH established ICCVAM as an ad hoc committee. The ICCVAM Authorization Act of 2000 directed NIH to establish the ICCVAM as a permanent interagency committee under NIH’s National Toxicology Program Interagency Center for the Evaluation of Alternative Toxicological Methods. ICCVAM is administered by the National Institute of Environmental Health Sciences. The act specifies that ICCVAM be composed of the heads (or their designees) of 15 agencies or subagencies, including EPA, agencies within HHS, and USDA. The National Institutes of Standards and Technology joined voluntarily in 2016. The act directed ICCVAM to, among other things, review and evaluate alternative test methods that may be acceptable for specific regulatory uses and to prepare biennial progress reports. Under the act, an alternative test method is one that reduces the number of animals required; refines procedures to lessen or eliminate pain or distress to animals or enhances animal well-being; or replaces animals with non- animal systems or one animal species with a species presumed to have less ability to feel pain, such as replacing a mammal with an invertebrate. In January 2018, ICCVAM published a strategic roadmap articulating its vision to meet its purpose. The Frank R. Lautenberg Chemical Safety for the 21st Century Act amended the Toxic Substance Control Act in 2016 to include language on the use of alternative methods. The act directs the Administrator of EPA to reduce and replace, to the extent practicable, scientifically justified, and consistent with the policies of the Toxic Substances Control Act, the use of vertebrate animals in the testing of chemical substances or mixtures under the Toxic Substances Control Act. The act also directs the Administrator to develop a strategic plan to promote the development and implementation of alternative test methods and strategies to reduce, refine, or replace vertebrate animal testing. HHS, USDA, and EPA Have Used a Variety of Methods to Ensure Researchers Consider Alternatives to Animals Methods HHS, USDA, and EPA have used to ensure that researchers consider alternatives to animal research include requiring researchers to describe and document their consideration of alternatives. In addition, USDA’s APIHS and HHS’s NIH help ensure that researchers consider alternatives by overseeing research facilities and these facilities’ animal care and use committees, including the committees’ review of animal research protocols. USDA and NIH also provide training to researchers and animal care and use committees to help ensure researchers have considered alternatives. HHS, USDA, and EPA Call for Written Descriptions of the Consideration of Alternatives and Recommend a Method for Identifying Alternatives For research that they conduct or fund, component agencies and offices within HHS, USDA, and EPA call for individual researchers to describe their consideration of alternatives to animal research. USDA’s APHIS and HHS’s NIH require research facilities to consider alternatives through the agencies’ implementation of the Animal Welfare Act regulations and Public Health Service Policy, respectively. EPA research is covered by the two laws to the extent that it uses animals covered by the Animal Welfare Act or Health Research Extension Act. Table 3 summarizes the factors that determine whether researchers are required under the acts to consider alternatives. The steps HHS, USDA, and EPA take to help ensure that agency researchers and the researchers that they fund or oversee meet the requirement to consider alternatives include (1) calling for written descriptions of researchers’ consideration of alternatives and (2) prescribing or recommending that researchers use searches, such as of databases of published scientific literature, to identify alternatives. Call for written descriptions. As specified in the Animal Welfare Act regulations and Public Health Service Policy, HHS, USDA, and EPA call for researchers to send written descriptions of research projects involving animals to animal care and use committees for their review and approval. In particular, the Animal Welfare Act regulations require these committees to determine that researchers have provided a written narrative description of the methods and sources they used to determine that alternatives were not available. The Public Health Service Policy requires that researchers’ institutions submit written descriptions of research projects to the committees and for the committees to determine that researchers’ procedures avoid or minimize discomfort, distress, and pain to animals, consistent with sound research design, among other things, and that researchers follow the U.S. Government Principles for the Utilization and Care of Vertebrate Animals Used in Testing, Research, and Training. The principles require the consideration of alternatives. In our review of protocol forms from our sample of 12 research facilities (including HHS, USDA, and EPA facilities), we found that all of the forms requested information on researchers’ consideration of alternatives, though the forms varied in the particular information they requested. The types of information requested included a rationale for involving animals and for the number of animals to be used, assurance that research activities do not unnecessarily duplicate previous experiments, and a description of the methods and sources used to determine that alternatives were not available. Several of the protocol forms required researchers to identify alternatives considered but not adopted. Recommended method for identifying alternatives. In their implementation of the Animal Welfare Act and Health Research Extension Act, respectively, USDA and NIH consider database searches as a best practice for researchers using animals covered by the acts to identify and consider alternatives to animal testing. A database search involves a researcher using keywords related to the planned use of animals to query citations in databases of published scientific literature. From April 1997 through July 2018, USDA maintained a policy, known as Animal Care Policy #12, in its animal care policy manual. In Policy #12, the agency recommended a database search as the most effective and efficient method for demonstrating compliance with the requirement to consider alternatives to painful or distressful procedures. According to USDA’s Deputy Administrator responsible for implementation of the Animal Welfare Act regulations, in July 2018, USDA placed the policy in inoperative status after determining that some research facilities and agency inspectors had misinterpreted the policy as a requirement. Moreover, in response to the 21st Century Cures Act, USDA is reviewing its animal care policy manual, including Policy #12, to ensure the policies in the manual conform with the Animal Welfare Act and its implementing regulations, harmonize with NIH guidance, and reduce researcher burden where possible. According to the Deputy Administrator, as of June 2019, USDA had not decided what, if anything, it would do to revise or replace Policy #12. According to a draft interagency report in response to the 21st Century Cures Act, USDA will make any revised and future policies involving the use of animals available for public comment using regulations.gov or a similar service. However, according to the Deputy Administrator, even though Policy #12 is inoperative, USDA continues to advocate for database searches, particularly through the USDA Animal Welfare Information Center’s provision of information to the scientific community about how to search for alternatives. According to a senior NIH official, NIH requires that agency researchers conduct database searches. Also, in a sample animal study proposal form NIH has provided to animal care and use committees, NIH recommends that researchers at other facilities conduct database searches. Furthermore, 11 of the 12 research facilities we reviewed (including HHS, USDA, and EPA facilities) used research protocol forms that required or recommended that their researchers conduct a database search for alternatives to animal research. Agencies may apply additional requirements to individual researchers at their own facilities or through grants they fund, in addition to applying the requirements of the Animal Welfare Act and Health Research Extension Act. For example, CDC’s Fort Collins, Colorado, facility requires researchers to provide assurance on their protocol forms that the facility’s animal care and use committee’s statistician reviewed the form to determine whether the research would use an appropriate number of animals or explain why a review of the number of animals did not occur. Similarly, the Chairman of APHIS’s National Wildlife Research Center committee told us that its animal care and use committee includes a biostatistician who conducts an analysis to ensure that the numbers of animals to be used will produce statistically significant results. USDA’s National Institute for Food and Agriculture requires applicants for funding from the agency’s Agriculture and Food Research Initiative Competitive Grants Program to use statistical power analysis, when appropriate, to determine the sample sizes of animals to be used in research. USDA officials told us that this type of analysis provides a justification for the number of animals needed to provide valid results and helps prevent the unnecessary use of animals. Agencies may also require information on animal use in proposals submitted by extramural researchers. For example, NIH instructs researchers to describe the use of animals in their work in a section of grant applications, contract proposals, and cooperative agreements. Specifically, when submitting a proposal, researchers must justify to agency officials and other reviewers that the species used is appropriate for the proposed research and explain why research goals cannot be accomplished using an alternative model, such as computational, human, invertebrate, or in vitro models. APHIS and NIH Oversee the Review and Approval of Animal Research Protocols by Animal Care and Use Committees APHIS and NIH help ensure that researchers consider alternatives through the agencies’ oversight of research facilities and these facilities’ animal care and use committees, including the committees’ review of animal research protocols. In particular, APHIS collects and reviews annual reports from federal and nonfederal research facilities in which the facilities are required to provide an assurance that researchers considered alternatives. The Animal Welfare Act requires APHIS to annually inspect nonfederal research facilities to determine whether the facilities are in compliance with the act. As part of a facility inspection, APHIS inspectors are to examine whether researchers have met the requirement to consider alternatives to any procedure likely to produce pain in or distress to species of animals covered by the act. According to APHIS officials, inspectors examine a sample of approved animal research protocols to check whether the protocol forms include a written narrative on the consideration of alternatives and to ensure that the facility’s animal care and use committee approved the protocol forms. The inspectors may issue citations of noncompliance if they find inadequate documentation that researchers associated with one or more protocols considered alternatives to procedures that may cause more than momentary or slight pain or distress to animals. APHIS provided us with inspection reports for fiscal years 2015 through 2018 in which inspectors issued 57 citations to research facilities for noncompliance with the Animal Welfare Act regulations that require researchers to consider alternatives to animals or issued “teachable moments.” The inspection reports included some citations that, according to APHIS officials, were incorrectly issued because inspectors interpreted the Policy #12 recommendations on database searches as requirements. In addition, NIH’s Office of Laboratory Animal Welfare is responsible for the general administration and coordination of the Public Health Service Policy and provides specific guidance, instruction, and materials to research facilities that receive funding from agencies covered by the act. For all such facilities, NIH is to review the facilities’ assurance documents describing their animal care and use programs. In particular, the Animal Welfare Assurance document is to describe the procedures—including review of animal research protocols—that the animal care and use committees follow to fulfill the directives of the NIH Public Health Service Policy. Further, NIH conducts site visits at a small number of facilities. The Public Health Service Policy states that each awardee institution is subject to review at any time by agency staff and advisors to assess the adequacy and accuracy of the institution’s compliance or expressed compliance with the policy, and this review may include a site visit. According to NIH officials, when agency staff conduct site visits, they examine the facility’s protocol form to confirm that its animal care and use committee requests information from researchers about their consideration of alternatives. NIH officials may also examine a sample of approved protocol forms during a site visit. According to NIH officials, the Office of Laboratory Animal Welfare conducted 38 site visits in fiscal years 2015 through 2018 and found one deficiency related to the consideration of alternatives. USDA and NIH Provide Training to Researchers and Facilities’ Animal Care and Use Committees to Help Ensure Alternatives Are Considered USDA and NIH have provided training to researchers and animal care and use committee members on the requirements of the Animal Welfare Act and the Health Research Extension Act. The training has addressed, among other things, the requirement to consider alternatives and has included advice on how to search for alternatives. Through its Animal Welfare Information Center, USDA provides training on how to conduct database searches for alternatives to animal research and assists individual researchers with their literature searches. According to USDA staff, the information center provides three workshops per year on meeting the requirements of the Animal Welfare Act, each lasting a day and a half. The workshops are open to anyone working with animals in research, including scientists, veterinarians, librarians, and animal care and use committee members. The center also gives workshops upon request at specific facilities. Additionally, the center’s website contains resources for conducting literature searches, and, according to a senior information center official, the center plans to put workshops into an online format that will be available upon demand. According to the official, the center conducted 137 database searches upon request in fiscal years 2014 through 2018. NIH has also provided training on the consideration of alternatives to help researchers meet their requirements under the Health Research Extension Act. For example, in 2014 NIH presented a webinar on searches for alternatives. The webinar, titled Meeting Requirements for Alternatives Searches and available on NIH’s website, provides advice on how to conduct database searches. For example, the webinar provided advice on the timing of the search, the search strategy, and particular databases to use. In September 2015, NIH presented a webinar demonstrating how to use NIH’s database of research projects to find researchers, projects, and publications that may help replace, reduce, and refine the use of animals in research. The NIH Office of Laboratory Animal Welfare provides on its website a sample animal study protocol form that emphasizes database searches for any procedures that cause more than momentary or slight pain or distress to the animals. In addition, according to a senior official from the NIH office overseeing the agencies’ intramural research using animals, researchers at NIH must complete an online course regarding animal use every 3 years. The course includes a section on replacing, reducing, and refining animal use and outlines how researchers are to report literature searches in order to show they considered alternatives. HHS, USDA, and EPA Have Facilitated the Development and Use of Alternatives to Animals in Research but Have Not Consistently Assessed the Effect of Their Efforts EPA, HHS, and USDA have facilitated the development, use, and promotion of alternative research methods through individual and collaborative efforts, including strategies for promoting the use of alternative methods and development of policies and guidance on alternative methods. The three agencies have also developed alternative research methods that rely on non-animal models and procedures to test how various products would affect humans. Additionally, the agencies have worked collaboratively with each other and with nonfederal stakeholders to promote alternative methods, in particular through ICCVAM, which is required to report to the public on its progress. However, ICCVAM and its member agencies have not routinely developed or reported metrics for assessing the effect that their efforts are having on animal use. EPA and Agencies within HHS and USDA Have Issued Strategies, Policies, and Guidance on Alternative Research Methods EPA has issued a strategic plan and FDA has issued a roadmap for the use of methods that may reduce animal use in assessments of the safety and efficacy of various products. Both agencies and others within HHS and USDA have also issued guidance on using alternatives to animal research in particular contexts, such as vaccine testing. Strategic Plan and Guidance for Reducing Animal Testing for EPA-Regulated Toxic Chemicals and Pesticides In June 2018, EPA’s Office of Chemical Safety and Pollution Prevention issued a strategic plan for the reduction of vertebrate animal testing for toxic chemicals regulated under the Toxic Substances Control Act. The office developed and issued this strategic plan to implement a provision in the Frank R. Lautenberg Chemical Safety for the 21st Century Act (Lautenberg Act) calling for such a plan. The strategic plan describes a multi-year process with incremental steps for adopting and integrating methods that do not use vertebrate animals in evaluating chemicals regulated by the Toxic Substances Control Act for their effect on human health and the environment. The strategic plan states that the agency’s long-term goal is to reduce and eventually eliminate vertebrate animal testing for chemicals regulated under the act. Pursuant to the strategic plan, in June 2018 EPA published a list of methods the agency had identified that require no vertebrate testing and that are capable of providing information of equivalent or better scientific reliability and quality than that which would be obtained from vertebrate animal testing. According to EPA, the agency plans to update the list at least once a year. EPA’s strategic plan calls for other near-term activities such as retrospectively identifying and evaluating the studies that it has requested and received for both new and existing chemicals. The plan states that EPA will complete this analysis in 2019 and use the results to support the future development of alternative methods to fit the agency’s needs. In May 2011, EPA’s Office of Chemical Safety and Pollution Prevention issued a strategic plan in response to a 2007 National Academies report calling for a more efficient and informative risk assessment process to predict and characterize potential human health and environmental hazards from exposures to pesticides. The office’s strategic plan envisions using a combination of computational and predictive modeling approaches, in vitro techniques, and targeted in vivo testing to supplement or replace the existing toxicity tests required in federal regulations for pesticide registration under the Federal Insecticide, Fungicide, and Rodenticide Act. Pursuant to this strategic plan, EPA’s Office of Chemical Safety and Pollution Prevention has issued guidance on data requirements for assessing pesticide safety that may reduce animal use. For example, EPA issued guidance in May 2013 on the data that the agency needs in order to adequately assess pesticide risks. The guidance also provided information to manufacturers on how to request waivers from the data requirements, which would enable the manufacturers to reduce animal use. EPA also issued guidance in November 2016 that allows pesticide manufacturers to request a waiver from the requirement to provide data on acute toxicity tests and that contains a policy statement waiving all acute lethality dermal studies for formulated pesticide products; such waivers can reduce the need for pesticide manufacturers to conduct tests using animals. For example, EPA reported granting a total of 223 waivers in fiscal years 2016 and 2017, pursuant to the agency’s May 2013 guidance for toxicity studies, which the agency estimated avoided the use of 85,000 animals and saved pesticide manufacturers $26.4 million in conducting toxicity studies. In February 2016, EPA announced an effort to evaluate and implement alternative methods for tests involving acute oral, dermal, and inhalation toxicity; skin and eye irritation; and skin sensitization. As part of this effort, in April 2018, EPA issued a draft policy to reduce the use of animals in testing chemicals to evaluate whether they cause an allergic reaction, inflammation, or sensitization of the skin. EPA’s policy describes conditions under which the Office of Chemical Safety and Pollution Prevention will accept alternative approaches to laboratory animal studies for identifying skin sensitization hazards. Roadmap for Developing and Evaluating Tools for Assessing FDA-Regulated Products and Guidance for Industry In December 2017, in response to direction from the FDA Commissioner, FDA developed a roadmap to foster the development and evaluation of emerging tools and methods that can improve toxicology methods for assessing the safety of FDA-regulated products. The roadmap does not have an explicit goal to replace, reduce, or refine animal testing but states that new methods may have the potential to do so. In that regard, the roadmap states that FDA will encourage medical product sponsors to submit a scientifically valid approach for using a new method early in the regulatory process and to engage in frequent communication with the agency about the suitability of that method. In addition, the roadmap recommended that FDA establish an organizing committee; conduct training; foster communication and collaboration with stakeholders, such as industry and academia; engage in research; and track and report annually on its progress. In June 2019, FDA posted its first annual report on its progress in implementing the roadmap. Previously, FDA had taken steps to reduce animal use by issuing guidance to members of industry seeking approval for FDA-regulated products. In general, FDA guidance states that industry may choose to use an approach—such as a non-animal testing method—other than one set forth in guidance as long as it complies with relevant statutes and regulations. FDA has also taken more specific steps to modify guidance to promote the use of alternative methods. For example, in 2012, FDA issued guidance to industry that states that firms may use non-animal alternative methods to test the toxicological safety of pharmaceutical drugs if the methods are appropriate or scientifically justified. In 2013, FDA issued guidance that, among other things, allowed industry to use in vitro assays rather than mice to detect toxins in shellfish meant for human consumption; this guidance subsequently played a role in the adoption of additional methods that do not employ animal use. Other HHS and USDA Agencies’ Guidance on the Use of Alternatives to Animal Research Other HHS and USDA agencies within our scope do not have strategic plans or roadmaps that promote a comprehensive strategy for alternative research methods, but some of the agencies have issued guidance to their own researchers or to regulated entities that may reduce animal use. For example, in 2017, APHIS updated its guidance to allow manufacturers of animal vaccines, inactivated bacterial products, and antibody products to request an exemption to animal safety testing if the products have a documented history of acceptable safety results and controlled manufacturing processes that ensure batch consistency and sterility. APHIS also issued a notice in 2017 of a testing option that can reduce by up to 50 percent the number of hamsters required for potency testing of vaccines for the bacterial disease leptospirosis, according to the notice. In addition, APHIS issued memorandums in 2013 and 2015 that provide guidance on in vitro techniques that researchers may use instead of animals to test the potency of vaccines. Some agencies have also adopted alternative methods for their researchers without issuing specific guidance to do so. For example, in September 2018, CDC began routine use of an in vitro procedure developed by the agency that allows its laboratories to test for botulism in human serum specimens without using mice. CDC officials stated that this method is fast and inexpensive and would reduce the need for hundreds of mice. CDC officials told us that their researchers plan to expand use of the in vitro procedure to test other types of specimens, further reducing the use of animals. HHS, USDA and EPA Have Developed Alternative Research Methods HHS, USDA, and EPA have made multiple efforts to develop alternative research methods that, according to the agencies, have reduced animal use or have the potential to do so. Some of these efforts target reducing the use of animals in a particular research context while others have broader applications in toxicology and computer modeling. In some cases, agency officials provided estimates of how their targeted efforts have reduced or may reduce animal use. Examples of targeted efforts include the following: CDC researchers told us they have evaluated a method that reduces the number of animals and time needed to produce kits that are distributed worldwide to identify influenza virus subtypes and thereby aid in strain selection for the influenza vaccine each season. Under the original method, antibodies for the kits were generated from blood samples in sheep that were later euthanized. The CDC researchers concluded that an alternative automated method that draws antibody-rich plasma from goats instead of blood from sheep could reduce the time needed to produce the kits and require fewer animals. According to FDA officials, FDA is collaborating with others on the development of an in vitro assay that will be used to test the potency of human rabies vaccines that manufacturers submit to FDA for approval. This new method will replace the animal-based assay that is part of the current license to manufacture rabies vaccine. The officials said that the animal-based assay uses 600 mice, on average, for each batch of vaccine submitted by a manufacturer. According to an ARS research paper, ARS worked with academic researchers to develop an in vitro method for feeding blood to ticks. According to ARS officials, the method allows researchers to reduce the number of animals used when studying disease transmission in animals via tick-borne pathogens. APHIS currently holds federal pesticide registrations with EPA for active ingredients formulated into end-use products, such as rodenticides, that APHIS uses to prevent damage to agriculture, endangered species, or critical habitats. According to APHIS officials, the agency uses an EPA- approved method for testing the risks to human health from new pesticide products that substantially reduces animal use. The method generally involves progressively increasing the pesticide dose on a relatively small number of animals compared to the previous method and waiting to observe whether the dose causes mortality before deciding whether to increase the dose in further testing. APHIS officials said the new method reduces animal use by 50 percent or more per test. EPA’s Endocrine Disruptor Screening Program Uses Alternatives to Animals Led by its Office of Science Coordination and Policy, EPA established the Endocrine Disruptor Screening Program in 1998 to fulfill a congressional mandate in the 1996 Food Quality Protection Act to develop a program to screen for certain chemicals (e.g., pesticides) that affect human hormones. EPA expanded the scope of the program to include screening the effects of chemicals on the human thyroid system and wildlife. The program began using automated, large-scale screening methods and computational models to evaluate and screen chemicals and, according to EPA, allows EPA to screen more chemicals in less time, use fewer animals, and reduce cost. Agencies’ broader efforts include the integration of advances in biology, chemistry, and computer science into areas of research, such as toxicology, that currently rely heavily on animal use. For example, EPA launched the Toxicity Forecaster in 2007 as an effort to use automated technologies to expose living cells or isolated proteins to chemicals and screen the cells or proteins when exposed to chemicals for changes in biological activity that suggest potential toxic effects. According to EPA documents, these methods could limit the number of required laboratory animal-based toxicity tests while quickly and efficiently screening large numbers of chemicals. According to EPA documents, in the first phase of this effort, which the agency completed in 2009, EPA evaluated more than 300 well-studied chemicals (primarily pesticides) that had extensive data from traditional animal-based toxicity testing; the agency then compared results from automated screening technologies with the results from the traditional animal tests. As of 2018, EPA had developed and made publicly available a library of toxicity data on more than 4,500 chemicals. The availability of the Toxicity Forecaster data has enabled EPA to reduce the need for animal testing in its Endocrine Disruptor Screening Program for identifying chemicals that may affect human hormone systems (see sidebar). Similarly, FDA has initiated a broad effort to incorporate greater use of computer modeling and simulation into its decision-making on FDA- regulated products. For example, FDA formed an agency working group on modeling and simulation in 2017. According to the Chair of the working group, it does not have an explicit objective to reduce animal testing, but such reduction is a potential benefit of the testing approaches the group is advancing. For example, the Chair said that modeling and simulation can help refine questions about products submitted for FDA approval and therefore could reduce the number of animal studies needed before clinical trials. EPA and NIH have provided funding to extramural researchers to develop alternative research methods. For example, from 2013 through 2018, EPA provided $24 million in funding for research on 3-D models containing human cells (these devices are also known as tissue chips) that can be used for tests that otherwise might be conducted using animals. In 2018, EPA also announced $4.25 million in funding for research to promote the development and use of alternative methods that reduce, refine, or replace vertebrate animal use for toxicity testing. Similarly, in 2017, NIH awarded a $962,000 grant to a research facility to conduct studies of an in vitro human bronchial tissue model for predicting the toxicity of inhaled chemicals. Additionally, while USDA’s National Institute for Food and Agriculture did not set aside a specific amount of funding, in May 2019 the agency made clear to applicants for its Welfare and Well-being of Agricultural Animals grant program that proposals that study ways to reduce the need for animals in research are eligible for funding in fiscal years 2019 and 2020. HHS, USDA, and EPA Have Collaborated with Each Other and with Other Federal and Nonfederal Stakeholders to Promote Alternative Methods HHS, USDA, and EPA have joined partnerships to develop, use, and promote alternative testing methods. For example, the agencies participate in ICCVAM, which states that its mission is to facilitate the development, validation, and regulatory acceptance of test methods that replace, reduce, or refine the use of animals. ICCVAM itself does not conduct research or validation studies on alternative methods. Instead, it relies on stakeholders including federal agencies that generate, require, or use toxicological data; companies that develop toxicological tests; and animal welfare organizations. According to committee guidelines, stakeholders can submit the results of their research to ICCVAM, and the committee then conducts evaluations and makes recommendations on submissions for regulatory uses that align with the needs and priorities of member agencies. Zebrafish Are Used as Alternatives to Other Animals in Research A zebrafish is a freshwater, tropical vertebrate fish that is widely used in pharmaceutical development and medical and scientific research due to certain qualities of its morphology and development as well as its inexpensive cost to use and maintain. Some of these qualities include genetic and structural similarities to other vertebrates that mimic human responses to certain genes involved in human diseases and its transparent embryonic development that enables researchers to use it as an alternative model for toxicity screening of drugs and chemicals. These qualities have led to the use of zebrafish embryonic models for automated, large-scale screening programs by the National Toxicology Program and the Environmental Protection Agency, among others. ICCVAM’s website contains information on current ICCVAM- recommended protocols for specific test methods, such as methods to test for eye corrosion and irritation and skin sensitization, and on events organized by NIH and others that are relevant to the replacement, reduction, or refinement of animal use in research. For example, the website has a link to a page on NIH’s website that has the slide presentations given at six webinars from 2017 through 2018 on the use of zebrafish in toxicology testing. Researchers may use zebrafish and their embryos in particular as a replacement for other animals, such as mice (see sidebar). ICCVAM maintains on its website a list of 108 alternative methods that, as of June 2019, had been accepted by one or more federal agencies. These include methods that ICCVAM and its member agencies contributed to developing or validating. However, according to ICCVAM’s strategic roadmap issued in January 2018, the committee concluded that its evaluations of new methods during its first 15 years were lengthy, inefficient, and resource intensive. ICCVAM concluded that researchers and test method developers often initiated the development of alternative methods with little input from federal agencies or regulated industries and, therefore, these methods did not always meet the needs of federal agencies. Consequently, these methods were either not accepted by federal agencies or were accepted by the agencies but not used by the regulated community. Recognizing these limitations, ICCVAM initiated a strategic shift in 2013 aimed at adjusting the validation of new test methods to be more responsive to the needs of federal agencies and other stakeholders. Accordingly, ICCVAM’s 2018 strategic roadmap set new objectives for reducing animal use, including the following: Connect the developers of alternative methods with the regulatory agencies and the regulated industries that would ultimately use the new technologies to increase the likelihood of the methods being successfully developed and implemented. Foster the use of efficient and flexible practices, such as public-private partnerships to promote communication and cooperation, to establish confidence in new methods. Encourage the adoption and use of new methods and approaches by federal agencies and regulated industries, such as through training programs on the use of new methods. ICCVAM has established workgroups to develop detailed implementation plans to address roadmap goals. According to the strategic roadmap, the implementation plans will include four key elements: (1) definition of testing needs; (2) identification of any available alternative tests and computer models; (3) a plan to develop integrated approaches to testing and assessment and defined approaches for interpreting data; and (4) a plan to address both scientific and nonscientific challenges, including regulatory challenges, such as international harmonization. As of June 2019, workgroups on acute systemic toxicity, eye and skin irritation, and skin sensitization had posted information concerning these elements on ICCVAM’s website. For example, each workgroup authored an article published in a peer-reviewed journal and posted on the ICCVAM website about the testing needs of regulatory agencies and information about available alternatives. Another interagency effort that has a goal of promoting the use of alternative methods is the Toxicology in the 21st Century (Tox21) Program. Formed in 2008, the program is a collaborative effort among NIH, FDA, and EPA to characterize the potential toxicity of chemicals by using cells and isolated molecular targets instead of laboratory animals. A central component of the program is its focus on developing and evaluating automated in vitro screening methods to assess the hazards of chemical substances. As of February 2018, the program had used this method to assess approximately 10,000 chemicals for their potential impacts on biological systems. According to NIH’s Tox21 website, these automated methods have yielded high-quality toxicity data on environmental substances in a fraction of the time that would have been required with traditional animal testing. To address key challenges in toxicology testing, the program’s federal partners developed a strategic and operational plan in March 2018 that expanded the focus of Tox21’s research activities to include developing alternative test systems that predict chemical toxicity in humans and addressing the technical limitations of and strengthening scientific confidence in current in vitro test systems. According to NIH, activities under the plan will lead to better predicting chemical toxicity to humans through using non-animal alternatives such as stem cells and computational models. Since 2011, federal agencies have also collaborated to develop devices containing human cells that can be used for tests that otherwise might be conducted using animals. See figure 1 for an example of such devices, known as tissue chips or human microphysiological systems. According to NIH officials, this evolving technology may reduce animal testing and produce results more relevant to human health. The interagency effort was initiated in September 2011 when the President announced the formation of a collaborative project between NIH, the Defense Advanced Research Projects Agency, and FDA to develop tissue chips loaded with living human cells to screen the efficacy, safety, and toxicity of drugs, vaccines, or biological products for humans. Subsequently, in July 2012, NIH launched the Tissue Chip for Drug Screening program, which provided 19 grants to research facilities to develop tissue chips that accurately model the structure and function of the human lung, liver, heart, and more. In September 2014, NIH announced a second phase of the program in which researchers would refine existing tissue chips and combine them into an integrated system that can mimic the complex functions of the human body. In one example of this collaboration, two project teams funded by the Defense Advanced Research Projects Agency—the Massachusetts Institute of Technology and the Wyss Institute at Harvard University—are working with NIH-funded researchers to develop platforms that integrate 10 tissue chips that each represent a separate human organ. Federal agencies have also collaborated with nongovernmental organizations on training to promote the use of alternative methods. For example, NIH and the People for the Ethical Treatment of Animals (PETA) International Science Consortium offered a webinar series from March through September 2016 on alternative approaches for assessing acute inhalation toxicity. Webinar presenters described alternative approaches for identifying substances likely to cause acute systemic toxicity through inhalation. Similarly, EPA collaborated with the PETA International Science Consortium and the Physicians Committee for Responsible Medicine on webinars in November 2018, February 2019, and April 2019 that addressed alternative methods for testing the effect of chemicals on skin, for predicting the effect of inhaled substances, and for identifying substances that cause irritation or inflammation in human respiratory systems. ICCVAM and Its Member Agencies Have Not Established a Workgroup to Develop Metrics on the Effect of Their Efforts on Animal Use ICCVAM’s strategic roadmap calls for its members to identify appropriate metrics for prioritizing activities, monitoring progress, and measuring success toward the goals described in the roadmap. However, ICCVAM and its member agencies have not routinely developed metrics that they could report to the public to demonstrate how their individual or collective efforts to encourage the use of alternative methods have affected or will affect animal use. HHS, USDA, and EPA officials, as well as ICCVAM’s roadmap, have cited challenges to measuring the results of ICCVAM and its member agencies’ efforts. For example, according to agency officials, differences in the regulatory contexts in which agencies use data generated through animal research—for example, in regulation of pesticides versus human or animal drugs—limit agencies’ ability to develop metrics that can be applied across multiple agencies. Furthermore, the ICCVAM roadmap states that measuring the actual impact of encouraging the adoption and use of new methods is difficult in the United States due to the limited ability to quantify animals used for toxicity testing. In particular, the Animal Welfare Act does not cover several species commonly used in research, including mice, rats, and birds bred for research and cold-blooded species such as fish. Therefore, research facilities are not required under the act to report their use of those species to APHIS, and the data APHIS receives from research facilities can only be used to track a subset of the total number of animals used for research in the United States. Although ICCVAM and its member agencies face challenges in developing metrics, the roadmap also states that agency-specific mechanisms to measure progress may exist, such as tracking the number of waivers granted for a particular animal test. For example, as discussed above, EPA has estimated the extent to which its granting of data waivers to pesticide manufacturers has reduced animal use and research costs. Additionally, some agencies have estimated the effect that a new alternative method could have on animal use. Moreover, officials from FDA and EPA said that their agencies are able to accept non-animal test data in lieu of animal test data if the data meet their regulatory needs. Measuring the frequency with which the agencies receive non-animal test data instead of animal data could be another mechanism for estimating changes in animal use. In addition, the ICCVAM Authorization Act of 2000 requires ICCVAM to prepare biennial public reports on its progress under the act—including its efforts to ensure that new and revised test methods are validated to meet the needs of federal agencies and to reduce, refine, or replace the use of animals in testing, among other things. ICCVAM, with support from NIH’s National Institute of Environmental Health Sciences, has issued the required biennial progress reports since 2001, including the most recent report issued in July 2018 that covers 2016 and 2017. However, the committee’s biennial progress reports, including the July 2018 report, provide few quantitative or qualitative assessments of the progress the member agencies have made, individually or collectively, toward reducing, refining, or replacing animal use in testing. ICCVAM’s strategic roadmap states that it envisions that workgroups will play a key role in implementing the goals of the strategic roadmap, but ICCVAM has not designated a workgroup to address the challenges related to metrics, similar to other workgroups that the committee has established to address the roadmap’s goals. According to officials from NIH’s National Institute of Environmental Health Sciences, which manages the committee, the strategic roadmap is a work in progress and developing metrics is the third of three roadmap goals. The ICCVAM Authorization Act of 2000 does not provide the National Institute of Environmental Health Sciences with authority to direct agencies to develop and report metrics. However, agency officials agreed that ICCVAM could facilitate the establishment or designation of a workgroup of member agencies to identify a range of potential quantitative and qualitative metrics that member agencies could use to assess their progress toward reducing, refining, or replacing animal use. By establishing or designating such a workgroup to develop metrics that the agencies could use to assess their individual or collective progress toward reducing, refining, or replacing animal use in testing and by incorporating those metrics in ICCVAM’s biennial progress reports, ICCVAM and its member agencies could better monitor progress across the range of the committee’s efforts and report the members’ progress to the public. Conclusions HHS, USDA, and EPA use a variety of methods to ensure that researchers—whether employed by or receiving research funding from these agencies—consider alternative methods to animal research. The agencies also have engaged in multiple efforts to expand the range of available alternatives. Under one of these efforts, ICCVAM’s strategic roadmap calls for its members to identify appropriate metrics for prioritizing activities, monitoring progress, and measuring success. The roadmap envisions that workgroups will play a key role in implementing the goals of the strategic roadmap. However, ICCVAM has not designated a workgroup to address the challenges related to developing and reporting metrics. In addition, ICCVAM has issued the required biennial progress reports since 2001, but the reports provide few quantitative or qualitative assessments of the progress member agencies have made, individually or collectively, toward reducing, refining, or replacing animal use in testing. By establishing or designating a workgroup to develop metrics to assess the progress member agencies have made, individually or collectively, toward reducing, refining, or replacing animal use in testing and by incorporating those metrics in ICCVAM’s biennial progress reports, ICCVAM and its member agencies could better monitor progress across the range of the committee’s efforts and report the members’ progress to the public. Recommendation for Executive Action The Director of the NIH’s National Institute of Environmental Health Sciences should (1) facilitate the establishment or designation of a workgroup of representatives of ICCVAM member agencies to develop metrics that the agencies could use to assess the progress they have individually or collectively made toward reducing, refining, or replacing animal use in testing and (2) incorporate those metrics into the committee’s biennial progress reports. (Recommendation 1) Agency Comments and Our Evaluation We provided a draft of this report to HHS, USDA, and EPA. HHS provided written comments on the draft, which are presented in appendix I. In its written comments, HHS stated that NIH concurred with our recommendation. NIH further commented that ICCVAM’s activities in support of promoting alternatives for animal use in testing do not extend to animal use in any other context, such as research or training. NIH explained that our use of the terms research and researcher to refer more generally to research, testing, teaching, or experimentation could cause misunderstanding. We understand that ICCVAM’s activities are focused on animal use in product testing. In addition, we intended our recommendation that the Director of the National Institute of Environmental Health Sciences facilitate the establishment or designation of an ICCVAM workgroup to be focused on product testing rather than on other types of animal research. However, for editorial reasons, we did not modify our report’s use of the terms research or researcher. HHS and EPA also provided technical comments, which we incorporated as appropriate. Among those comments, HHS’s FDA officials stated that the agency encourages the use of alternatives to animal testing and supports the principles of replacement, reduction, and refinement, but if no alternative exists, animal testing may be the most appropriate way to meet certain regulatory requirements to ensure the safety and efficacy of medical products. In its technical comments, EPA cited a September 2019, memorandum EPA’s Administrator issued after we sent our draft report to the agencies for comment. The memorandum commits the agency to take several steps to reduce, replace, and refine animal testing requirements. For example, the Administrator committed EPA to reducing its requests for, and funding of, whole and live mammal studies by 30 percent by 2025 and eliminating all mammal study requests by 2035. We acknowledge EPA’s announcement but did not assess it in our review of federal efforts to facilitate the use of alternative research methods. We are sending copies of this report to the appropriate congressional committees, the Secretary of Agriculture, the Secretary of Health and Human Services, the Administrator of the Environmental Protection Agency, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff members have any questions about this report, please contact us at (202) 512-3841 or morriss@gao.gov or neumannj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. Appendix I: Comments from the Department of Health and Human Services Appendix II: GAO Contact and Staff Acknowledgments GAO Contacts Staff acknowledgments In addition to the individuals named above, Joseph Cook (Assistant Director), Rodney Bacigalupo, Kevin Bray, Ross Campbell (Analyst-in- Charge), Tara Congdon, Hayden Huang, Amber Sinclair, and Kari Terrio made key contributions to this report.


Background Federal agencies conduct a variety of procurements that are reserved for small business participation through small business set-asides. These set-asides can be for small businesses in general, or they can be specific to small businesses that meet additional eligibility requirements in the Service-Disabled Veteran-Owned Small Business, Historically Underutilized Business Zone (HUBZone), 8(a) Business Development (8(a)), and WOSB programs. The WOSB program enables federal contracting officers to identify and establish a sheltered market, or set- aside, for competition among women-owned small businesses (WOSB) and economically disadvantaged women-owned small businesses (EDWOSB) in certain industries. WOSBs can receive set-asides in industries in which SBA has determined that women-owned small businesses are substantially underrepresented. To determine these industries, SBA is required to conduct a study to determine which North American Industry Classification System (NAICS) codes are eligible under the program and to report on such studies every 5 years. Additionally, businesses must be at least 51 percent owned and controlled by one or more women who are U.S. citizens to participate in the WOSB program. The owner must provide documents demonstrating that the business meets program requirements, including a document in which the owner attests to the business’s status as a WOSB or EDWOSB. According to SBA, as of early October 2018, there were 13,224 WOSBs and 4,488 EDWOSBs registered in SBA’s online certification database. SBA’s Office of Government Contracting administers the WOSB program by, among other things, promulgating regulations and conducting eligibility examinations of businesses that receive contracts under a WOSB or EDWOSB set-aside. According to SBA, as of October 2018, there were two full-time staff within the Office of Government Contracting whose primary responsibility was the WOSB program. Initially, the program’s statutory authority allowed WOSBs to be self- certified by the business owner or certified by an approved third-party national certifying entity as eligible for the program. Self-certification is free, but some third-party certification options require businesses to pay a fee. Each certification process requires businesses to provide signed representations attesting to their WOSB or EDWOSB eligibility. Businesses must provide documents supporting their status before submitting an offer to perform the requirements of a WOSB set-aside contract. In August 2016, SBA launched certify.sba.gov, which is an online portal that allows firms participating in the program to upload required documents and track their submission and also enables contracting officers to review firms’ eligibility documentation. According to the Federal Acquisition Regulation, contracting officers are required to verify that all required documentation is present in the online portal when selecting a business for an award. In addition, businesses must register and attest to being a WOSB in the System for Award Management, the primary database of vendors doing business with the federal government. In 2011, SBA approved four organizations to act as third-party certifiers. According to SBA data, these four third-party certifiers completed a total of about 3,400 certifications in fiscal year 2017. In 2014 we reviewed the WOSB program and found a number of deficiencies in SBA’s oversight of the four SBA-approved third-party certifiers and in SBA’s eligibility examination processes, and we made related recommendations for SBA. In addition, in 2015 and 2018 the SBA Office of Inspector General (OIG) reviewed the WOSB program and also found oversight deficiencies, including evidence of WOSB contracts set aside for ineligible firms. In both reports, the SBA OIG also made recommendations for SBA. Further, in July 2015, we issued GAO’s fraud risk framework, which provides a comprehensive set of key components and leading practices that serve as a guide for agency managers to use when developing efforts to combat fraud in a strategic, risk-based way. SBA Has Implemented One of the Three Changes Made by the 2015 NDAA As of early May 2019, SBA had implemented one of the three changes that the 2015 NDAA made to the WOSB program—sole-source authority. The two other changes—authorizing SBA to implement its own certification process for WOSBs and requiring SBA to eliminate the WOSB self-certification option—had not been implemented. The 2015 NDAA did not require a specific time frame for SBA to update its regulations. SBA officials have stated that the agency will not eliminate self-certification until the new certification process for the WOSB program is in place, which they expect to be completed by January 1, 2020. In September 2015, SBA published a final rule to implement sole-source authority for the WOSB program (effective October 2015). Among other things, the rule authorized contracting officers to award a contract to a WOSB or EDWOSB without competition, provided that the contracting officer’s market research cannot identify two or more WOSBs or EDWOSBs in eligible industries that can perform the requirements of the contract at a fair and reasonable price. In the final rule, SBA explained that it promulgated the sole-source rule before the WOSB certification requirements for two reasons. First, the sole-source rule could be accomplished by simply incorporating the statutory language into the regulations, whereas the WOSB certification requirements would instead require a prolonged rulemaking process. Second, SBA said that addressing all three regulatory changes at the same time would delay the implementation of sole-source authority. As of early May 2019, SBA had not published a proposed rule for public comment to establish a new certification process for the WOSB program. Previously, in October 2017, an SBA official stated that SBA was about 1–2 months away from publishing a proposed rule. However, in June 2018, SBA officials stated that a cost analysis would be necessary before the draft rule could be sent to the Office of Management and Budget for review. In response to the SBA OIG recommendation that SBA implement the new certification process, SBA stated that it would implement a new certification process by January 1, 2020. Further, in June 2018, SBA officials told us that they were evaluating the potential costs of a new certification program as part of their development of the new certification rule. On May 3, 2019, SBA officials explained that they expected to publish the proposed rule within a few days. In December 2015, SBA published an advance notice of proposed rulemaking to solicit public comments to assist the agency with drafting a proposed rule to implement a new WOSB certification program. In the notice, SBA stated that it intends to address the 2015 NDAA changes, including eliminating the self-certification option, through drafting regulations to implement a new certification process. The advance notice requested comments on various topics, such as how well the current certification processes were working, which of the certification options were feasible and should be pursued, whether there should be a grace period for self-certified WOSB firms to complete the new certification process, and what documentation should be required. Three third-party certifiers submitted comments in response to the advance notice of proposed rulemaking, and none supported the option of SBA acting as a WOSB certifier. One third-party certifier commented that such an arrangement is a conflict of interest given that SBA is also responsible for oversight of the WOSB program, and two certifiers commented that SBA lacked the required resources. The three third-party certifiers also asserted in their comments that no other federal agency should be allowed to become an authorized WOSB certifier, with one commenting that federal agencies should instead focus on providing contracting opportunities for women-owned businesses. All three certifiers also proposed ways to improve the current system of third-party certification—for example, by strengthening oversight of certifiers or expanding their number. The three certifiers also suggested that SBA move to a process that better leverages existing programs with certification requirements similar to those of the WOSB program, such as the 8(a) program. In the advance notice, SBA asked for comments on alternative certification options, such as SBA acting as a certifier or limiting WOSB program certifications to the 8(a) program and otherwise relying on state or third-party certifiers. SBA Has Not Fully Addressed Deficiencies in Oversight and Program Implementation SBA has not fully addressed deficiencies we identified in our October 2014 report, and these recommendations remain open. First, we reported that SBA did not have formal policies for reviewing the performance of its four approved third-party certifiers, including their compliance with their agreements with SBA. Further, we found that SBA had not developed formal policies and procedures for, among other things, reviewing the monthly reports that certifiers submit to SBA. As a result, we recommended that SBA establish comprehensive procedures to monitor and assess the performance of the third-party certifiers in accordance with their agreements with SBA and program regulations. In response to our October 2014 recommendation, in 2016 SBA conducted compliance reviews of the four SBA-approved third-party certifiers. The compliance reviews included an assessment of the third- party certifiers’ internal certification procedures and processes, an examination of a sample of applications from businesses that the certifiers deemed eligible and ineligible for certification, and an interview with management staff. SBA officials said that SBA’s review team did not identify significant deficiencies in any of the four certifiers’ processes and found that all were generally complying with their agreements. However, one compliance review report described “grave concerns” that a third- party certifier had arbitrarily established eligibility requirements that did not align with WOSB program regulations and used them to decline firms’ applications. SBA noted in the report that if the third-party certifier failed to correct this practice, SBA could terminate the agreement. As directed by SBA, the third-party certifier submitted a letter to SBA outlining actions it had taken to address this issue, among others. In January 2017, SBA’s Office of Government Contracting updated its written Standard Operating Procedures (SOP) to include policies and procedures for the WOSB program, in part to address our October 2014 recommendation. The 2017 SOP discusses what a third-party-certifier compliance review entails, how often the reviews are to be conducted, and how findings are to be reported. The 2017 SOP notes that SBA may initiate a compliance review “at any time and as frequently as the agency determines is necessary.” In March 2019, SBA provided an updated SOP, which includes more detailed information on third-party compliance reviews, such as how SBA program analysts should prepare for the review. However, the updated SOP does not provide specific time frames for how frequently the compliance reviews are to be conducted. In addition, in April 2018, SBA finalized a WOSB Program Desk Guide that discusses how staff should prepare for a compliance review of a third-party certifier, review certification documents, and prepare a final report. In March 2019, SBA provided GAO with an updated WOSB Program Desk Guide that contains information comparable to that in the 2018 version. Both Desk Guides do not describe specific activities designed to oversee third-party certifiers on an ongoing basis. Per written agreements with SBA, third-party certifiers are required to submit monthly reports that include the number of WOSB and EDWOSB applications received, approved, and denied; identifying information for each certified business, such as the business name; concerns about fraud, waste, and abuse; and a description of any changes to the procedures the organizations used to certify businesses as WOSBs or EDWOSBs. In our October 2014 report, we noted that SBA had not followed up on issues raised in the monthly reports and had not developed written procedures for reviewing them. At that time, SBA officials said that they were unaware of the issues identified in the certifiers’ reports and that the agency was developing procedures for reviewing the monthly reports but could not estimate a completion date. In interviews for our March 2019 report, SBA officials stated that SBA still does not use the third-party certifiers’ monthly reports to regularly monitor the program. Specifically, SBA does not review the reports to identify any trends in certification deficiencies that could inform program oversight. Officials said the reports generally do not contain information that SBA considers helpful for overseeing the WOSB program, but staff sometimes use the reports to obtain firms’ contact information. SBA’s updated 2019 SOP includes information on reviews of third-party certifier monthly reports, but it does not contain information on how staff would analyze the reports or how these reports would inform SBA’s oversight of third-party certifiers and related compliance activities, such as eligibility examinations. On May 3, 2019, SBA officials stated that, earlier in the week, they had initiated monthly meetings with the third-party certifiers. SBA officials explained that they intended to continue holding these monthly meetings to discuss best practices and potential issues related to the approval and disapproval of firms and to improve collaboration. Although SBA has taken steps to enhance its written policies and procedures for oversight of third-party certifiers, it does not have plans to conduct further compliance reviews of the certifiers and does not intend to review certifiers’ monthly reports on a regular basis in a way that would inform its oversight activities. SBA officials said that third-party certifier oversight procedures would be updated, if necessary, after certification options have been clarified in the final WOSB certification rule. However, ongoing oversight activities, such as regular compliance reviews, could help SBA better understand the steps certifiers have taken in response to previous compliance review findings and whether those steps have been effective. In addition, leading fraud risk management practices include identifying specific tools, methods, and sources for gathering information about fraud risks, including data on trends from monitoring and detection activities, as well as involving relevant stakeholders in the risk assessment process. Without procedures to regularly monitor and oversee third-party certifiers, SBA cannot provide reasonable assurance that certifiers are complying with program requirements and cannot improve its efforts to identify ineligible firms or potential fraud. Further, it is unclear when SBA’s final rule will be implemented. As a result, we maintain that our previous recommendation should be addressed—that is, that the Administrator of SBA should establish and implement comprehensive procedures to monitor and assess the performance of certifiers in accordance with the requirements of the third-party certifier agreement and program regulations. SBA also has not fully addressed deficiencies we identified in our October 2014 report related to eligibility examinations. We found that SBA lacked formalized guidance for its eligibility examination processes and that the examinations identified high rates of potentially ineligible businesses. As a result, we recommended that SBA enhance its examination of businesses that register for the WOSB program to ensure that only eligible businesses obtain WOSB set-asides. Specifically, we suggested that SBA should take actions such as (1) completing the development of procedures to conduct annual eligibility examinations and implementing such procedures; (2) analyzing examination results and individual businesses found to be ineligible to better understand the cause of the high rate of ineligibility in annual reviews and determine what actions are needed to address the causes, and (3) implementing ongoing reviews of a sample of all businesses that have represented their eligibility to participate in the program. SBA has taken some steps to implement our recommendation, such as including written policies and procedures for WOSB program eligibility examinations in an SOP and a Desk Guide. However, SBA does not collect reliable information on the results of its annual eligibility examinations. According to SBA officials, SBA has conducted eligibility examinations of a sample of businesses that received WOSB program set-aside contracts each year since fiscal year 2012. However, SBA officials told us that the results of annual eligibility examinations—such as the number of businesses found eligible or ineligible—are generally not documented. As a result, we obtained conflicting data from SBA on the number of examinations completed and the percentage of businesses found to be ineligible in fiscal years 2012 through 2018. For example, based on previous information provided by SBA, we reported in October 2014 that in fiscal year 2012, 113 eligibility examinations were conducted and 42 percent of businesses were found to be ineligible for the WOSB program. However, during our more recent review, we received information from SBA indicating that 78 eligibility examinations were conducted and 37 percent of businesses were found ineligible in fiscal year 2012. In addition, SBA continues to have no mechanism to look across examinations for common eligibility issues to inform the WOSB program. As we noted in 2014, by not analyzing examination results broadly, the agency is missing opportunities to obtain meaningful insights into the program, such as the reasons many businesses are deemed ineligible. Further, SBA still conducts eligibility examinations only of firms that have already received a WOSB award. In our October 2014 report, we concluded that this sampling practice restricts SBA’s ability to identify potentially ineligible businesses prior to a contract award. SBA officials said that while some aspects of the sample characteristics have changed since 2012, the samples still generally consist only of firms that have been awarded a WOSB set-aside. Restricting the samples in this way limits SBA’s ability to better understand the eligibility of businesses before they apply for and are awarded contracts, as well as its ability to detect and prevent potential fraud. We recognize that SBA has made some effort to address our previous recommendation by documenting procedures for conducting annual eligibility examinations of WOSB firms. However, leading fraud risk management practices state that federal program managers should design control activities that focus on fraud prevention over detection and response, to the extent possible. Without maintaining reliable information on the results of eligibility examinations, developing procedures for analyzing results, and expanding the sample of businesses to be examined to include those that did not receive contracts, SBA limits the value of its eligibility examinations and its ability to reduce ineligibility among businesses registered to participate in the WOSB program. These deficiencies also limit SBA’s ability to identify potential fraud risks and develop any additional control activities needed to address these risks. As a result, the program may continue to be exposed to the risk of ineligible businesses receiving set-aside contracts. In addition, in light of these continued deficiencies, the implementation of sole-source authority without addressing the other changes made by the 2015 NDAA could increase program risk. For these reasons, we maintain that our previous recommendation that SBA enhance its WOSB eligibility examination procedures should be addressed. SBA has also not addressed previously identified issues with WOSB set- asides awarded under ineligible industry codes. In 2015 and 2018, the SBA OIG reported instances in which WOSB set-asides were awarded using NAICS codes that were not eligible under the WOSB program, and our analysis indicates that this problem persists. Specifically, our analysis of data from the Federal Procurement Data System–Next Generation (FPDS–NG) on all obligations to WOSB program set-asides from the third quarter of fiscal year 2011 through the third quarter of fiscal year 2018 found the following: 3.5 percent (or about $76 million) of WOSB program obligations were awarded under NAICS codes that were never eligible for the WOSB program; 10.5 percent (or about $232 million) of WOSB program obligations made under an EDWOSB NAICS code went to women-owned businesses that were not eligible to receive awards in EDWOSB- eligible industries; and 17 of the 47 federal agencies that obligated dollars to WOSB program set-asides during the period used inaccurate NAICS codes in at least 5 percent of their WOSB set-asides (representing about $25 million). According to SBA officials we spoke with, WOSB program set-asides may be awarded under ineligible NAICS codes because of human error when contracting officers are inputting data in FPDS–NG or because a small business contract was misclassified as a WOSB program set-aside. Rather than review FPDS–NG data that are inputted after the contract is awarded, SBA officials said that they have discussed options for working with the General Services Administration to add controls defining eligible NAICS codes for WOSB program set-aside opportunities on FedBizOpps.gov—the website that contracting officers use to post announcements about available federal contracting opportunities. However, SBA officials said that the feasibility of this option was still being discussed and that the issue was not a high priority. Additionally, as of November 2018, the WOSB program did not have targeted outreach or training that focused on specific agencies’ use of NAICS codes, and SBA officials did not identify any targeted outreach or training provided to specific agencies to improve understanding of WOSB NAICS code requirements (or other issues related to the WOSB program). On May 6, 2019, an SBA official provided information that SBA has initiated a review to determine federal agencies’ use of ineligible NAICS codes and that SBA plans to share the findings with agencies and also provide training to procurement center representatives. Congress authorized SBA to develop a contract set-aside program specifically for WOSBs and EDWOSBs to address the underrepresentation of such businesses in specific industries. In addition, federal standards for internal control state that management should design control activities to achieve objectives and respond to risks, and that management should establish and operate monitoring activities to monitor and evaluate the results. Because SBA does not review whether contracts are being awarded under the appropriate NAICS codes, it cannot provide reasonable assurance that WOSB program requirements are being met or identify agencies that may require targeted outreach or additional training on eligible NAICS codes. As a result, WOSB contracts may continue to be awarded to groups other than those intended, which can undermine the goals of and confidence in the program. Federal Contracts to WOSB Set-Asides Remain Relatively Small While federal contract obligations to all women-owned small businesses and WOSB program set-asides have increased since fiscal year 2012, WOSB program set-asides remain a small percentage. Specifically, federal dollars obligated for contracts to all women-owned small businesses increased from $18.2 billion in fiscal year 2012 to $21.4 billion in fiscal year 2017. Contracts awarded to all women-owned small businesses within WOSB-program-eligible industries also increased during this period—from about $15 billion to $18.8 billion, as shown in figure 1. However, obligations under the WOSB program represented only a small share of this increase. In fiscal year 2012, WOSB program contract obligations were 0.5 percent of contract obligations to all women- owned small businesses for WOSB-program-eligible goods or services (about $73.5 million), and in fiscal year 2017 this percentage had grown to 3.8 percent (about $713.3 million) (see fig. 1). In summary, the WOSB program aims to enhance federal contracting opportunities for women-owned small businesses. However, as of early May 2019, SBA had not fully implemented comprehensive procedures to monitor the performance of the WOSB program’s third-party certifiers and had not taken steps to provide reasonable assurance that only eligible businesses obtain WOSB set-aside contracts, as recommended in our 2014 report. Without ongoing monitoring and reviews of third-party certifier reports, SBA cannot ensure that certifiers are fulfilling their requirements, and it is missing opportunities to gain information that could help improve the program’s processes. Further, limitations in SBA’s procedures for conducting and analyzing eligibility examinations inhibit its ability to better understand the eligibility of businesses before they apply for and potentially receive contracts, which exposes the program to unnecessary risk of fraud. Also, since SBA does not expect to finish implementing the changes in the 2015 NDAA until January 1, 2020, these continued oversight deficiencies increase program risk. As a result, we maintain that our previous recommendations should be addressed. In addition, SBA has not addressed deficiencies related to WOSB program set-asides being awarded under ineligible industry codes. Although SBA has updated its training and outreach materials for the WOSB program to address NAICS code requirements, it has not developed a process for periodically reviewing FPDS–NG data, and has yet to provide targeted outreach or training to agencies that may be using ineligible codes. As a result, SBA is not aware of the extent to which individual agencies are following program requirements and which agencies may require targeted outreach or additional training. Reviewing FPDS–NG data would allow SBA to identify those agencies (and contracting offices within them) that could benefit from such training. Without taking these additional steps, SBA cannot provide reasonable assurance that WOSB program requirements are being met. As such, we made one recommendation in our March 2019 report to SBA. We recommended that SBA develop a process for periodically reviewing FPDS–NG data to determine the extent to which agencies are awarding WOSB program set-asides under ineligible NAICS codes, and take steps to address any issues identified, such as providing targeted outreach or training to agencies making awards under ineligible codes. As of May 2019, this recommendation remains open. Chairman Golden, Ranking Member Stauber, and Members of the Subcommittee, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time. GAO Contact and Acknowledgments If you or your staff have any questions about this testimony, please contact William Shear, Director, Financial Markets and Community Investment at (202) 512-8678 or shearw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. GAO staff who made key contributions to this testimony are Andrew Pauline (Assistant Director), Tarek Mahmassani (Analyst in Charge), and Jennifer Schwartz. Other staff who made key contributions to the report cited in the testimony were Allison Abrams, Pamela Davidson, Jonathan Harmatz, Tiffani Humble, Julia Kennon, Rebecca Shea, Jena Sinkfield, Tyler Spunaugle, and Tatiana Winger. This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.