Background Several bureaus within Interior are responsible for the leasing, permitting, and inspecting of mineral extraction activities on federal lands and waters. Interior’s Bureau of Land Management (BLM) is responsible for onshore activities and manages approximately 700 million acres of subsurface mineral rights throughout the country, including the acreage it leases to companies for oil and gas development. At the end of fiscal year 2016, about 41,000 oil and gas leases accounted for approximately 28.2 million acres in 32 states, according to BLM data. For offshore oil and gas activities, the Bureau of Ocean Energy Management is generally responsible for leasing and resource planning and evaluation, among other functions, and the Bureau of Safety and Environmental Enforcement is generally responsible for permitting and inspecting as well as verifying production volumes on offshore leases, among other functions. Under the Outer Continental Shelf Lands Act, as amended, Interior is responsible for leasing and managing approximately 1.71 billion offshore acres. To begin the leasing process, Interior holds auctions through which companies may secure the rights to federal leases that allow them to drill for oil and gas upon meeting certain conditions. Once a company obtains a lease, it may conduct further exploration and subsequently determine whether it would like to drill a well. If a company plans to drill, it must first secure a permit from Interior. To secure a permit to drill under an onshore lease, a company must submit an application for a drilling permit to the appropriate BLM field office. BLM officials then evaluate the company’s proposal to ensure that it conforms to the relevant BLM land use plan for the area as well as applicable laws and regulations, including those focused on protecting the environment. To secure a permit to drill on offshore leases, a company must submit an application for a drilling permit to the Bureau of Safety and Environmental Enforcement, where it is reviewed for completeness and whether all technical elements conform to applicable regulations. Once a company secures a permit and begins producing, oil and gas is transported to market and sold. As part of this process, companies may elect to process the natural gas into various products before its sale. Under ONRR regulations, companies may deduct certain costs associated with transportation and natural gas processing from the royalties due. Companies can continue to produce oil and gas until the lease is no longer capable of producing in paying quantities, regardless of the length of the lease. To ensure compliance with applicable laws, regulations, and other requirements, both BLM and the Bureau of Safety and Environmental Enforcement have inspection and enforcement programs that are designed to verify that companies comply with all requirements at the lease site, including those related to measuring oil and gas volumes. The authority for inspecting wells and leases for this purpose is derived from FOGRMA. The act requires the Secretary of the Interior to develop guidelines that specify the coverage and frequency of inspections. Interior has delegated responsibilities for implementing the act to BLM for onshore leases and to the Bureau of Safety and Environmental Enforcement for offshore leases. ONRR’s Role in Collecting, Disbursing, and Verifying Royalties ONRR’s oversight of federal royalties includes collecting company-paid royalties, disbursing these royalties to appropriate accounts, and verifying the company-paid royalties through its compliance activities. Collecting: Companies that obtain federal onshore or offshore oil and gas leases are typically obligated to pay royalties on any oil or gas they produce from the leases and then sell. As a condition of producing oil and gas under federal and Indian leases, companies are required to submit two key monthly reports to ONRR—one specifying the total production and disposition of oil and gas and the other stating the royalties due based on production. However, because of various leasing and development arrangements made by companies, these two reports are often submitted by different companies. The companies physically developing the lease, referred to as the operators, are responsible for reporting the production volumes to ONRR in monthly production reports. The companies with a financial interest in the lease, referred to as the payors, are responsible for reporting the cash royalty owed on the federal and Indian oil and gas production in their monthly royalty reports. Each month, payors are to calculate the royalty payment owed to the federal government using the four key variables illustrated in the following equation: Royalty payment = ((volume sold x sales price) less deductions) x royalty rate Companies are to submit monthly production and royalty reports via a web-based portal to ONRR’s royalty information technology (IT) system. In addition to filing the royalty report with ONRR, companies typically make the actual cash royalty payment via an electronic fund transfer to an account at the Department of the Treasury (Treasury). Disbursing: Once ONRR reconciles the self-reported royalty payment data from the monthly royalty reports with the payments to Treasury, ONRR is to disburse the royalties from the Treasury account to the appropriate federal, state, tribal, or Individual Indian Money (IIM) accounts. All these transactions are to be recorded and stored in ONRR’s IT system. Verifying: ONRR is responsible for verifying royalties through its compliance program, which includes ensuring that the royalty revenues generated from the sale of oil and gas extracted from leased federal lands are accurately reported and paid. In conducting its compliance activities, ONRR is to assess the elements of the royalty equation: commodity price, volume of oil and gas, transportation and processing allowances, and royalty rate. ONRR also is to ensure that all relevant laws, regulations, and lease terms have been followed. ONRR has two key statutory requirements for its compliance program: FOGRMA and the Federal Oil and Gas Royalty Simplification and Fairness Act of 1996 (RSFA). FOGRMA requires that ONRR establish a comprehensive auditing system to provide the capability to accurately determine oil and gas royalties. RSFA directs ONRR not to conduct audit activities if it and the relevant state determine that the cost of conducting or requiring the audit exceeds the expected amount to be collected by the activity, based on the most current 12 months of activity. ONRR’s Work Planning Group identifies which companies or leases will be subject to compliance activities. The three primary levels of compliance activities ONRR conducts are audits, compliance reviews, and data mining—each of which provides a varying degree of assurance that royalties are accurately paid. Audits: According to ONRR documents, an audit involves detailed examinations of companies’ royalty payments and corresponding reporting to ONRR. As part of an audit, ONRR staff are to assess the accuracy and completeness of the companies’ self-reported production and royalty data compared to third-party documents, such as sales contracts and oil and gas sales receipts from pipeline companies. According to ONRR documents, it is to design its audits to ensure that royalty payments and other obligations to ONRR are in substantial compliance with applicable lease terms, federal laws and regulations, and other policies. Compliance reviews: ONRR describes compliance reviews as an analysis designed to determine the reasonableness of company- reported production and royalty data. In contrast to audits, compliance reviews are quicker, more limited checks on the accuracy and completeness of companies’ self-reported data and do not include systematically examining the underlying source documentation used to generate the self-reported data. Data mining: ONRR began its data mining program in 2011 and officially organized it within the compliance program beginning in fiscal year 2018. Data mining is a partially automated activity to identify and resolve data errors prior to audits and compliance reviews. According to ONRR officials, data mining examines large sets of company- reported data for certain common errors, such as irregularities in the volume of oil or gas extracted. Officials stated that data mining generally identifies obvious data errors that ONRR staff work with companies to correct. The process companies are to follow to produce oil and gas from federal leases, bring it to market, transmit required data to Interior, and pay royalties is outlined in figure 1. The State and Tribal Royalty Audit Committee FOGRMA authorizes the Secretary of the Interior to enter into cooperative agreements with states to share oil and gas royalty management information and carry out inspection, audit, investigation, and enforcement activities on federal and Indian lands. Currently, the nine states that are members of STRAC have delegated authority to conduct compliance activities for federal lands in their respective state. These agreements form the framework of ONRR’s relationship with states for mineral revenue compliance activities. A governor or other appropriate official with delegation authority may request that Interior enter into a cooperative agreement with a state by sending a letter to the Director of ONRR. States may also elect to end these agreements at their discretion with a 120-day notice. States have a vested interest in ensuring that all royalties are paid accurately because states receive a portion of the royalties that the federal government collects, including additional collections resulting from compliance activities identifying underpayment. ONRR also reimburses states for the costs of performing approved and eligible compliance activities, including compliance activities under the cooperative agreement. State audit offices that have entered into agreements with ONRR are to submit yearly work plans identifying the compliance activities they propose to conduct in the next fiscal year, which ONRR is to review and approve. Member states can conduct both audits and compliance reviews, and ONRR requires that the state auditors follow the procedures established in generally accepted government auditing standards and ONRR’s audit and compliance review manuals. To ensure that compliance activities are conducted in accordance with generally accepted government auditing standards and relevant ONRR manuals, states are to undergo an external peer review every 3 years, during which they are assessed on their adherence to the standards and manuals and whether they provided corrective actions to any identified problems. Performance Management Information and Controls A key practice in results-oriented management for federal agencies is establishing agency-wide, long-term strategic goals. The Government Performance and Results Act of 1993 (GPRA), which was significantly enhanced by the GPRA Modernization Act of 2010 (GPRAMA), requires federal agencies, among other things, to develop strategic plans with long-term, outcome-oriented goals; annual goals linked to achieving the long-term goals; and annual reports on the results achieved, as assessed through the use of performance measures and targets. Federal departments and agencies must comply with these requirements and are to follow associated Office of Management and Budget guidance when developing their agency-wide strategic plans. We have reported that these requirements also can serve as leading practices for strategic planning at lower levels within federal agencies, such as planning for individual divisions, programs, or initiatives. These leading practices include defining the mission and goals of an agency or a specific program and developing and using performance measures that allow an agency to track its progress toward its mission and goals. ONRR issued a fiscal year 2017 strategic priorities document that contains the agency’s mission statement: “to collect, account for, and verify natural resource and energy revenues due to states, American Indians, and the U.S. Treasury.” ONRR stated in the document that it planned to achieve Interior’s strategic goals to (1) timely disburse 98 percent of federal and Indian revenues, (2) close 85 percent of Interior’s OIG and GAO recommendations targeted for implementation in fiscal year 2017, and (3) report results of ONRR’s supporting performance measures for Interior’s strategic goals on total ONRR compliance collections and a 3-year average compliance return on investment. ONRR also stated that it planned to create an ONRR strategic plan. History of Oil and Gas Royalty Oversight Challenges In the 1970s and early 1980s, we and Interior’s OIG reported on Interior’s management of the oil and gas revenue collection system. Interior’s OIG issued five reports critical of the program from 1969 through 1977 that raised concerns about royalty collections. In 1981, we reported that Interior was not collecting potentially hundreds of millions in royalties due from federal oil and gas leases. In response, in 1981, the Secretary of the Interior established the Commission on Fiscal Accountability of the Nation’s Energy Resources, also known as the Linowes Commission, to investigate allegations of irregularities in royalty payments, among other issues. The Linowes Commission raised a number of concerns, and its 1982 report stated that management of royalties for the nation’s energy resources had been a failure for more than 20 years. The report found that because the federal government had not adequately managed this multibillion-dollar enterprise, the oil and gas industry was not paying all the royalties it rightly owed. The report cited a range of problems, including the failure to verify data that companies reported as well as late payments and underpayments. Following this report, Interior and Congress took actions aimed at improving revenue collection, including reorganizing oil and gas revenue collections under a new bureau within Interior, passing FOGRMA in 1982, and passing RSFA in 1996. In December 2006, Interior’s OIG analyzed ONRR’s compliance processes and issued a report that made several recommendations to improve these processes and the agency’s systems for tracking them. The report identified deficiencies with how ONRR maintained compliance-related information and recommended changes for how ONRR measures its compliance activities’ performance. In 2007, Interior’s Subcommittee on Royalty Management—a subcommittee of the Royalty Policy Committee—issued a report that reiterated several of the findings from Interior’s OIG report on ONRR and further stated that several aspects of royalty management activities required prompt and, in some cases, significant management attention. In particular, the report included over 100 recommendations for improving Interior’s management of oil and gas resources, including recommendations related to audit, compliance, and enforcement. Appendix I provides a list of the subcommittee’s recommendations and the status of their implementation, according to Interior documents and interviews with Interior officials. We identified several challenges with Interior’s management of federal oil and gas in the 2000s. In February 2011, in part because of the challenges identified in our past work, we added Interior’s management of federal oil and gas resources to our list of program areas at high risk for fraud, waste, abuse, and mismanagement. In the March 2019 update of our High-Risk List, we found that Interior had made progress improving its management of federal oil and gas resources. However, additional steps are needed to improve Interior’s royalty determination and collection. Recent ONRR Initiatives ONRR, according to officials, has begun implementing several initiatives that seek to make the agency operate more effectively. In March 2017, ONRR initiated Boldly Go, an effort to assess its organizational structure and identify and implement potential improvements. According to ONRR officials, this initiative was in response to March 2017 comments from the Secretary of the Interior, in which he said the department, in general, should undergo a “bold restructuring.” ONRR officials said that the Boldly Go organizational restructuring was implemented in October 2017 and included several changes to how ONRR conducts its compliance work. Before the reorganization, audits and compliance reviews were part of the same management group—referred to as the Audit and Compliance Management group. After the reorganization, audits and compliance reviews are managed separately. Audits now have their own management group, referred to as Audit Management. According to ONRR officials, the new Audit Management group conducts audits of multiple companies and properties and will attempt to identify more systemic misreporting issues common to those companies and properties. ONRR staff who conduct compliance reviews were moved into the same management group as the data mining staff in the new Compliance Management group. According to ONRR officials, the merger occurred because both groups use similar data sources to conduct less in-depth checks of the royalty data than audits. Additionally, officials stated that putting these activities under the same management could assist in better targeting companies for similar compliance issues. Prior to the reorganization, identifying and selecting cases for audits and compliance reviews was a function of the Audit and Compliance Management group. After the reorganization, this function was moved to a new Analytics and Risk Management group that is also tasked with using data analytics methods, such as computerized analysis of spatial and geographic data, to better identify noncompliant royalty payments. ONRR is also in the process of implementing a new electronic compliance case management and work paper tool referred to as the Operations and Management Tool (OMT). According to ONRR documents, OMT is to combine multiple systems into one and is intended to serve a variety of functions. ONRR documents state that OMT is designed to be a single standardized system that reduces manual data entry, creates a single system of record for ONRR case data, offers error checks to eliminate data entry errors, and provides greater transparency for outside auditors. One ONRR official stated that the agency plans to have ONRR’s data mining, compliance review, and audit teams all using OMT to manage their compliance work in 2019. According to some ONRR and state audit officials, ONRR piloted OMT’s electronic compliance case management system in North Dakota in 2018, and ONRR expects to offer OMT as an option to other STRAC partners for their audit and compliance review case management needs. Finally, the agency introduced a new auditor training curriculum in April 2018. Shortly after new auditors are hired, they are expected to begin ONRR’s training program, and according to ONRR’s training manual, they are expected to complete the training within 2 years of their hire dates. According to ONRR officials, courses will also be available to existing audit staff upon request. ONRR Reported Generally Meeting Annual Royalty Program Compliance Goals, but Its Goals May Not Align with the Agency’s Mission ONRR reported generally meeting its annual royalty compliance goals for fiscal years 2010 through 2017. To meet its compliance goals, ONNR used all three levels of compliance activities—audits, compliance reviews, and data mining—each of which provides a different level of assurance. However, ONRR’s compliance goals may not align with the agency’s mission to ensure the accuracy of royalty payments and other statutory requirements. ONRR Reported Generally Meeting Compliance Goals and Revised Its Goals Multiple Times ONRR reported generally meeting its annual compliance goals—those from Interior’s strategic plan and bureau-specific goals—for its royalty compliance program for fiscal years 2010 through 2017, and the agency made multiple revisions to its goals during this period. Our analysis of Interior’s annual budget justifications for fiscal years 2010 through 2017 found that ONRR reported meeting its compliance goals for 6 of the 8 fiscal years we reviewed (see table 1). According to ONRR officials we interviewed, the 2 years when the agency did not report meeting its compliance goals largely resulted from a shift in the focus of its goals that created a short-term misalignment of planned work and available resources. During fiscal years 2010 through 2017, ONRR revised its annual compliance goals multiple times. These included both compliance goals supporting Interior’s strategic plans covering fiscal years 2007 through 2018 and bureau-specific goals. In the revisions to its compliance goals, ONRR generally shifted from goals focused on the extent to which its compliance program was ensuring the accuracy of royalty payments to those focused on the efficiency of the program. ONRR’s accuracy goals, which included conducting compliance activities to cover a specific percentage of royalties, companies, or properties, helped it assess the extent to which it was ensuring the accuracy of royalty payments. That is, by measuring the portion of, for example, royalties subject to compliance activities, it was able to quantify the percentage of royalties that were reasonably correct or accurate. ONRR’s efficiency goals, which included conducting compliance activities to obtain a certain return on investment and additional amount of royalty collections, helped it assess whether resources spent on compliance activities were used cost effectively. According to ONRR officials, these revisions were made in an effort to continually improve its compliance performance. Table 2 identifies ONRR’s annual compliance goals for fiscal years 2010 through 2017 and establishes two categories for these goals corresponding to ONRR’s requirements under FOGRMA and the RSFA. Appendix II provides more detailed information on ONRR’s annual compliance goals and the agency’s reported compliance program performance. Conduct compliance activities on specified percentage of companies (coverage) X Conduct compliance activities on specified percentage of properties (coverage) X Conduct compliance activities on specified percentage of companies (coverage) X Conduct compliance activities on specified percentage of companies (coverage) X Conduct compliance activities on specified percentage of companies (coverage) X Conduct compliance activities on specified percentage of payors and operators (coverage) X Conduct compliance activities on specified percentage of payors and operators (coverage) Generate specified return on investment from compliance activities Generate specified return on investment from compliance activities Generate specified amount in total additional royalties Generate specified return on investment from compliance activities Generate specified amount in total additional royalties Legend: FOGRMA = Federal Oil and Gas Royalty Management Act of 1982, as amended; N/A = not applicable; RSFA = Federal Oil and Gas Royalty Simplification and Fairness Act of 1996; — = does not apply. While it is within ONRR’s purview to revise its compliance goals or targets, frequent changes may complicate management’s ability to assess performance over time because consistent goals are needed as a baseline from which to assess performance. For example, for the time period we reviewed, ONRR revised its compliance goals or goal targets nearly every year. This makes it difficult to assess, for example, how variations in resource allocations to and among its compliance activities may have affected the compliance program’s performance. The following are the types of compliance goals that ONRR used and revised for the period: Royalty coverage goal. Prior to fiscal year 2010, one of ONRR’s compliance goals was to conduct compliance activities on a specified percentage of royalties within 3 years of the date it received payment. In December 2006, Interior’s OIG issued an audit report that found, among other things, that the royalty compliance coverage goal had reduced the number of companies and properties subject to compliance work. The report stated that ONRR should consider modifying its compliance program strategy to ensure appropriate coverage of properties and companies within a reasonable time frame even if this resulted in a reduction in the overall percentage of dollars covered and recommended that ONRR develop separate performance measures for companies and properties subjected to compliance coverage. ONRR concurred with the recommendation and developed an implementation action plan. For fiscal year 2010, ONRR eliminated its royalty coverage goal in response to the OIG recommendation. Company/operator/payor and property coverage goals. For fiscal year 2010, ONRR revised its compliance goals to address company and property coverage, or conducting compliance activities—including audits and compliance reviews—on a certain percentage of companies and properties. ONRR’s fiscal year 2010 budget justification stated that the new compliance goals would reflect the cumulative percentage of unique companies and properties covered by audits, compliance reviews, or the royalty-in-kind compliance strategy. For fiscal year 2010, ONRR’s company coverage goal was to cumulatively conduct compliance activities on 57.6 percent of companies that paid royalties from fiscal years 2008 through 2012. ONRR’s property coverage goal was to cumulatively conduct compliance activities for 35 percent of properties where oil and gas had been extracted and sold from fiscal years 2008 through 2012. For fiscal year 2011, ONRR revised its compliance goals, eliminating the property coverage goal. ONRR also revised its company coverage goal to cumulatively conduct compliance activities on 66 percent of companies that paid royalties for fiscal years 2011 through 2016. ONRR further revised this goal for fiscal year 2014 to consider operators and payors instead of companies. In fiscal year 2014 ONRR established a compliance goal of conducting compliance activities on 90 percent of operators and payors but dropped the goal to 52 percent for fiscal year 2015. The goal for covering a percentage of operators and payors was eliminated beginning in fiscal year 2016, which left ONRR without a compliance goal addressing its coverage of royalty payments. Agency officials we interviewed told us that they eliminated ONRR’s company coverage goal because they concluded that the compliance program was reviewing too many companies and properties with smaller royalty payments, which officials deemed an inefficient use of limited compliance resources. ONRR officials added that budgetary constraints and the complexity of company bankruptcies and consolidation in the oil and gas industry also contributed to the goal’s elimination. Additionally, in 2008, ONRR established a data mining program to examine large sets of operator-reported data to identify royalty and reporting errors, such as when the production volumes that payors and operators reported for the same lease did not match. This work led to additional royalty collections, but ONRR did not consider these results when calculating its annual performance measure for company and property coverage. According to officials, data mining was the responsibility of ONRR’s Royalty Reporting group and was not considered compliance work. Return on investment goal. ONRR has had a goal for return on investment for fiscal years 2010 through 2017 that measured the efficiency of the compliance work that all of its program areas performed. However, Interior elevated this goal from a bureau-specific goal to a strategic plan goal for fiscal year 2017. This goal is a ratio of costs to collections for compliance activities—and is to assess whether ONRR collected additional royalties for every additional dollar the agency spends on compliance reviews, audits, and data mining. To account for variations in collections and oil and gas prices, ONRR is to calculate its performance on return on investment based on the royalties from the previous 3 years. For example, the return on investment the agency reported for fiscal year 2017 was based on revenues collected from fiscal years 2014, 2015, and 2016. According to ONRR officials, the goal for fiscal year 2017—to collect an additional $2 in royalties for every $1 spent on compliance activities—was developed based on trends from prior years. Achieving this return on investment would indicate that ONRR met its goal. Total additional royalty collections goal. In fiscal year 2016, ONRR developed a bureau-specific goal for total additional royalties collected from compliance reviews, audits, and data mining. The goal for fiscal year 2016 was to collect an additional $110 million from compliance activities. In the following fiscal year, 2017, ONRR elevated this goal to a strategic plan goal and kept the amount the same, at $110 million in additional royalties. ONRR Used All Levels of Compliance Activities to Generally Meet Goals To generally meet its compliance goals during fiscal years 2010 through 2017, ONRR used all levels of its compliance activities: audits, compliance reviews, and data mining. The number of audits completed annually generally remained the same for fiscal years 2010 through 2017, declining slightly from 162 in 2010 to 153 in 2017. During the same time period, the number of completed compliance reviews decreased, declining from 1,233 in 2010 to 683 in 2017 (see fig. 2). During this time frame, ONRR’s Data Mining group increased the number of exceptions resolved to address instances of incorrectly reported data from 4,323 in 2010 to over 26,000 in 2017. Our analysis of ONRR’s data on compliance activities showed that adding data mining financial results in 2011 was associated with a decrease in the return on investment for ONRR’s other compliance activities. Prior to including data mining, compliance reviews earned a 6 to1 return on investment, STRAC compliance work earned about a 4 to1 return on investment, and audits earned a 2 to1 return on investment. By the end of fiscal year 2017, data mining proved to be far more cost-effective for royalty compliance, with a return on investment of 9 to1. During the same time, return on investment declined for all compliance reviews (including STRAC compliance work) and all audits (see fig. 3). According to ONRR officials, the reason for this decline was that data mining was identifying royalties that might otherwise have been identified through audits or compliance reviews. Additionally, ONRR officials we interviewed stated that data mining has been more cost-effective than audits or compliance reviews in identifying additional royalties. Officials we interviewed stated that data mining often identifies more simple reporting errors. Return on investment is an indicator of the efficiency of ONRR’s compliance program. As long as ONRR is collecting more royalties through its compliance activities than it is spending on identifying those royalties, the federal government will obtain additional revenues. According to ONRR officials we interviewed, the agency does not calculate the potential additional royalty revenues that would be generated if it conducted additional compliance activities. However, ONRR officials said they do calculate the effect of reduced funding on compliance activities. For example, ONRR stated in its fiscal year 2018 budget justification document that reductions in its budget for compliance work would directly result in reductions to additional royalty collections. ONRR’s Goals May Not Align with Agency Mission and Statutory Requirements to Account for Royalty Payments ONRR’s fiscal year 2017 compliance goals, the most recent compliance goals we reviewed, may be useful for assessing certain aspects of ONRR’s performance but may not be effectively aligned with the agency’s stated mission or fulfill other statutory requirements. ONRR’s 2017 strategic priorities document states that the agency’s mission is to collect, account for, and verify energy revenues. Additionally, statutory requirements under RSFA direct ONRR not to conduct audit activities if it and the relevant state determine that the cost of conducting or requiring the audit exceeds the expected amount to be collected by the activity, based on the most current 12 months of activity. ONRR’s fiscal year 2017 return on investment compliance goal helps the agency comply with RSFA by assessing whether the agency’s compliance program is cost- effective. Moreover, ONRR’s statutory requirements under FOGRMA require that it establish a comprehensive auditing system to provide the capability to accurately determine oil and gas royalties, among other requirements. However, ONRR’s fiscal year 2017 compliance goals do not sufficiently address its mission or FOGRMA requirements, in part, because its goals do not address accuracy—or consider the extent to which its compliance work is covering, for example, royalty payments. While ONRR previously had coverage goals, agency officials told us that they eliminated their company and property coverage goals because they concluded the compliance program was reviewing too many companies and properties with smaller royalty payments. ONRR officials told us that this was deemed an inefficient use of limited compliance resources. However, it is difficult for ONRR to provide reasonable assurance that it is accurately collecting royalties when it does not have data on the extent to which, for example, royalties or companies were subject to compliance activities. According to agency officials we interviewed, ONNR stopped tracking these data when ONRR eliminated its coverage goals for fiscal year 2016. As a result, ONRR could be determining that it is meeting its current annual compliance goals but potentially doing so by examining a small percentage of royalties or companies. For example, ONRR may be able to achieve a 2 to 1 return on investment, but only conduct compliance activities on 10 percent of the approximately $5 billion in royalties paid in calendar year 2017. This raises questions about the extent to which ONRR can provide reasonable assurance that its compliance program is assessing the accuracy of oil and gas royalty payments because it does not have a goal for, or data on, the amount of royalties subject to compliance activities. Finally, because ONRR no longer has a coverage goal—which helps it assess the extent to which it has ensured the accuracy of royalty payments—it does not track the amount of royalties subject to its differing level of compliance activities. ONRR has established a compliance program with three activities—audits, compliance reviews, and data mining—each of which offers varying levels of assurance for determining the accuracy of royalty payments. However, the extent to which its compliance program allows ONRR to accurately determine and collect royalty payments is unclear because the agency does not track each compliance activity’s contribution toward a coverage goal. Interior’s OIG reported a similar finding in December 2006. In its report, the OIG found that ONRR’s compliance goal for coverage of royalties was misleading because it weighed audits and compliance reviews equally, although the two compliance activities provided differing levels of assurance about whether royalties were accurately paid. The OIG recommended that ONRR should revise the compliance goal to account for each compliance activity separately. While ONRR did not concur with establishing a goal for each of the compliance activities, it agreed to internally track separate measures for them. According to ONRR documentation, the agency took steps to identify what amount of royalties was covered by audits or compliance reviews but did not report this information. When ONRR eliminated its coverage goals for fiscal year 2016, it no longer tracked information on the extent to which royalty payments were subject to its different levels of compliance activities. By establishing a coverage goal (e.g., identifying the number of companies or percentage of royalties subject to compliance activities over a set period of time) that aligns with the agency’s mission and tracking the extent to which each of its compliance activities contributes toward this goal, ONRR would have greater assurance that its compliance program has the capability to accurately determine oil and gas royalties. ONRR’s Process to Select Compliance Cases Is Not Documented and May Not Align with Compliance Goals ONRR’s process to select compliance cases for audits and compliance reviews is not documented. Additionally, the agency does not have performance measures for determining whether its case selection process aligns with the agency’s compliance goals. Finally, while ONRR has a risk model to assist in selecting compliance cases, it has not analyzed the effect the risk model has had on its selection process. ONRR Does Not Have a Documented Case Selection Process for Audits and Compliance Reviews ONRR does not have a documented case selection process with procedures for how to select cases. According to ONRR officials, ONRR’s Work Planning Group reviews royalty information on federal oil and gas leases and selects leases from specific companies or properties to undergo either an audit or a compliance review. These officials also stated that while the process for selecting cases for audits and compliance reviews differs, the agency has no written procedures for either compliance activity on how cases should be selected. For audits, ONRR officials we interviewed told us that cases are generally selected based on research from ONRR’s recently established Analytics and Risk Management group, which includes the relocated Work Planning Group and other offices that analyze particular aspects of the oil and gas industry, such as pricing. According to these officials, the work planners or analytics staff review a variety of royalty payment and oil and gas production information to identify trends and outliers that may indicate potential royalty noncompliance. The officials told us that they also consider other factors in their selection decisions, such as whether a company was new—and therefore may be unfamiliar with how to correctly report royalties—or had undergone a change in ownership—which can lead to reporting errors. Additional factors that ONRR officials told us they considered were referrals by ONRR staff based on recently completed compliance activity on a specific company or property and the risk scores for the relevant companies and properties generated from the agency’s compliance risk model. For compliance reviews, ONRR officials we interviewed told us that the Work Planning Group includes information from a Go/No-go analysis, which they said allows ONRR to make a decision early in the process to cost effectively decide whether to initiate a compliance review. According to ONRR officials, the use of the Go/No-go analysis began in fiscal year 2015 as means to better ensure that the compliance activities they select will identify a finding of royalty noncompliance. According to the officials, the Work Planning Group then compares the list of companies and properties to other sources of information, such as the findings of recently completed work and ONRR’s royalty compliance risk model, to select cases based on the group’s professional judgment. Overall, ONRR officials we interviewed said that the Work Planning Group maintains a small pool of cases for either an audit or compliance review for when staff become available after completing other work. ONRR officials said that as there was no requirement that they develop documented procedures for case selection, they rely on the experience and training of members of the Work Planning Group to review the available information and select cases based on requests from the Audit Management and Compliance Management groups. Under federal standards for internal control, management should establish an organizational structure, assign responsibility, and delegate authority to achieve the entity’s objectives, including the documentation of the internal control system. Documentation provides a means to retain organizational knowledge and mitigate the risk of consolidating that knowledge to a few personnel, as well as a means to communicate that knowledge as needed to external parties, such as external auditors. By developing a documented case selection process that includes procedures for how to select compliance cases, ONRR could better ensure that it retains the organizational knowledge needed to carry out the process effectively and can defend it to external parties. ONRR Does Not Have Performance Measures for Determining Whether the Way Such Cases Are Selected Contributes to Overall Compliance Goals ONRR does not have performance measures for determining the extent to which cases selected align with the agency’s compliance goals. ONNR’s Work Planning Group is responsible for selecting cases—that is, companies or properties—to undergo a compliance review or audit. As mentioned previously, ONRR’s fiscal year 2017 goals for its compliance program are to achieve a specified return on investment and total amount of additional royalties collected from the cases it selects to undergo compliance activities. However, according to ONRR officials we interviewed, these goals are not considered when selecting cases. Rather, these officials told us that the Work Planning Group attempts to select cases for compliance activities that are the most likely to result in a finding of royalty noncompliance. A finding of noncompliance for a company can result from a variety of circumstances, such as reporting an incorrect volume of oil or gas sold or claiming allowances for transportation and processing costs above established limits. ONRR officials stated that the agency’s IT system tracks whether a completed compliance case resulted in a finding, but the agency does not regularly assess the percentage of completed cases that produce findings. Prior to 2015, ONRR had performance measures for determining the extent to which cases selected aligned with its compliance goals but stopped using these measures after it made changes to the goals. Prior to fiscal year 2010, for example, ONRR had a goal for conducting compliance activities on a certain percentage of royalties within 3 years from the date it received payment. To support this goal, ONRR sought to select companies with relatively high royalty dollar amounts. ONRR then assessed its performance toward achieving this goal by reviewing all completed audits and compliance reviews over a 3-year period and calculating the percentage of total royalties paid over this period from completed compliance cases. For example, in fiscal year 2008, ONRR reported that compliance cases covered 69 percent of royalties received in calendar year 2004. However, as we noted previously, the 2006 OIG report found that the focus on coverage of royalties resulted in ONRR providing limited coverage of its universe of companies and properties. Additionally, ONRR officials we interviewed confirmed that selecting cases with higher royalty amounts to achieve the royalty coverage goal resulted in more limited coverage of companies because the goal directed ONRR toward repeatedly selecting many of the same large companies each year for compliance activities. In response to the recommendations in the OIG’s report as well as ONRR’s own recognition of the reduced company coverage resulting from its selection of companies that pay high royalties, ONRR transitioned to a new performance measure for case selection along with a new compliance goal in fiscal year 2010. ONRR’s new performance measure assessed the number of unique companies and properties for selected compliance activities. This new performance measure, according to ONRR officials, was driven by ONRR’s new compliance goal for cumulatively covering a certain percentage of unique companies and properties over a 3-year period. According to ONRR officials, after the company and property coverage goal was in place for approximately 5 years, officials determined that this goal and corresponding performance measure was driving the compliance case selection process to select too many companies and properties with smaller royalty payments, which they deemed an inefficient use of limited compliance resources. As a result, ONRR officials told us that the agency decided to change its compliance goal in fiscal year 2015 to focus on return on investment and total additional dollars collected. However, ONRR did not establish a corresponding performance measure for its compliance case selection process that would determine the extent to which cases selected contributed to ONRR’s compliance goals. As stated previously, we have reported that the requirements in GPRA and GRPAMA for establishing performance metrics serve as leading practices for divisions, programs, and initiatives. Performance measures help agencies make resource decisions, provide managers information on which to base their organizational and management decisions, and create powerful incentives to influence organizational and individual behavior. Furthermore, successful performance measures are aligned with division and agency-wide goals and missions. According to ONRR officials we interviewed, they have not established performance measures for determining whether the way such cases are selected aligns with the agency’s compliance goals because there is no specific requirement to do so. By developing performance measures (e.g., establishing a specified percentage of compliance cases that identify findings of royalty noncompliance or total additional royalties) that assess whether the agency is selecting cases that are helping it achieve its compliance goals, ONRR would be able to better monitor its performance in achieving its goals and whether changes to its selection process affect its performance. ONRR Has Not Analyzed the Effectiveness of Its Risk Model on Case Selection ONRR has developed a model that assesses the risk of noncompliance for companies and properties. Officials from the Work Planning Group use this model to inform their compliance case selections. However, it is unclear whether use of the model has improved case selection because ONRR has not analyzed the model’s effect on such selections. ONRR began a pilot program in 2006 to analyze the risk factors for royalty noncompliance, which included developing a quantitative risk model. In December 2006, Interior’s OIG recommended that ONRR consider additional factors that may indicate a risk of noncompliant royalty payments when making case selection decisions. In addition to the factors that ONRR was using to select cases to help achieve its compliance goals, such as cases with high royalty dollars, the OIG recommended that ONRR incorporate other risk factors, including companies or properties having a history of underreported royalties and falsely reported information to other federal agencies, such as the Environmental Protection Agency. In December 2007, Interior’s Subcommittee on Royalty Management reiterated the importance of using a risk-based process for compliance and made a number of related recommendations to ONRR. Among these were that ONRR should fully implement the quantitative model it was developing as part of its pilot program. Additionally, the subcommittee recommended that ONRR evaluate its risk model’s performance and then establish a process to continually validate and update the model to ensure that it remains effective. In response to these recommendations, ONRR worked with a contractor from 2006 through 2012 to develop an initial risk model. This model evaluated the risk of royalty noncompliance for each lease based on four characteristics: the type of lease, the specific location of the lease, the region of the country the lease was in, and the type of commodity extracted. The model looked at a number of indicators of risk, which were grouped into four overall risk drivers: complexity of the oil and gas market, complexity of regulations, commodity-specific practices, and transparency of the market. According to officials from the Work Planning Group we interviewed, they used the risk scores generated from this model to help inform the list of compliance cases to be reviewed the following year. These officials told us that they stopped using the scores from this model around 2012 for two reasons. First, the model allowed for the scores to be manually weighted based on the judgment of those selecting the compliance cases, and this weighting process was believed to have eventually hurt the accuracy of the risk scores. Second, agency officials determined that the risk scores the model was producing did not correlate closely with cases resulting in significant findings. In 2013, ONRR tasked a different contractor with developing a new set of risk models. According to an initial development document, ONRR requested separate risk models for companies and properties that would determine the propensity for a company to submit an incorrect royalty payment using historical royalty compliance data that ONRR and third- party sources provided. The contractor produced two risk models, one that assigned a risk score to companies and one to properties. The risk scores—which ranged from 0 to 100—attempted to quantify the risk of royalty noncompliance. The initial models were completed in 2014, and ONRR began including the risk scores from these models in the data that the Work Planning Group reviewed during the case selection process. Documents from the contractor show that the models then went through an initial validation process using the results of cases that the contractor selected when the models were instituted in 2014 and completed cases from 2012 onward. According to documents summarizing the contractor’s efforts, the validation showed a correlation between higher risk scores on the company model and cases that resulted in findings and additional royalty revenues. However, the contractor reported that higher risk scores on the property model did not correlate with either findings or additional royalty revenues. The documents we reviewed also included a number of recommendations to ONRR to improve its risk modeling, including adding third-party and commercial data sources, adding data sources from within ONRR and other oil and gas bureaus within Interior, and attempting to redefine property risk and building a new property risk model. However, according to officials we interviewed, they have not yet acted on any of these recommendations. ONRR requested that the contractor update the models with data from recent royalty reporting and completed compliance cases, which it did in both 2015 and 2018 but does not do either regularly or periodically. According to ONRR officials, the Work Planning Group currently considers the risk scores based on the company model when selecting cases but does not consider the risk scores based on the property model, as the group considers those scores less reliable. ONRR officials told us that they do not believe that their current risk approach is entirely effective and are considering having staff from the Analytics and Risk Management group develop a risk model for the agency. To date, ONRR has not analyzed how the use of the risk scores has affected case selection or findings of royalty noncompliance and is therefore unable to identify whether its risk model is effective. As a result, the agency does not have sufficient information to make a decision on whether to continue using the model as it exists today, consider potential improvements, or discontinue the model in favor of another approach. Federal standards for internal control state that management should design control activities to achieve objectives and respond to risks, such as by comparing actual performance to planned or expected results and analyzing significant differences. By periodically analyzing whether the risk model is effectively identifying potential royalty noncompliance and whether the model’s results are being effectively used to assist in case selection, and making changes to the model (e.g., updating it) or developing a new model based on this analysis, ONRR would be better able to determine how to proceed with using risk analysis to inform its case selections. STRAC Members Reported They Are Satisfied with ONRR Coordination but Do Not Have Documented Processes, and Compliance Activities in Their Work Plans Do Not Align with ONRR Goals STRAC Officials Expressed Satisfaction with ONRR’s Coordination on Royalty Compliance STRAC officials we interviewed from the nine member states that had agreements with ONRR for conducting royalty compliance generally expressed satisfaction with ONRR’s coordination of compliance activities, including both the frequency of interaction as well as support for budget and training. STRAC officials from all nine member states generally expressed satisfaction with the frequency of interaction between STRAC and ONRR. STRAC officials stated that this interaction occurred primarily through three mechanisms. First, ONRR and STRAC hold semiannual in-person meetings. At these meetings, STRAC officials said that attendees discuss a range of topics. For example, at the March 2018 STRAC meeting in Sacramento, California, which we attended, there were two training sessions as well as a session on updates to ONRR’s IT systems. Second, ONRR and STRAC hold quarterly teleconferences. These teleconferences, according to STRAC officials, are opportunities for both ONRR and STRAC to highlight any significant or systematic issues that they may be identifying in their compliance activities. Third, ONRR assigned agency points of contact to each STRAC member state for technical questions. The STRAC officials stated that ONRR has been responsive when they have reached out with questions or concerns. Overall, STRAC officials from seven of the nine member states said that coordination with ONRR had improved over the past approximately 10 years. STRAC officials from two of the member states attributed this improvement to ONRR leadership’s concerted effort to work more effectively with STRAC. Additionally, STRAC officials from the majority of member states generally expressed satisfaction with the support ONRR has provided STRAC member states with respect to resources and training. STRAC officials from seven of nine member states told us that the current contracted budget was sufficient to conduct oversight of their states’ federal oil and gas royalties. STRAC officials from two member states stated that the budget was insufficient. One official stated that the budget did not allow the state to review all of the federal properties for which it was responsible. The officials from the other state indicated that a larger budget would allow the state to hire additional auditors. According to these officials, additional auditors could help the state conduct compliance activities on more royalty payors and in particular small royalty payors that may not be as familiar with the requirements for federal royalty payments. STRAC officials from several member states said that the flat budget that ONRR provided over the past several years may lead to changes in their federal royalty compliance activities. For example, one official stated that without additional funding in the future, the state may have to move more experienced and higher paid auditors to state royalty compliance activities, thus leaving less experienced and lower paid auditors to conduct federal royalty compliance activities. Another official stated that the state had offered less training and reduced the amount of funds for travel to address potential budget shortfalls. Additionally, another official stated that flat budgets could make it difficult to offer staff merit pay increases. Finally, officials from seven of the nine STRAC member states said ONRR provided sufficient training on policies, procedures, and IT systems used to conduct compliance activities on federal oil and gas royalties. A STRAC official from one member state said that ONRR’s training had improved recently, while another official said that support had improved. However, STRAC officials from three member states expressed uncertainty about ONRR’s training for companies. These officials stated that they would like to understand the content of the training so they would better understand how ONRR is training companies to report royalties. STRAC Members’ Processes for Selecting Compliance Cases Are Not Documented None of the nine STRAC member states had documented case selection processes. Specifically, officials from all nine STRAC member states we interviewed said that either they did not have, or were unable to provide, documented procedures for the processes they used to select federal oil and gas compliance cases. Rather, STRAC officials stated that they relied on a variety of factors to select cases for compliance reviews. Staff expertise about companies and properties was the factor that all nine STRAC officials identified as key for case selection. For example, one official stated that she had over 10 years of experience and therefore knew what companies or properties to review. Another official stated that because staff also work on state tax audits, they can use knowledge from that work to help identify compliance cases. Another factor officials identified as assisting in the case selection process was ONRR’s company and property risk scores, though they were given varying degrees of consideration. Other factors officials identified included referrals from BLM or ONRR, and risk scores generated from their own models. Under federal standards for internal control, management should establish an organization structure, assign responsibility, and delegate authority to achieve the entity’s objectives, including the documentation of the internal control system. Documentation provides a means to retain organizational knowledge and mitigate the risk of consolidating that knowledge to a few personnel, as well as a means to communicate that knowledge as needed to external parties, such as external auditors. Because STRAC members do not have a documented process, ONRR cannot, for example, assess whether STRAC members are selecting compliance cases in a manner that aligns with ONRR’s compliance goals or that future STRAC members will know how to select compliance cases. In the agreements between the nine STRAC members and ONRR, the agency includes terms and conditions that the members agree to, but ONRR does not require STRAC members to have documented procedures for compliance case selection. By including in ONRR’s future agreements with STRAC members requirements to develop a documented case selection process, including procedures for how to select compliance cases and how to document which factors were considered in selection decisions, ONRR could better assess whether members select cases that align with the agency’s compliance goals. STRAC Compliance Activities Described in Work Plans Do Not Align with ONRR Goals We reviewed STRAC members’ annual work plans to determine whether the compliance activities discussed aligned with ONRR compliance goals. STRAC member agreements from eight of the nine STRAC members included language that the “state will contribute to ONRR’s GPRA goals and thereby the performance goals of this Agreement by performing audits, compliance reviews and other investigations in coordination with ONRR. The yearly performance goals are listed on the state’s annual work plan.” However, when we reviewed the STRAC members’ corresponding annual work plans, we found no information on how the members’ compliance activities contributed to ONRR’s goals. For example, several of the STRAC members’ work plans included information on the leases and properties selected for compliance activities but did not include information on how those selections would contribute to ONRR’s compliance goals. In addition, the majority of STRAC officials from member states said they did not consider ONRR’s compliance goals for return on investment or total additional royalty collections when selecting compliance cases. When we asked STRAC members about their goals, three of nine STRAC member states noted that they had compliance program goals. For example, one STRAC member’s goal was to “maximize revenue to the state” and “implement on behalf of ONRR and the state, a constantly improving and efficient royalty audit program.” Another member’s goal was to “protect the US Citizens’ Federal Mineral Interest within the boundaries of the state by ensuring that a fair value, as established by the federal regulations, is received.” Officials from the other six STRAC members told us that their states do not have goals for federal oil and gas royalty compliance activities because ONRR does not require that they do so. For STRAC members that did not have compliance goals, officials provided examples of informal goals—or goals that were not documented. For example, one STRAC official reported that the state’s goal was to try to audit 50 percent of royalties paid to the state every 2 years. Another STRAC official stated the state tries to review major market areas in the state once every 7 years. When we compared STRAC officials’ responses on their goals to ONRR’s broader compliance goals for return on investment and total additional royalty collections, we found that the majority of states’ compliance goals did not align with ONRR’s goals. Federal standards for internal control state that management should define objectives clearly to enable the identification of risk and define risk tolerances, such as by defining objectives in alignment with the organization’s mission, strategic plan, and performance goals. In requiring eight of the nine STRAC members to conduct compliance activities consistent with the agency’s compliance goals, ONRR was following these standards. However, ONRR approved the STRAC members’ work plans, although those work plans did not specify how the described members’ compliance activities would contribute to ONRR’s goals as the agency stated they would in the agreements between the seven of the nine STRAC members and ONRR. By requiring STRAC members to describe in their annual work plans how their compliance activities would align with ONRR’s current compliance goals, ONRR would have better assurance that activities were aligned with its compliance goals. Finally, ONRR does not track STRAC member states’ contributions against its annual compliance goals. ONRR has the data available to track these contributions because the results of STRAC members’ compliance activities are retained in ONRR’s IT system. For example, we obtained reports on the aggregate overall return on investment of STRAC members and reviewed individual data entries from STRAC members’ work that included a data field for revenue collections. According to regulations, if a state accepts delegated authority, it is to assist ONRR in meeting the requirements of GPRA as well as in developing and endeavoring to comply with ONRR’s Strategic Plan and Performance Measurements. Because ONRR does not track STRAC member states’ contributions toward its annual compliance goals, the agency has limited information for assessing whether the funding they are providing to STRAC members is achieving its goals. ONRR officials we interviewed stated that they do not track states’ contributions to ONRR’s overall compliance goals as there is no requirement to do so. However, by tracking the performance of each state and its contribution toward ONRR’s compliance goals, ONRR could better assess the effectiveness of states’ performance in supporting the agency’s mission of ensuring accurate royalty payments. Conclusions ONRR is taking steps intended to improve its royalty compliance program and better verify that all royalties paid on the sale of oil and gas extracted from leased federal lands are accurate. These steps include reorganizing the management structure of its compliance program, implementing new systems for managing compliance cases electronically, and instituting a training curriculum for newly hired auditors. However, although ONRR reported generally meeting its compliance goals for fiscal years 2010 through 2017, its current goals may not align with the agency’s mission or other statutory requirements. For example, ONRR’s fiscal year 2017 compliance goals do not sufficiently address its mission to collect, account for, and verify revenues, in part, because its goals do not address accuracy, such as through a coverage goal. Establishing a coverage goal (e.g., identifying the number of companies or percentage of royalties subject to compliance activities over a set period) that aligns with the agency’s mission, and tracking the extent to which each of its compliance activities contributes to this goal, would provide ONRR more reasonable assurance that its compliance program is assessing the extent to which oil and gas royalty payments are accurate. Furthermore, ONRR’s audits, compliance reviews, and data mining efforts each provide a different level of assurance that royalties are accurately paid, but the agency does not measure how each of the compliance activities contributes to the FOGRMA requirement to establish a system with the capability to accurately determine and collect royalties in a timely manner. By tracking the extent to which each of its compliance activities contributed to any future coverage goal, ONRR would have greater assurance that its compliance program has the capability to accurately determine oil and gas royalties. In addition, ONRR’s compliance program relies on its Work Planning Group, which is responsible for reviewing information on companies and properties to select cases for audits or compliance reviews. The Work Planning Group, however, does not have a documented case selection process. By developing a documented case selection process that includes procedures for how to select compliance cases, ONRR could better ensure that it retains the organizational knowledge needed to effectively select compliance cases and defend the process in external reviews. In addition, ONRR does not have performance measures to determine the extent to which cases selected align with ONRR’s compliance goals. By developing performance measures (e.g., establishing a specified percentage of compliance cases that identify findings of royalty noncompliance or total additional royalties) that assess whether the agency is selecting cases that are helping it achieve its compliance goals, ONRR would be able to better monitor its performance in achieving its goals and whether changes to its selection process affect performance. Moreover, since 2006, ONRR has worked to develop a model to assess the risk of royalty noncompliance for use in its compliance case selection process. After several iterations with two contractors, ONRR began using the risk scores from its model to assist with case selection in 2014. However, according to ONRR officials, the agency is considering discontinuing the use of its current model in favor of one that is internally developed. ONRR has not analyzed how the use of the risk scores has affected case selection or findings of royalty noncompliance and is therefore unable to identify whether its risk model is effective. By periodically analyzing whether the risk model is effectively identifying potential royalty noncompliance and whether the model’s results are being effectively used to assist in case selection and making changes to the model (e.g., updating it) or developing a new model based on this analysis, ONRR would be better able to determine how to proceed with using risk analysis to inform its case selections. Furthermore, none of the nine STRAC members had documented case selection processes. In the agreements between the nine STRAC members and ONRR, the agency includes terms and conditions that the members agree to, but ONRR does not require STRAC members to have documented procedures for compliance case selection. By including requirements in ONRR’s agreements with STRAC members to develop a documented case selection process, including procedures for how to select compliance cases and how to document which factors were considered in selection decisions, ONRR could better assess whether members select cases that align with the agency’s compliance goals. Additionally, ONRR does not require that STRAC members specify how their compliance activities included in annual work plans contribute to ONRR’s compliance goals, although those goals appear on the work plans. ONRR approved the work plans but did not specify how the members’ compliance activities would contribute to its goals as the agency stated they would in the agreements between eight of the nine STRAC members and ONRR. By requiring STRAC members to describe in their annual work plans how their compliance activities would align with ONRR’s current compliance goals, the agency would have better assurance that activities were aligned with its performance goals. Lastly, ONRR does not track STRAC members’ contributions toward its annual compliance goals though it has the data to do so. By tracking the performance of each state and its contribution toward ONRR’s compliance goals, ONRR could better assess the effectiveness of states’ performance in supporting its mission of ensuring accurately royalty payments. Recommendations for Executive Action We are making a total of seven recommendations to ONRR. Specifically: The Director of ONRR should establish an accuracy goal (e.g., identifying the number of companies or percentage of royalties subject to compliance activities over a set period of time) that aligns with the agency’s mission of collecting, accounting for, and verifying royalty payments. In doing so, ONRR should track the extent to which each compliance activity (audits, compliance reviews, and data mining) contributes toward achieving this goal. (Recommendation 1) The Director of ONRR should develop a documented case selection process that includes procedures for how to select all compliance cases. (Recommendation 2) The Director of ONRR should develop performance measures (e.g., having a specified percentage of compliance cases identify findings of royalty noncompliance or total additional royalties) that assess whether the cases the agency is selecting are helping it achieve its compliance goals. (Recommendation 3) The Director of ONRR should periodically analyze whether the risk model is effectively identifying potential royalty noncompliance and whether the model’s results are being effectively used to assist in case selection, and should use this analysis to make changes to the model (e.g., updating it) or develop a new model. (Recommendation 4) The Director of ONRR should include requirements in ONRR’s agreements with STRAC members to develop a documented case selection process, including procedures for how to select compliance cases and how to document which factors were considered in selection decisions. (Recommendation 5) The Director of ONRR should require STRAC members to describe in their annual work plans how their compliance activities would align with ONRR’s current compliance goals. (Recommendation 6) The Director of ONRR should track the performance of the compliance work of each state STRAC member and the contribution that each state makes to ONRR’s compliance goals. (Recommendation 7) Agency Comments We provided a draft of this report to Interior for review and comment. Interior concurred with all seven recommendations. Agency comments are reproduced in appendix III. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Secretary of the Interior, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-3841 or ruscof@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff members who made major contributions to this report are listed in appendix IV. Appendix I: Status of Royalty Policy Committee’s Subcommittee on Royalty Management We reviewed and summarized recommendation closure documentation that the Department of the Interior (Interior) provided for royalty compliance recommendations made to the department by the Royalty Policy Committee’s Subcommittee on Royalty Management in 2007 and Interior’s Office of Inspector General (OIG) in 2006. For the subcommittee recommendations, Interior officials told us that the subcommittee did not assess the implementation of its recommendations. As a result, we present information that Interior provided on its decision about the status of the recommendations and a summary of actions taken. (See table 3.) For the OIG recommendations, we present the status of recommendations according to the OIG and a summary of the actions according to Interior. (See table 4.) We did not independently assess the implementation of the recommendations. Appendix II: ONRR’s Annual Compliance Goals and Performance See table 5 for detailed information on the Office of Natural Resources Revenue’s (ONRR) performance goals, including goal type, goal, fiscal year goal, fiscal year performance, and long-term target for performance goal. Appendix III: Comments from the Department of the Interior Appendix IV: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Christine Kehr (Assistant Director), Glenn C. Fischer (Analyst-in-Charge), Tim Bober, John Delicath, Sarah Detweiler, Wil Gerard, Cindy Gilbert, Michael Kendix, Eli Lewine, Ben Licht, Anne Stevens, and Sara Sullivan made important contributions to this report.