Background Federal agencies and our nation’s critical infrastructures rely on information technology systems that are highly complex and dynamic, technologically diverse, and often geographically dispersed. This complexity increases the difficulty in identifying, managing, and protecting the numerous operating systems, applications, and devices comprising their systems and networks. Further, federal systems and networks are at an increased risk of attack. This is due to those systems often being interconnected with other internal and external systems and networks, including the internet. Cloud computing relies on internet-based interconnectivity and resources to provide computing services to customers, while intending to free customers from the burden and costs of maintaining the underlying infrastructure. As federal agencies increasingly use cloud computing to perform their missions, the implementation of effective information security controls becomes more important. The effective implementation of a standardized process for securing cloud environments could reduce risks to agency systems and information maintained on an agency’s behalf. The Federal Information Security Modernization Act of 2014 (FISMA) was enacted to provide a comprehensive framework for ensuring the effectiveness of information security controls over information resources that support federal operations and assets. The act requires federal agencies to develop, document, and implement an information security program, and evaluate the program’s effectiveness. FISMA also requires OMB to develop and oversee the implementation of policies, principles, standards, and guidelines on information security in federal agencies, except with regard to national security systems. The law assigns OMB the responsibility of requiring agencies to identify and provide information security protections commensurate with assessments of risk to their information and information systems. In addition to implementing an agencywide security program, FISMA requires agencies to ensure the security of information and systems maintained by or on behalf of the agency. The law also applies to systems used or operated by a contractor or other organization on behalf of the agency, such as IT resources provided via cloud services. In December 2010, OMB issued a plan for improving IT management that included provisions for a decision framework to migrate IT services to cloud environments. Since then, OMB has developed cloud computing requirements, issued a number of cloud-related documents, and established FedRAMP. OMB cloud-related documents include: Federal Cloud Computing Strategy, which was intended to accelerate the government’s use of cloud computing by requiring agencies to evaluate safe, secure cloud computing options before making any new investments. Security Authorization of Information Systems in Cloud Computing Environments, which established FedRAMP in December 2011. 2019 Federal Cloud Computing Strategy, issued in June 2019, updates the 2011 Federal Cloud Computing Strategy and provides agencies with additional guidance on implementing cloud solutions and emphasizes cloud security as one of the three pillars of successful cloud adoption. In addition, the FedRAMP PMO established a framework for authorizing cloud services and guidance to help participants, including all agencies, implement it. According to the program management office, the framework is based on NIST guidance that agencies are supposed to follow. In addition to the framework, the program management office issued guidance on how agencies can leverage existing security authorization packages. Agencies Can Select from a Number of Cloud Service and Deployment Models Agencies can select different cloud services to support their missions. These services can range from a basic computing infrastructure on which agencies run their own software, to a full computing infrastructure that includes software applications. In defining cloud service models, NIST identifies three primary models, as follows: Infrastructure as a Service (IaaS). The cloud service provider delivers and manages the basic computing infrastructure of servers, software, storage, and network equipment. The agency provides the operating system, programming tools and services, and applications. Platform as a Service (PaaS). The cloud service provider delivers and manages the infrastructure, operating system, and programming tools and services, which the agency can use to create applications. Software as a Service (SaaS). The service provider delivers one or more applications and all the resources (operating system and programming tools) and underlying infrastructure, which the agency can use on demand. In addition, agencies can choose from a variety of arrangements for obtaining cloud services (called cloud deployment models), ranging from a private cloud for one organization to sharing a public cloud. NIST identified the following four cloud deployment models: Private cloud. The service is set up specifically for one organization, although there may be multiple customers within that organization and the cloud may exist on or off the customer’s premises. Community cloud. The service is set up for organizations with similar requirements. The cloud may be managed by the organizations or a third-party and may exist on or off the organization’s premises. Public cloud. The service is available to the general public and is owned and operated by the service provider. Hybrid cloud. The service is a composite of two or more of the three deployment models (private, community, or public) that are bound together by technology that enables data and application portability. These deployment models differ from each other in the number of consumers they serve, the nature of various consumers’ data that may be present in the cloud environment, and the amount of control consumers have over their data. A private cloud can allow for its consumers to have ultimate control in selecting who has access to that cloud environment. Community clouds and hybrid clouds allow for a mixed degree of consumers’ control and knowledge of other consumers. A public cloud allows access by all interested consumers, but, in doing so, should not allow one consumer who uses it to know or control data that belong to other consumers of that environment. FedRAMP Is a Government-wide Program for Authorizing Cloud Services Established by OMB and managed by GSA, the FedRAMP program is intended to provide a standardized approach to securing systems, assessing security controls, and continuously monitoring cloud services used by federal agencies. According to GSA, this approach is a “do once, use many times” framework that potentially lowers government costs, eliminates duplications, and ensures the consistent application of federal security requirements. The goals of FedRAMP are to: ensure that cloud-based services used by government agencies have adequate safeguards in place; eliminate the duplication of effort to assess security controls, and reduce risk management costs; and enable rapid and cost-effective procurement of information systems/service for federal agencies. The program’s key participants are the FedRAMP PMO, JAB, federal agencies, cloud service providers, and third-party assessor organizations. FedRAMP PMO. FedRAMP’s PMO is headed by GSA and serves as the facilitator of the program. The office’s responsibilities include managing the program’s day-to-day operations, creating guidance and templates for agencies and cloud service providers to use for developing, assessing, authorizing, and continuously monitoring cloud services per federal requirements (e.g., FISMA). JAB. The JAB is made up of chief information officers from the Department of Defense (DOD), DHS, and GSA. It is the primary governing and decision-making body of the program. The JAB is responsible for defining and establishing FedRAMP baseline security controls and accreditation criteria for third-party assessment organizations. The JAB is also responsible for issuing a provisional authorization to operate (P-ATO) for cloud services it determines will be leveraged across most of the federal government. Federal agencies. They are consumers and, in some cases, providers of cloud services. Agencies are responsible for ensuring that cloud services which process, transmit, or store government information, use FedRAMP’s baseline security controls before they issue subsequent authorizations for using those cloud services. Cloud service providers (CSP). These providers include commercial firms and some federal agencies that offer cloud services to agencies. Providers are required to meet the FedRAMP security requirements and implement the program’s baseline security controls. Providers work with an independent third-party assessment organization to conduct an initial system assessment, create security assessment documentation per the program’s requirements, and comply with federal requirements for incident reporting, among others. Third-party assessment organizations. These FedRAMP accredited assessors perform initial and periodic assessments of cloud providers’ controls to ensure they meet the program’s requirements. In addition, these assessors must be accredited through FedRAMP if they are assessing a cloud provider seeking a provisional authorization from the JAB. For details on the roles and responsibilities of other entities involved with the program, see table 6 in appendix II. The FedRAMP Security Assessment Framework Outlines Key Artifacts for Authorizing Cloud Services In December 2015, the FedRAMP PMO developed a security assessment framework that is to be followed by the cloud service providers (providers) and agencies seeking to authorize cloud services through the program. In addition to outlining roles and responsibilities, the framework provides agencies and cloud service providers with guidance on elements key to issuing authorizations for using cloud services through the program. These elements are critical to developing the information system or cloud service authorization package. Authorization packages include, but are not limited to the following artifacts: a control implementation summary, the security plan, the security test plan and assessment report, and remedial actions plan. These artifacts are described in table 1. FedRAMP provides agencies with two options for authorizing cloud services. The first option, called a JAB authorization, involves the agency authorizing the cloud service based on a provisional authorization issued by the board. The second option, called an agency authorization, involves the agency issuing an authorization after either sponsoring a cloud service provider through FedRAMP, or by leveraging another agency’s FedRAMP authorization of that cloud service provider. Using either of these options, the agency is to review the authorization package for that cloud service prior to issuing its authorization. In reviewing the package, the agency is to consider the cloud service’s system impact level (low impact, moderate impact, or high impact), and deployment model, among other things, to help determine which authorization option is more appropriate. After an agency has reviewed the package and made a risk-based decision to authorize a cloud service for use, it is to formally document this decision in an authorization letter. The agency official authorizing the cloud service must provide a copy of the letter to the FedRAMP PMO. The PMO uses the information to verify agency use and keep other agencies informed of any changes to a provider’s authorization. Agencies Increased Their Use of FedRAMP, but Many Continued to Use Cloud Services Not Authorized through FedRAMP As of July 2019, all 24 CFO Act agencies participated in FedRAMP. According to the program management office’s documentation, from June 2017 through July 2019, these agencies’ use of FedRAMP authorizations increased from 390 authorizations to 926 authorizations. Specifically, the number of JAB authorizations increased from 155 to 317—a 105 percent increase. Further, the total number of agency sponsored and –leveraged authorizations increased, from 235 to 609—a 159 percent increase. Figure 1 illustrates the increase in the number of FedRAMP authorizations for the 24 agencies from June 2017 through July 2019. Agencies Reported a Higher Number of Authorizations for Software as a Service than for Other Cloud Services Survey responses from 23 of 24 CFO Act agencies indicated that the highest number of cloud service authorizations through FedRAMP were for Software as a Service. Software as a Service accounted for 331 of the 590 reported authorizations or 56 percent. For the other two services, Infrastructure as a Service and Platform as a Service, agencies reported issuing 153 authorizations (26 percent) and 106 authorizations (18 percent), respectively. Figure 2, depicts the authorizations by agency and cloud service and shows that 18 of 23 agencies issued more authorizations for Software as a Service than Platform as a Service or Infrastructure as a Service. In addition, while agencies are consumers of cloud services, some agencies also serve as cloud service providers to other federal agencies. Four of 24 agencies reported that they served as cloud service providers to other federal agencies in FY 2017. All four agencies reported that their cloud services received authorizations that were approved through FedRAMP and used by other federal agencies. These four agencies reported a total of seven cloud services with an agency authorization and one cloud service with a provisional authorization from the JAB. Agencies Reported Using Cloud Services That Were Not Authorized through FedRAMP OMB required all agencies to use FedRAMP for authorizing cloud services by June 2014, and by June 2017, all of the 24 CFO Act agencies were using the program. However, the agencies also used cloud services that were not authorized through the program. In responding to our survey, the majority of the agencies (15 of 24) reported that they used cloud services that were not authorized through FedRAMP. For instance, one agency reported that it used 90 cloud services that were not authorized through FedRAMP and the other 14 agencies reported using a total of 157 cloud services that were not authorized through FedRAMP. Seven agencies responded that they only use cloud services authorized through FedRAMP. Two agencies did not provide a response for this question. Agencies provided varying explanations for using cloud services that were not authorized through FedRAMP. For example, officials from two of the agencies stated that they were unable to identify providers authorized through the program that could meet their unique needs. An official from a third agency noted that the efforts to meet the program’s requirements were labor-intensive and that it was too expensive for the providers to become compliant with FedRAMP. In addition, that official stated that providers did not want to pursue FedRAMP compliance unless they had enough demand from federal customers. An official from a fourth agency stated that some of that agency’s cloud services were considered to be private and, thus, did not need to be authorized through the program. Nevertheless, according to that official, the agency performed its own authorization actions to ensure that FedRAMP requirements were met. In a similar example, an official at another agency noted that it took a significant amount of time for a provider to complete the FedRAMP process and that the agency had to issue its own authorization while the provider was going through the process. That authorization had not yet been approved through FedRAMP. The survey responses of cloud service providers were consistent with the agencies’ responses and indicated that multiple agencies were using cloud services that were not authorized or approved through FedRAMP. For example, 31 of 47 providers that responded to our survey reported that, during FY 2017, agencies had used their cloud services and those services were not authorized by FedRAMP. According to one cloud service provider, agencies were using 30 of its cloud services that were not authorized through FedRAMP. Another cloud service provider reported that agencies were using nine of its cloud services that were not authorized through the program. Officials from the FedRAMP program management office also provided several reasons why agencies did not use the program for all of their cloud services. For example, one PMO official indicated agencies had misperceptions of the program, its process, and resources required for a FedRAMP authorization. The official also specified that agencies did not use the program for all their cloud services because of internal resource constraints based on other competing agency priorities. Based on our work, another potential reason that agencies authorize cloud services outside of the FedRAMP program is that OMB has not adequately monitored compliance with this requirement. As mentioned earlier, OMB has issued a number of policies encouraging agencies to adopt cloud computing solutions and requiring agencies to use FedRAMP for authorizing cloud services. Nevertheless, OMB has not monitored agencies’ compliance or held agencies accountable for complying with the requirement to ensure that agencies are using the program to authorize their cloud services. According to an OMB technical specialist, the office collects and reviews data from the FedRAMP Marketplace to monitor agencies’ use of the program. However, the office does not collect data on the extent to which federal agencies are using cloud services authorized outside of the program or oversee agencies’ compliance with using FedRAMP. As a result, if OMB does not monitor or hold agencies accountable for using the FedRAMP program, OMB and federal agencies have reduced assurance that security controls required by the program are being consistently implemented. Additionally, OMB may lack information on agencies’ needs for cloud services. Selected Agencies Did Not Consistently Address Key Elements of FedRAMP’s Authorization Process Although the four selected agencies included key documents supporting FedRAMP’s authorization process, they did not consistently include key information in those documents. Specifically, these four agencies did not consistently or fully address required information in system security plans, security assessment reports, and remedial action plans. In addition, the agencies did not always prepare their authorizations approving the use of cloud services. Agencies’ Authorization Packages Included Control Implementation Summaries FedRAMP recommends that agencies use the FedRAMP Control Implementation Summary (CIS) when leveraging cloud services for their systems. In addition, FedRAMP specifies that agencies are to use NIST guidance when addressing their individual or shared control implementation responsibilities when leveraging cloud services. All 10 authorization packages we reviewed contained a summary, which identified agencies’ control implementation responsibilities as well as that of the cloud service providers. Selected Agencies Did Not Consistently Document Required Information in System Security Plans An objective of system security planning is to improve the protection of information system resources. A system security plan provides an overview of the security requirements for a system or cloud service and describes the controls that are in place or planned to meet those requirements. To identify controls that an agency will need to document on its security plan, the agency reviews the CIS which lists both the agency and CSP’s security control responsibilities. Further, NIST guidelines state that federal agencies’ system security plans should identify: an explicitly defined authorization boundary for the system, how the system operates in terms of mission and business processes, the security categorization of the system including supporting rationale, the operational environment of the system and connections to other information systems, the security controls in place or planned for meeting security requirements, including a rationale for supplementing controls, and a review and approval by the authorizing official or designated representative prior to plan implementation. As shown in table 2, the four selected agencies had documented security plans for 10 systems. However, the agencies had not consistently addressed the required information in their plans. As illustrated above, the security plans for the nine selected systems did not fully address all required information. For example, three plans partially identified the operational environment of the system, such as identifying external connections which could include the cloud service the agency system was leveraging. In addition, nine plans did not fully address the extent to which security controls were in place, including those listed as the agency’s responsibility. Further, agencies did not provide complete support that their authorizing officials had reviewed and approved the plans for five systems. Specifically, agencies provided signed letters indicating that the agencies initially approved the plans. However, agencies did not provide documentation to show that subsequent changes to the system security plan after the date of the signed letters were reviewed and approved by the authorizing official. Additionally, one agency had an expired letter. Until agencies fully address required information in their security plans, including the controls relied on by the cloud service provider, they have reduced assurance that security controls are in place and operating as intended. Selected Agencies’ Security Assessment Reports Did Not Consistently Summarize Control Effectiveness NIST specifies that organizations document the results of security assessments in a security assessment report. According to FedRAMP’s guidance, agencies are to use the Control Implementation Summary to identify controls that are their responsibility and assess agency-specific controls, inclusive of any agency controls that are shared with providers. The security assessment report is to summarize the control testing and describe whether the tested controls were effectively in place. As shown in table 3, agencies did not always summarize the testing of controls on security assessment reports. The four agencies prepared security assessment reports for each of the 10 selected systems. However, agencies summarized the results of control tests for only three of the 10 systems reviewed. USAID summarized the test results in the security assessment report for the agency system we reviewed, but the other three agencies did not consistently summarize their results. For example, HHS did not summarize test results for three controls for one system and six controls for another system. GSA did not summarize tests results for 17 controls for one of its systems. If security assessment reports do not fully summarize the test results, agencies may have limited assurance that the controls intended to protect agency data in the cloud environment are in place and operating effectively. Selected Agencies’ Remedial Action Plans Did Not Include Required Information A remedial action plan assists agencies in identifying, assessing, prioritizing, and monitoring progress in correcting security weaknesses that are found in information systems. NIST guidelines specify that organizations develop a remedial action plan, also referred to as a plan of action and milestones, to document the organization’s planned actions to correct weaknesses or deficiencies noted during the assessment of security controls of the information system. In addition, FedRAMP guidance stated that all agencies should follow FISMA which requires agencies to have a process for documenting remedial actions to address any deficiencies in the information security policies, procedures, and practices. OMB requires that remedial action plans include the following information: a description of the specific weakness; the name of the office or organization responsible for resolving the weakness; an estimate of the funding required to resolve the weakness, including the anticipated source of funding; an estimated completion date for resolving the weakness; key milestones with estimated completion dates; any changes to the key milestones and completion dates; the source of the identified weakness (e.g. security assessment, program review, inspector general audit, etc.); and the status of the corrective action (ongoing, completed, etc.). As shown in table 4, the four selected agencies documented remedial action plans for each of the selected systems, but did not consistently identify required information. As illustrated above, three plans partially identified the office responsible for addressing the weakness. Two plans did not include changes to information regarding key milestones and completion dates and two partially included the information. Further, two agencies partially identified the source of the weakness for three systems while a third agency did not identify any sources for the selected system. Until agencies include all required elements in their remedial action plans, they will be less likely to effectively assess, prioritize, and monitor efforts to resolve weaknesses in their systems. Selected Agencies Did Not Consistently Prepare and Provide Authorization Letters to the FedRAMP PMO OMB defines an authorization to operate as an official management decision where a federal official or officials authorize the operation of information system(s) and accept the risk to agency operations and assets, individuals, and other organizations based on the implementation of security and privacy controls. OMB requires agencies to use FedRAMP processes when granting authorizations to operate for their use of cloud services. According to FedRAMP PMO guidance, authorizing officials should document the authorization of (1) the agency system supported by the cloud service and (2) the cloud service used by the agency. Additionally, the agency should provide a copy of its authorization letter for the cloud service (cloud service authorization letter) to the FedRAMP program management office so that the office can verify the agency’s use of the service and keep agencies informed of any changes to a provider’s authorization status. As shown in Table 5, agencies did not consistently prepare and provide the FedRAMP PMO with the cloud service authorization letter. GSA prepared both system and cloud service authorization letters for its two selected systems. However, the other three agencies did not consistently prepare the letters. Specifically, USAID did not consistently prepare letters authorizing the cloud service and the system supported by the cloud service. In addition, HHS and EPA did not consistently prepare letters authorizing their use of the cloud services. Further, EPA, HHS, and USAID did not consistently provide the FedRAMP PMO with authorization letters for cloud services. Although GSA and an HHS component, CDC, provided cloud service authorization letters to the FedRAMP PMO, only HHS included the requirement to provide the letter to the FedRAMP PMO in its guidance. Three of the four selected agencies did not include this requirement in their guidance. Not including this requirement in their security guidance could be a potential reason for agencies’ inconsistent implementation. If agencies do not provide copies of their cloud service authorization letters to the program management office, the office may not have accurate information on which agencies are using approved cloud services. Further, the lack of such information could result in the office being delayed in notifying agencies when a service provider’s authorization has been revoked or a provider has experienced a security incident. Agencies provided various reasons for not including required information in FedRAMP authorization documents. Such reasons included the agency was restricted from documenting proprietary information concerning the cloud service provider’s portion of the shared control in the security plan and the agency was tracking all remedial actions, but the agency did not include them in the plan it provided to us. By not including the required information, agencies have reduced assurance that controls over cloud services have been effectively implemented. Program Participants Reported Improved Security and other Benefits, but also Identified Challenges FedRAMP participants identified a number of the program’s benefits, such as improved security of agencies’ data and increased efficiency for providers to obtain authorizations. Participants also cited a number of challenges, such as the agency resources needed for authorizing a cloud service or the resources needed by the provider to implement the program’s requirements. To address challenges, GSA has taken steps to improve the program, but its guidance on FedRAMP’s requirements and participant’s responsibilities was not always clear and the program’s process for monitoring the status of security controls over cloud services was limited. Participants Identified Various Challenges with Implementing FedRAMP FedRAMP participants indicated that implementing certain elements of the program were challenging. Participants specifically identified the authorization process, remedial actions, and time and resources as key challenges. Authorization process and requirements. Complex authorization process. Surveyed participants—agencies and cloud service providers—responded that simplifying the agency authorization process would help them to better understand and manage their ongoing authorizations and continuous monitoring efforts. For example, 17 of 23 agencies, responding to this question, identified the agency authorization process as an area for improvement as did 30 of 47 surveyed cloud service providers. Survey respondents indicated that the agency authorization process should be streamlined to be less-restrictive and time-consuming. Agencies also reported that overcoming the complexity of the authorization process was one of their largest hurdles. According to the Director of FedRAMP, the FedRAMP PMO encourages agencies to streamline their agency authorization processes to be less- restrictive and time-consuming. Limitations with reviewing authorization packages. Agencies also identified reviewing authorization packages as a challenge. Agencies reported in the survey and during interviews that there were limitations in their ability to review cloud security packages prior to selecting a cloud service provider. Agencies that are currently using or want to evaluate specific FedRAMP authorized cloud services are able to access FedRAMP security packages directly through the FedRAMP Secure Repository, located on OMB MAX portal. However, agencies are given a 30-day period to access packages, which one agency official stated is too short of a time period for them to properly review documentation. Although access is limited to 30 days, agencies are able to renew the access by sending an email to the FedRAMP program management office. The Director of FedRAMP indicated that agencies can work directly with cloud service providers to obtain additional permissions to the package to save, print, email, post, publish, or reproduce. In addition, agencies expressed challenges with restrictions on downloading the packages, which limited their ability to automate their review of packages and subsequent monitoring of changes to the services security posture. Agencies also cited challenges with sharing review-related information due to the restrictive nature of cloud service nondisclosure agreements. The Director of FedRAMP mentioned that agencies can work directly with cloud service providers to obtain additional access permissions to their packages. Lack of uniform guidance for selecting cloud services. Federal agencies suggested that uniform guidance on authorization packages could assist FedRAMP customers in making better risk-based decisions in selecting cloud services. Agency officials we interviewed stated the quality and reviews of authorization packages approved through FedRAMP varied. Officials stated that inconsistencies in both FedRAMP agency and JAB provisional authorization packages have required some agencies to perform additional work. According to the officials, while the JAB process takes longer, the review appears to be more detailed than the agency process. Officials noted that improving guidance on reviewing authorization packages could help with the consistency and quality of the agency package reviews. The FedRAMP PMO has taken action and published guidance during our engagement to address more details of the authorization process. In addition, according to the Director of FedRAMP, the FedRAMP PMO launched a series of training events between February 2018 and June 2019 that provided detailed guidance into the package review process. Need for improved collaboration and coordination. Participants also identified opportunities for improving collaboration and coordination. Federal agencies suggested that improved collaboration among federal agencies in leveraging cloud services could provide transparency on the cloud service providers and the services other agencies are using. This could inform agencies on whether those services could be adopted to fit the need of their missions. Agencies also mentioned that FedRAMP PMO could improve its coordination across federal agencies and cloud service providers to provide consistent information and help facilitate opportunities to improve the program. For example, three participants suggested improving cross-agency collaborations for cloud authorizations. Additionally, one survey participant noted that improved collaboration within the cloud service provider community could provide a better understanding of the impacts and associated cost of potential changes to program’s policies or requirements before they are made. According to officials from the FedRAMP PMO, their standard practice is to solicit feedback from industry and agency stakeholders prior to release of significant guidance. They added that they plan to continue collaborating with agency and industry partners. Remedial action process. In responding to our survey, 9 of 23 agencies reported that the lack of clarity on actions taken to resolve weaknesses in systems supporting cloud services was a major or moderate challenge. Specifically, two agencies cited this area as a major challenge and seven as a moderate challenge. Two agencies suggested that the program management office could make improvements by providing better visibility and traceability of the remedial action process to inform agencies on the risks associated with a cloud service. Participants responded that the remedial action process could be improved by having structured procedures for aggregating system vulnerabilities and deficiencies. This would provide agencies with better information on weaknesses identified by cloud service providers or their third party assessors in order to better consider risks prior to the purchase or use of cloud services. Additionally, agencies cited the need for improvements to the consistency of remedial action plans. Specifically, agencies cited the need for a consistent format and content of remedial action plans among security packages. Further, one cloud service provider stated that outcome-based performance metrics were a better measure of monitoring the status and effectiveness of the ongoing authorization and assessment of cloud services, as opposed to only relying on remedial action plans. According to the Director for FedRAMP, the FedRAMP PMO developed additional remedial action guidance in February 2018 and a dedicated webpage specific to the remedial action process in January of 2018. Additionally, the Director noted that for all JAB provisional authorizations, the FedRAMP PMO and JAB analyzes raw data on vulnerability scans and provides a one-page summary report that is available to agencies within the OMB MAX portal. Commitment of time and resources to complete and maintain an agency authorization. The amount of time to complete an agency authorization to operate for a cloud service was cited as one of the most challenging aspects of FedRAMP. In responding to our survey, six agencies cited the commitment of time and resources for agency authorizations as a major challenge; five agencies identified it as a moderate challenge; and six as a minor challenge. One responding agency mentioned that the time and costs associated with completing and maintaining an ongoing agency authorization was burdensome to both the agency and cloud vendor. This burden was due to a lack of allocated agency resources to continue implementing the program’s requirements. In response to this challenge, the program management office has streamlined the authorization process for low-risk systems to allow for risk-based decisions that can reduce the time and resources required for an agency authorization. In addition, 36 of 47 cloud service providers responding to our survey indicated that the significant amount of resources required to implement the program’s requirements for an authorization was a major or moderate challenge. Additionally, JAB technical representatives identified many of the challenges and opportunities for improving the program that agencies and cloud service providers identified. In addition, the officials stated that the FedRAMP PMO is aware of these issues and has taken steps to address them. According to the JAB technical representatives, the FedRAMP PMO’s program intended improvements include, but are not limited to, updates to guidance and education resources, plans to automate the continuous monitoring process with vulnerability scanning tools, and reduced time and costs associated for completing the authorization process for both customer agencies and cloud service providers. According to the Director for FedRAMP, the FedRAMP PMO has continued to make enhancements based on industry and agency feedback. The official reported that numerous guidance documents, relating to continuous monitoring, the agency authorization process, and FedRAMP designations have been released during our engagement. The official also mentioned that the PMO actively seeks feedback from stakeholders and that additional opportunities for FedRAMP training was available. GSA Took Steps to Improve FedRAMP, but Program Guidance Was Not Always Clear and the Process for Monitoring Security Controls Was Limited GSA has taken a number of steps to improve FedRAMP. Among other things, the office has provided updated instructions for completing authorization packages and established and updated its training portal to help agencies and cloud service providers better understand the steps required for obtaining an authorization. In addition, the office has taken steps to streamline the authorization process and provided additional guidance on continuous monitoring of security controls over cloud services. Nevertheless, FedRAMP’s requirements and guidance on implementing controls were not always clear and the program’s process for monitoring the status of security controls over cloud services was limited. Clarity in program requirements and responsibilities. Agencies reported challenges with understanding FedRAMP’s requirements and the process for granting an agency authorization. Specifically, agencies cited the need for clearer guidance on requirements and agency responsibilities for completing and maintaining an authorization. Eight agencies reported the clarity of FedRAMP requirements associated with the agency authorization process as a moderate challenge; whereas nine identified it as a minor challenge and no agencies reported it as a major challenge. Five agencies reported this was not a challenge. In addition, 20 of 24 surveyed agencies indicated that additional guidance describing roles and responsibilities would be very or moderately useful to their participation in FedRAMP. Further, 37 of 47 cloud service providers specified that additional guidance for describing the security roles and responsibilities between agencies and cloud service providers was needed. Both agencies and cloud service providers commented that existing guidance for using the program does not fully address control implementation roles and responsibilities and that a process should be established to address these issues. Officials from selected agencies also indicated that responsibilities were not always clearly detailed. Specifically, HHS, GSA, and USAID officials stated that guidance for using FedRAMP could be clearer on helping define roles and responsibilities between agencies and providers in implementing security controls for cloud services. The JAB technical representatives we interviewed acknowledged that while control implementation responsibilities between the agency and cloud service provider are defined in the Control Implementation Summary, in some cases, shared responsibilities are not clearly delineated. The JAB technical representatives stated that the unclear shared responsibilities could lead to inconsistent implementation of certain controls between the agency and its provider. According to the Director of FedRAMP, it is the cloud service providers’ responsibility to ensure the spreadsheet identifying control responsibilities are completed accurately and consistently. Our analysis of agency documentation of required information in authorization packages found that the cause of selected agencies’ gaps in required information for security plans, security assessment reports and remedial action plans were due in part, to unclear guidance for implementing their control responsibilities. If responsibilities are not clear, agencies may have reduced ability to ensure that controls over the cloud services they authorized are in place and effective. Limited capabilities for continuously monitoring security controls. FedRAMP’s continuous monitoring process does not allow for an automated review of control requirements by agencies with security management tools. According to NIST SP 800-137, security continuous monitoring is maintaining an ongoing awareness of information security, vulnerabilities, and threats to support organizational risk management decisions. In addition, NIST mentions that timely, relevant, and accurate information is vital, particularly when resources are limited and agencies must prioritize their efforts. According to the program’s officials, they will be working with NIST to incorporate automation into the authorization process. Based on our work and survey responses from agencies and cloud service providers, a number of weaknesses with the program’s continuous monitoring process existed. For example, copy-protected PDFs, Word documents, and Excel spreadsheets comprised the remedial action plans and other documents supporting continuous monitoring of FedRAMP cloud service provider controls. Because of the static nature of the documents, including restrictions on copying information concerning cloud service provider controls, the documents could not be readily integrated with agencies’ automated security management tools in providing ongoing awareness of control implementation. Further, agency staff would have to spend time manually accessing and reviewing the documents each time they needed to determine the status of a cloud service’s implementation of a particular control. Agency personnel would also have to confirm that the documents they reviewed were the most current version. According to the Director of FedRAMP, agencies may request unrestricted access to the security package directly from the provider. Agencies’ survey responses also indicated that: 1) remedial action plans, used in continuous monitoring, were not updated consistently, 2) the manual process did not allow for automated data feeds into their continuous monitoring tools, and 3) restrictions on copying documents reduced information sharing within the agency. Further, 21 of 23 agencies responded that FedRAMP’s continuous monitoring of cloud security controls was a needed area of improvement. Cloud service providers also reported difficulties (36 of 47) with implementing continuous monitoring which could highlight the need for further improvements. In response, the Director of FedRAMP indicated that as of October 30, 2018, the FedRAMP PMO consolidated all continuous monitoring guidance documents, templates, and blog posts to a single webpage for ease of access by program stakeholders. JAB technical representatives also acknowledged challenges with implementing continuous monitoring such as difficulties with using continuous monitoring reports to assess the security posture of a cloud service. According to JAB technical representatives, agencies are responsible for reviewing continuous monitoring reports from the cloud service providers, but not all agencies could effectively conduct continuous monitoring. For example, an agency’s continuous monitoring efforts could be affected from not receiving a timely notification that its cloud service provider has uploaded the required monthly continuous monitoring updates, including updates to remedial actions. According to the Director of FedRAMP, the OMB MAX portal provides the capability for agencies to receive automatic notifications when there is an update to the continuous monitoring. Agencies can enable updates by selecting the “Watch this Page” option in the menu bar. While the FedRAMP PMO recommends agencies to enable this feature, agencies were not aware of the feature. As a result, agencies may not be aware that such updates have taken place and tend to be reliant on a providers’ ability to ensure that effective security practices are in place. The JAB technical representatives commented that as cloud services evolve and mature, the continuous monitoring process needs to become more automated and user-friendly to provide real-time awareness of the security status of cloud services. Until the PMO allows for more options to automate continuous monitoring, agencies may have less assurance that they will receive timely information on the extent that controls are being effectively implemented for the cloud services they are using. In addition, as more federal agencies move toward DHS’s Continuous Diagnostics and Mitigation program, automation may become even more important. Conclusions Although federal agencies increased their use of FedRAMP, they continued to authorize the use of cloud services that had not been approved through the program. While OMB requires agencies to use FedRAMP to authorize the use of cloud services, it did not monitor or ensure that agencies used the program to authorize cloud services. As a result, agencies have less assurance that security controls over cloud services have been consistently implemented. The selected agencies did not fully address key elements necessary for implementing the FedRAMP authorization process. Agencies did not consistently address required information for implementing controls, summarizing control tests, and tracking corrective actions. In addition, agencies also did not always provide the FedRAMP PMO with their cloud service authorization letters. By not fully addressing these elements, agencies have less assurance that they have effectively implemented security controls intended to protect their data in cloud environments and that those controls operating as intended. FedRAMP participants identified a number of benefits as well as challenges with the program. Among other benefits, several agencies indicated that FedRAMP improved of the security of their data. However, participants identified challenges with the program and areas where the program could be improved. GSA has taken a number of actions toward improving and furthering the program’s progress, nonetheless unclear guidance and limitations with FedRAMP’s continuous monitoring process could hamper the program’s effectiveness and result in agencies implementing the program unevenly. Recommendations for Executive Action We are making a total of 25 recommendations—1 recommendation to OMB and 24 recommendations to the 4 selected agencies in our review, including additional recommendations to GSA as the FedRAMP program lead. The Director of OMB should establish a process for monitoring and holding agencies accountable for authorizing cloud services through FedRAMP. (Recommendation 1) The Administrator of GSA should direct the Director of FedRAMP to clarify guidance to agencies and cloud service providers on program requirements and responsibilities. (Recommendation 2) The Administrator of GSA should direct the Director of FedRAMP to improve the program’s continuous monitoring process by allowing more automated capabilities, including for agencies to review documentation. (Recommendation 3) The Administrator of GSA should update security plans for selected systems to include the description of security controls and reviews and approvals plan. (Recommendation 4) The Administrator of GSA should update the security assessment report for the selected system to identify the summarized results of control effectiveness tests. (Recommendation 5) The Administrator of GSA should update the list of corrective actions for selected systems to identify the responsible office and estimated funding required and anticipated source of funding. (Recommendation 6) The Administrator of GSA should develop guidance requiring that cloud service authorization letters be provided to the FedRAMP program management office. (Recommendation 7) The Secretary of HHS should direct the Director of CDC to update the security plan for the selected system to identify the authorization boundary, the system operational environment and connections, a description of security controls, and the individual reviewing and approving the plan and date of approval. (Recommendation 8) The Secretary of HHS should direct the Director of CDC to update the security assessment report for the selected system to identify the summarized results of control effectiveness tests. (Recommendation 9) The Secretary of HHS should direct the Director of CDC to update the list of corrective actions for the selected system to identify the specific weaknesses, funding source, changes to milestones and completion dates, identified source of weaknesses, and status of corrective actions. (Recommendation 10) The Secretary of HHS should direct the Administrator of CMS to update the system security plans for selected systems to identify a description of security controls. (Recommendation 11) The Secretary of HHS should direct the Administrator of CMS to update the security assessment report for selected system to identify the summarized results of control effectiveness tests. (Recommendation 12) The Secretary of HHS should direct the Administrator of CMS to update and document the CMS remedial action plan for the selected system to identify the anticipated source of funding. (Recommendation 13) The Secretary of HHS should direct the Administrator of CMS to prepare letters authorizing the use of cloud services for the selected systems and submit the letters to the FedRAMP program management office. (Recommendation 14) The Secretary of HHS should direct the Director of NIH to update security plans for selected systems to identify the authorization boundary, system operation in terms of mission and business processes, operational environment and connections, and a description of security controls. (Recommendation 15) The Secretary of HHS should direct the Director of NIH to update the security assessment report for selected systems to identify summarized results of control effectiveness tests. (Recommendation 16) The Secretary of HHS should direct the Director of NIH to update the NIH list of corrective actions for selected systems to identify estimated funding and anticipated source of funding, key milestones with completion dates, and changes to milestones and completion dates. (Recommendation 17) The Secretary of HHS should direct the Director of NIH to submit the division’s letters authorizing the use of cloud services for the selected systems to the FedRAMP program management office. (Recommendation 18) The Administrator of EPA should update security plan for the selected operational system to identify a description of security controls, and the individual reviewing and approving the plan and date of approval. (Recommendation 19) The Administrator of EPA should update the security assessment report for the selected operational system to identify the summarized results of control effectiveness tests. (Recommendation 20) The Administrator of EPA should update the list of corrective actions for the selected operational system to identify the specific weakness, estimated funding and anticipated source of funding, key remediation milestones with completion dates, changes to milestones and completion dates, and source of the weaknesses. (Recommendation 21) The Administrator of EPA should prepare the letter authorizing the use of cloud service for the selected operational system and submit the letter to the FedRAMP program management office. (Recommendation 22) The Administrator of EPA should develop guidance requiring that cloud service authorization letter be provided to the FedRAMP program management office. (Recommendation 23) The Administrator of USAID should update the list of corrective actions for the selected system to include the party responsible for addressing the weakness, and source of the weakness. (Recommendation 24) The Administrator of USAID should prepare the letter authorizing the use of cloud service for the selected system and submit the letter to the FedRAMP program management office. (Recommendation 25) Agency Comments and Our Evaluation We provided a draft of this report to OMB and the 24 CFO Act agencies for review and comment. In response, we received comments from OMB and the four agencies (GSA, HHS, EPA, and USAID) to which we made recommendations. Specifically, in comments provided via email on October 15, 2019, an OMB Associate General Counsel stated that OMB neither agreed nor disagreed with our draft recommendation that it establish a process for monitoring and enforcing agency compliance with its guidance on using FedRAMP. The official asserted that OMB does not have a mechanism for enforcing agencies’ compliance with its guidance on FedRAMP. However, we believe OMB can and should hold agencies accountable for complying with its policies. Policies without accountability mechanisms present the risk that the benefits expected from their implementation will likely not be realized. To ensure our position is clearly stated, we modified the recommendation to state that OMB should establish a process for monitoring and holding agencies accountable for authorizing cloud services through FedRAMP. In addition, the OMB Associate General Counsel stated that the report did not appropriately reflect FedRAMP’s progress. We disagree. Although identifying the program’s progress was not one of our objectives, we highlighted several areas throughout the report where progress was achieved such as the agencies’ increasing use of the program to authorize cloud services and the development of additional guidance and training opportunities for using the program. The OMB Associate General Counsel also commented on the duration of the audit. Additionally, OMB commented that our use of surveys on agencies and cloud service providers’ use of FedRAMP did not address whether the program was meeting its overall objectives, but presented more of a perception. As discussed in the scope and methodology for this review, and consistent with our objectives, the purpose of the surveys was to obtain program participants’ views on the benefits, challenges, and their use of the program. Additionally, our review, as designed, including our timelines, allowed us the opportunity to best assess the implementation of the program. OMB also provided technical comments, which we have incorporated into our report as appropriate. In its written comments, GSA concurred with each of our six recommendations. The agency stated that it is developing a plan to address the recommendations. GSA’s comments are reprinted in appendix IV. In written comments, HHS concurred with each of our 11 recommendations. One operating division, CDC, noted that our observations were narrowly focused on authorization artifacts and did not take their FISMA compliant authorization process into account. We disagree. Our reviews of their FedRAMP authorization processes included procedures for reviewing security practices that are required under FISMA. The department stated that it would work with its operating divisions to address our recommendations. HHS’s comments are reprinted in appendix V. The agency also provided technical comments, which we incorporated into the report as appropriate. EPA provided written comments, in which it disagreed with the findings for two recommendations, partially agreed with the findings for one recommendation and disagreed with two other recommendations. EPA disagreed with the finding supporting our recommendation to update the security plans for the two selected systems to identify specific required information The agency stated that one of the systems we selected for review was no longer in production and not used for EPA's operations. Nevertheless, the agency stated that its chief information security officer would coordinate with the agency’s information security officers to ensure that security plans for the systems used to support its operations include all required information. We acknowledged in the report that EPA discontinued the system after we completed our review of the system’s authorization package. However, our recommendation in the draft report did not clearly convey that it was intended only for the operational system. Thus, we revised the recommendation to specify the system in operation. EPA disagreed with the finding supporting our recommendation to update the security control assessment report for one of the selected systems to identify the summarized results of control effectiveness tests. The agency stated that it used a FedRAMP certified third-party assessor that provided full documentation of control test results. However, neither the security assessment report nor other documents that EPA provided to us summarized information on how the agency tested the effectiveness of its corrective actions to rectify a critical control that had previously failed. As a result, EPA had limited assurance that it had effectively implemented a control that was intended to protect agency data in the cloud environment. Accordingly, we believe that our recommendation is warranted. EPA partially agreed with the finding supporting our recommendation to update the list of corrective actions for the selected systems to identify specific required information. The agency stated that one of the systems we selected for review was no longer in production and not used for EPA's operations. In addition, the agency said that the Chief Information Security Officer would coordinate with agency information security officers to ensure that plans of corrective actions and milestones include all required information, as appropriate. We acknowledged in the report that EPA discontinued its use of the system after we completed our review of the system’s authorization package. However, our recommendation in the draft report did not clearly convey that it was intended only for the operational system. As a result, we revised the recommendation to specify the system in operation. EPA disagreed with our recommendation that the agency prepare letters authorizing the cloud services for the selected systems and submit the letters to the FedRAMP program management office. The agency stated that one of the systems we selected for review was no longer in production and not used for EPA's operations. We acknowledged in the report that EPA had discontinued the system after we completed our review of the system's authorization package. However, our recommendation in the draft report did not clearly convey that it was intended only for the operational system. We have revised the recommendation accordingly. EPA also stated that it prepares and sends authorization letters for cloud services to the FedRAMP PMO. However, at the time of our review, the FedRAMP PMO stated it had not received the cloud service authorization letter from EPA for the selected operational system. We believe that our revised recommendation for EPA to prepare and send the cloud service authorization to the FedRAMP PMO for the operational system is warranted. EPA disagreed with our recommendation that the agency develop guidance requiring cloud service authorization letters to be provided to the FedRAMP program management office. The agency stated that it had a standard operating procedure in which the EPA Chief Information Security Officer forwards the letters to the FedRAMP program management office. However, the agency did not provide us a copy of the standard operating procedure or otherwise demonstrate that it had such an operating procedure. Thus, we continue to believe that the recommendation is warranted. EPA’s comments are reprinted in appendix VI. The agency also provided technical comments, which we incorporated into the report, as appropriate. Further, in written comments, USAID concurred with two of our three recommendations, but did not concur with the third. Specifically, USAID concurred with the two recommendations for the agency to update the list of corrective actions for the selected system and prepare the letter authorizing the use of cloud services supporting the system and submit it to the FedRAMP program management office. However, USAID did not concur with our recommendation to update the system security plan for the selected system to identify the authorization boundary, system operational environment and connections, and a description of security controls. The agency provided additional information that it had documented the authorization boundary, system operational environment and connections, and security controls for the selected system. Upon our review of the information, we agreed that the agency had sufficiently documented these items. Accordingly, we revised our report to reflect the agency’s actions and withdrew the recommendation from the report. USAID’s comments are reprinted in appendix VII. In addition to the aforementioned responses, two agencies—the Department of Veterans Affairs and the Social Security Administration— provided written responses stating that they had no comments on the draft report. These agencies’ responses are reprinted in appendixes VIII and IX, respectively. Also, the Department of Justice provided technical comments, which we incorporated into the report as appropriate. Sixteen CFO agencies provided emails stating that they had no comments on the draft report. These agencies were the Departments of Agriculture, Commerce, Defense, Education, Energy, Homeland Security, Housing and Urban Development, the Interior, Labor, State, Transportation, and the Treasury; as well as the National Aeronautics and Space Administration, National Science Foundation, Nuclear Regulatory Commission, and Office of Personnel Management. We did not receive a response from one agency—the Small Business Administration. We are sending copies of this report to appropriate congressional committees, the Director of the Office of Management and Budget, the 24 CFO Act agencies; and other interested parties. This report will also be available at no charge on our website at http://www.gao.gov. If you or your staff have any questions on matters discussed in this report, please contact Gregory C. Wilshusen at (202) 512-6244 or WilshusenG@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix X. Appendix I: Objectives, Scope, and Methodology Our objectives were to determine the extent to which 1) federal agencies used FedRAMP to authorize the use of cloud services, 2) selected agencies addressed key elements of the program’s authorization process, and 3) program participants identified FedRAMP benefits and challenges. The scope of our review included the 24 agencies covered by the Chief Financial Officers Act. To address the three objectives, we developed one survey for the 24 agencies and another survey for 83 cloud service providers identified by the FedRAMP Program Management Office (PMO) as participating in the program. We administered these web-based surveys between April and November 2018. We sent two follow-up email messages to all nonrespondents and subsequently attempted to contact the remaining nonrespondents by telephone or email at least twice more. To inform our survey questions and options, we designed our questionnaire based on FedRAMP PMO documentation and interviews with the 24 agencies and cloud service providers. We pretested the surveys with three major federal agencies, three cloud service providers, and one internal GAO group. We requested that agency chief information officers and chief information security officers review and confirm the results of the survey. We received completed surveys from 24 of 24 agencies (a 100 percent response rate) for our agency survey and 47 of the 83 cloud survey providers identified (a 57 percent response rate) for our cloud service provider survey. Not all survey respondents provided answers to all survey questions. With any survey, error can be introduced with respect to measurement of concepts, representation of respondents, and other factors, and we took steps to minimize these errors. We conducted a nonresponse bias analysis to determine whether certain cloud service providers might have been more or less likely to respond to the survey than others. Specifically, we examined whether a cloud service provider’s service model (e.g., Software as a Service, Infrastructure as a Service, Platform as a Service), impact level (e.g., high, moderate, low), or deployment model (e.g., government, hybrid, private) was related to whether the CSP responded to the survey. We found that a higher share of cloud service providers that provide Software as a Service (SaaS) responded to the survey than those that provide Infrastructure as a Service (IaaS). In addition, we found that a higher share of cloud service providers that deployed in the government community cloud responded to the survey than those that deployed in the public cloud. These results suggest that cloud service providers that utilize certain service or deployment models were more likely to reply to the survey than others. As a result, the responses of the cloud service provider survey represent only those cloud service providers that participated in this survey, and are not generalizable to cloud service providers as a whole. Despite these limitations, the survey results provide insight into the experiences and views of cloud service providers that did respond. In addition to the surveys, to address our first objective, we examined 2017, 2018, and 2019 Joint Authorization Board (JAB) and agency authorization data from the 24 agencies to determine if there were an increase, decrease, or no change in the usage of the program. We also interviewed knowledgeable officials from the 24 agencies and FedRAMP PMO to obtain their views on the program. To address our second objective, we selected four agencies from the 24 agencies based on those with the highest and lowest amount of FedRAMP PMO reported FedRAMP authorizations as of June 15, 2017. We selected the four agencies by dividing them into three equal groups of eight agencies based on the highest to lowest number of FedRAMP PMO reported service authorizations. We selected at least one agency with the highest number of authorizations through FedRAMP in each group, unless we conducted prior FedRAMP work with the agency. Given that two agencies in the third group had the same number of services authorized, we selected both agencies as one had a higher number of reported provisional authorizations through the FedRAMP Joint Authorization Board process and the other had the higher number of reported authorizations through the FedRAMP agency process. To avoid a duplication of our efforts given limited resources, we excluded DOD because another GAO team was reviewing the department’s cloud- related efforts, which included leveraging FedRAMP authorizations. As a result, we selected the Department of Health and Human Services, General Services Administration, the Environmental Protection Agency, and the United States Agency for International Development for our review. Because HHS is a large federated agency, we selected three operating divisions for a more detailed review. The three operating divisions included the Centers for Disease Control and Prevention (CDC), Centers for Medicare and Medicaid (CMS), and National Institutes of Health (NIH). We selected these divisions based on their extensive usage of cloud service providers authorized through FedRAMP. To select the agency systems’ authorization packages for review, we first identified six cloud services based on FedRAMP PMO data that indicated as of June 15, 2017, the 24 agencies used these cloud services the most. We then requested the selected agencies to provide us with an inventory of systems that relied on the six cloud services in fiscal years 2017 and 2018. From these inventories, we selected 10 agency systems. However, due to sensitivity concerns, we are not disclosing the names of the systems in this report. The case studies we selected are not generalizable to the other agencies covered by the Chief Financial Officers Act. However, it may show the potential FedRAMP issues other agencies face. For each agency system, we reviewed security authorization documentation, including: cloud service provider documentation, such as the Control Implementation Summary on agency and cloud service provider responsibilities to determine the extent agencies documented selected core controls and consistently documented responsibilities in the system security plan; security plans to determine the extent to which plans documented and implemented selected identified core security controls, and met FedRAMP and National Institute of Standards and Technology (NIST) elements; security assessment reports to determine if the effectiveness of selected core controls had been assessed and operating as intended; the extent to which agencies documented remedial action plans for selected systems to determine if they met FedRAMP or Office of Management and Budget (OMB) elements; and authorization letters to determine the extent appropriate officials approved a cloud service and agency system for use. To select identified core controls as part of our authorization documentation review, we identified and selected 24 security controls from the 97 identified core controls. Then, to determine the agencies’ compliance with the FedRAMP authorization process to assure the protection of agency data, we compared the authorization documentation with the Federal Information Security Modernization Act of 2014, the Federal Risk and Authorization Management Program guidance, including the program’s Security Assessment Framework, OMB guidance, and NIST Special Publication 800-53 Revision 4. Each authorization package area was examined and reviewed by an analyst and each conclusion was corroborated by a second analyst. Where there was disagreement in the assessment, analysts discussed their analysis and reached a consensus. In addition, we interviewed security representatives and management officials from our selected agencies to determine the effectiveness of the FedRAMP authorization process in reviewing the controls necessary for securing agency data in the cloud, and potential rationale for deficiencies identified in authorization documentation. We also interviewed FedRAMP PMO and OMB staff on their efforts related to the FedRAMP authorization process. To address our second and third objectives, we also interviewed JAB technical representatives to obtain their views on the benefits and challenges of FedRAMP. Additionally, we obtained information about how the JAB technical representatives reviewed authorization packages. To determine the reliability of the data used to select agencies and of other data to address our three objectives, we assessed the following: FedRAMP program management office points of contact list provided for active cloud service providers and federal agency users of FedRAMP, FedRAMP program management office data on the 24 CFO Act agencies’ fiscal years 2017, 2018, and 2019 JAB and agency authorizations, FedRAMP program management office data on cloud service provider participation and agency usage of FedRAMP as of June 15, 2017, Agency inventory of systems relying on selected cloud services, Cloud service provider authorization documentation contained within Cloud service provider and agency reported third-party assessment organizations’ security assessment reports, and Agency plans of actions and milestones. To assess the reliability of the information received and reviewed on the FedRAMP marketplace, we collected and reviewed information on agencies’ quality control procedures and asked program officials relevant questions on the FedRAMP authorization log standard operating procedure. We reviewed GSA program officials’ responses to our data reliability questions such as: how the information was generated, how current the data provided was, how frequently it was updated, and how the data was accurately and consistently entered into the system used. The limitation FedRAMP officials noted was that the data generated was based on voluntarily provided authorization to operate letters submitted to the FedRAMP program management office by each of the CFO Act agencies. To ensure that the agency systems we reviewed relied on selected cloud service provider products, we had agencies confirm their use of the service supporting the agency’s system. We then compared the selected services with agencies’ annual FISMA reporting to OMB along with system security documentation (e.g. system security plans) to determine whether the cloud service services we selected were applicable to the selected agency system. A limitation with this method of selection is if an agency’s inventory is inaccurate, we would need to reselect a system. For this review, one agency’s inventory and system was incomplete resulting in removing that agency system from our selection. To confirm agencies’ virtual access to packages in OMB’s repository or a cloud service provider’s repository, we obtained screen captures of web portal contents from the FedRAMP PMO. We compared these screen captures with our own virtual access to the packages. We also obtained additional information from the FedRAMP PMO on how it ensures the accuracy and reliability of the cloud service provider package information. One limitation of this method is that cloud service providers could update documentation where access was outside of OMB MAX portal, and the PMO may not be immediately aware of package updates. To verify the accuracy and reliability of plans of actions and milestones provided by agencies, we compared the agency’s plans of actions and milestones with required OMB elements. We also requested that agencies describe how they generated the plans of action and milestones provided to us, identify the quality control procedures used, and any limitations to the data they provided. We evaluated the materiality of the information we obtained and compared it to our audit objectives. We assessed the reliability of the information by reviewing related documents and internal controls such as agency policies and procedures as well as examining packages stored in OMB’s MAX portal and cloud service provider repositories. We also interviewed knowledgeable agency officials. Through these methods, we concluded that the information was sufficiently reliable for the purposes of our reporting objectives. We conducted this performance audit from November 2016 to December 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: FedRAMP Roles and Responsibilities Appendix II: FedRAMP Roles and Responsibilities Roles and responsibilities Issues policies which define the key requirements and capabilities of the FedRAMP program. Oversees and reports on agencies’ implementation of information security requirements, including implementation of FedRAMP. Develops processes for agencies and providers to request FedRAMP security authorization; Creates a framework for agencies to leverage security authorization packages; Establishes a centralized and secure repository for authorization packages that agencies can leverage to grant security authorizations; Coordinates with the National Institute of Standards and Technology ( NIST) and American Association for Laboratory Accreditation to implement a formal conformity assessment to accredit assessors; Develops templates for standard contract language and service level agreements , Memorandum of Understanding and/or Memorandum of Agreement; and Is led by GSA and serves as a liaison to ensure effective communication among all participants. Defines and updates the FedRAMP security authorization requirements; Approves accreditation criteria for third-party assessment organizations; Reviews security assessment packages of cloud service providers to grant provisional authorizations; Ensures provisional authorizations are reviewed and updated regularly; and Notifies agencies of changes to or removal of provisional authorizations. Advises FedRAMP on FISMA compliance guidance and assists in developing the standards for the accreditation of independent third-party assessment organizations (3PAO). Distributes FedRAMP information to federal CIOs and other representatives through cross- agency communications and events. Assists government-wide and agency-specific efforts to provide adequate, risk-based and cost- effective cyber security; Coordinates cyber security operations and incident response; Develops continuous monitoring standards for ongoing cyber security of federal Information systems; and Develops guidance on agency implementation of the Trusted Internet Connection program with cloud services. Appendix IV: Comments from the General Services Administration Appendix V: Comments from the Department of Health and Human Services Appendix VI: Comments from the Environmental Protection Agency Appendix VII: Comments from the U.S. Agency for International Development Appendix VIII: Comments from the Department of Veterans Affairs Appendix IX: Comments from the Social Security Administration Appendix X: GAO Contact and Staff Acknowledgments GAO Contact Gregory C. Wilshusen, (202) 512-6244 or wilshuseng@gao.gov. Staff Acknowledgments In addition to the individual named above, Sara Ann W. Moessbauer, (Director), Larry Crosland (Assistant Director), Rosanna Guerrero (Analyst-in-Charge), Sher’rie Bacon, Nabajyoti Barkakati , Christina Bixby, David Blanding, Chris Businsky, Fatima Jahan, David Plocher, Dana Pon, Carl Ramirez, Cynthia Saunders, and Priscilla Smith made significant contributions to this report.