Written by Kip Peters

How to Evaluate SaaS Security

This is part 3 of a 4-part series called Breaking the Barrier to SaaS Implementation. This blog series explores best practices in vetting SaaS vendors to ensure data protection and streamlined workflows throughout product design, manufacturing, and lifecycle support.

To fully determine whether a cloud-based solution meets your security requirements, manufacturing organizations need to first understand the value of their organization’s data and their internal control environment. Then they must apply that information against the security capabilities and features provided by potential cloud providers. When an organization understands how the precautions and features implemented by a cloud provider (specifically software as a service or SaaS for this blog) support their internal requirements, they are able to make better decisions about their data protection and avoid potential risks.


Before you begin: Understand yourself

As mentioned in my last blog, the heart of risk management is to understand the risks you are managing. This begins with knowing what I’m going to call your internal data and control environment. What data are you planning to use in the SaaS solution and what is its classification at your organization? Normally, the higher the classification of data—not necessarily a government term—the more protective an organization is of the data. Intellectual property might be classified at the organization’s highest level, whereas public information is probably at the lowest. In most organizations, this information drives the security requirements required of any solution. Rather than look at the security of a SaaS solution somewhat independently, companies must consider the SaaS solution in light of their data’s defined requirements.

With this information, you can compare the features and capabilities of SaaS providers against the posture expected by your organization. So let’s move to the next step.

Then: Evaluate the SaaS Solution

Once you understand yourself, it’s time to understand and assess the SaaS solution your organization is considering. Many SaaS providers will be able to share some sort of report on their security posture. Organizations can also use their own internal checklists or questionnaires to evaluate the security of the provider. Most organizations will do both and tailor their self-assessment based on what they can obtain from the provider. Here is some guidance on both of those steps.

1. Gather Evidence
Many SaaS providers can provide one or more pieces of evidence to indicate how well they have implemented their security. The first is the Cloud Security Alliance’s (CSA) Security Trust Assurance and Risk (STAR) Program. The STAR Program provides both a means for a SaaS provider (actually any cloud provider) to validate and offer proof of their security controls, and for customers to review the organizations that meet their required level of assurance. There are multiple levels of STAR assurance, including a provider self-assessment and a third-party audit-based attestation and/or certification. Assuming a SaaS provider has engaged in the STAR Program, an organization can review a point-in-time self-assessment or third-party assessment to evaluate the control environment of the provider.

I also recommend reviewing the System and Organization Controls (SOC) suite of services. Provided by CPAs, SOC evaluations follow guidance from the American Institute of CPAs (AICPA). There are three main SOC internal control reports – SOC 1, SOC 2, and SOC 3:

  • The SOC 1 is related to internal controls over financial reporting and is generally irrelevant for this discussion. 
  • The SOC 3 is a broad report on the assurance levels of a provider’s security and privacy controls, but at a level high enough that these reports are generally made publicly available and provide little use to the potential customer that needs to understand a provider’s controls to any level of detail. 
  • Most relevant to our discussion here are the SOC 2 reports, which today are based upon the 2017 Trust Services Criteria established by the AICPA’s Assurance Services Executive Committee.

The SOC 2 provides detailed information concerning the security, availability, and processing integrity of a service provider’s processing environment. There are two types of SOC 2 reports. The type 1 version describes the provider’s system and how suitably designed the controls are. The type 2 version expands on the type 1 by assessing the controls’ operating effectiveness. By leveraging a SOC 2 Type 2, a potential customer can obtain a solid reference of the design and effectiveness of a provider’s controls.

In addition, there is a special version of the SOC 2 Type 2 that includes criteria from the CSA Cloud Controls Matrix. Available only from an auditors certified to provide CSA STAR attestations, a qualified report would essentially provide CSA STAR Attestation for a cloud service provider. 

The final potential evidence opportunity for our purposes is certification with ISO/IEC 27001. While ISO/IEC 27001 addresses a more “standard” information security management system, there isn’t a specific standard to the cloud. That said, ISO/IEC 27017 provides a code of practice for both cloud service providers and cloud service customers. A cloud service provider uses ISO/IEC 27017as an extension to ISO/IEC 27001 requirements and guidance while going through the certification process.

Similar to the CSA STAR Attestation, a cloud service provider can leverage both the ISO/IEC 27001 protocol and the CSA Cloud Controls Matrix (using CSA STAR Certification approved auditors) to receive what is known as CSA STAR Certification.

2. Perform a Self-Assessment
Depending on what the SaaS provider can share proactively, potential customers may decide to conduct their own self-assessment. As you can guess, this can take many, many forms. More organizations than ever before are getting serious about third-party risk management, and are therefore implementing tools to help manage those risks. These tools, known as Governance, Risk, and Compliance or Integrated Risk Management solutions, often include security questionnaires that can be leveraged to assess the security assurance of a third-party. There are many varieties of these solutions, and I encourage you to be sure to understand if these questionnaires are appropriate for SaaS providers. A questionnaire intended for non-cloud providers will generally provide a false sense of risk when applied to cloud service providers.

I recommend using CSA’s evaluation that I have mentioned in a previous blog, called the Consensus Assessments Initiative Questionnaire (CAIQ). The CAIQ is available in both a full and a lite version and can be used by an organization to assess a cloud provider. The CAIQ is industry-neutral and very well accepted as a good way to either supplement or replace current cloud assessment practices at any organization.

One final bit of advice on self-assessment: keep in mind your industry and other laws or standards you must comply with. This goes back to my point about understanding yourself. More and more, cloud-specific guidance or standards are being released that you should use as a source of evaluation for cloud providers. For example, the PCI Security Standards Council has issued cloud computing guidelines as has the U.S. Department of Health & Human Services in regards to HIPAA. In addition, some of your customers may have requirements that you will need to evaluate and ensure are met by cloud providers. These things will all need to be considered as you move forward.

Have a digital twin project? Learn common challenges, opportunities, and tips to succeed.

Connect to the source of truth, empower product innovation, and improve design reviews.

Create a connected customer experience & bring them into the product lifecycle.

Inundated with IoT data? Get insights into your smart factory.

8 Things to look for when determining a SaaS provider

Let’s say you now have some evidence and you’re either building your own questionnaire, adapting an existing questionnaire, or you’ve received a questionnaire back from a SaaS provider. What should you look for or think about as you evaluate what you have on your desk? As a manufacturer, here is a list of some of the key things to consider when evaluating a SaaS provider. Please keep in mind that this is not a complete list and only highlights some of the key aspects that should be considered. 

  1. Manage your users and their access. If possible, leverage multifactor authentication and your own identity solution with federated access to the SaaS solution. Ensure that the system’s users have appropriate access by their role, and make sure that user rights are either adapted or removed as quickly as possible if their role changes or they leave your organization. 
  2. Use strong encryption end-to-end; from your endpoints to the SaaS solution, while stored at the SaaS solution, and in all other potential environments. 
  3. Secure APIs. If you use APIs, and in particular if you use internet-accessible APIs, ensure they are protected appropriately even if they are management-related APIs. 
  4. Understand how the SaaS provider keeps their service current. How do they install security patches, ensure versions are current, and keep the solution free from security vulnerabilities? 
  5. Understand how the provider evaluates their security. They should be doing very frequent vulnerability assessments as well as internal and external penetration tests to ensure their solution exhibits an appropriate level of assurance. 
  6. Have visibility of cloud resources and users. How will the provider give you logging and monitoring information from the SaaS solution for your own security systems? How will the provider monitor the solution themselves? What tools will they provide for you to use to maintain that visibility? 
  7. Understand how you and the SaaS provider will work together to respond to incidents – of all kinds, but in our case, specific to security. How will they let you know of known or suspected incidents, and when? How will you let them know if you identify a problem? How do you call out and understand requirements and expectations? 
  8. Remain vigilant. Once you decide to move forward, continue to monitor and re-evaluate your SaaS provider. Whether it’s an annual review of their renewed certification or attestation, an annual re-issuance of your security questionnaire, or other periodic reviews of information shared by the SaaS provider, make sure you are regularly ensuring that they meet requirements. As time goes on, it’s imperative to maintain the initial level of confidence in the provider. 

By this point, you should have a good sense of the security posture of any SaaS provider under evaluation. The final step is to collaboratively put a contract in place between your organization and the winning vendor. In my final blog, I will explore some of the key contract components that should be in place.

Stay current with digital transformation news once a week.

About Kip Peters

Kip Peters has over 25 years of experience in information assurance, data security, risk management, and technology leadership. He is an expert on compliance regulations and security laws and how they apply to both cloud computing and to high-risk industries including manufacturing, financial, and defense.