Omegapoint security review questionnaire

This post defines a set of questions Omegapoint uses for security reviews. The purpose is to be able to cover many aspects of security for a cloud-native DevOps team, during a two-hour interview with the team.

The questions are based on experience from both building and reviewing systems using CIS Controls v8. Each main question has a set of sub questions with mappings to verifications in Omegapoints verifications of CIS Controls v8 for cloud-native DevOps teams. But it can of course be adapted to other kinds of teams as well.

The goal is that with these questions the team should be focused on building secure systems, not compliance. Note that we do not devalue compliance requirements. The point we are making is that the primary focus for the team should be on building secure high-quality systems, not meeting the minimal set of compliance requirements.

Before using the set of questions below it is important to first do a threat model exercise which, for the given scope (team and system, not all systems for a customer), will help the team:

• Understand the system and its boundaries from a security perspective.
• Identify data information classes and zones (trust boundaries).
• Understand and define access control models, e g multi-tenancy.
• Identify concerns, weaknesses, and mitigations of security characteristics.
• Understand threat actors, impacts, and the team’s threat priorities.

Such a threat model, together with system documentation, should answer the questions:

• How does the architecture address Zero trust, Least Privilege and Defense in depth strategies? In example network segmentation, encryption at rest, transport layer security, authentication and authorization at the application layer.
• How does the architecture address requirements from relevant reference architectures and compliance? In example GDPR, DORA, NIS2 etc.

The threat model will cover CIS v8 Control 12, and 16. If no threat model exercise has been performed, you will need to ask these questions as well.

The following sections will provide details for each of the main questions:

1. How do you go from commit to deployed code in production? [Build]
2. How do you assert that code is high-quality and secure? [Build]
3. How do you work with maintaining your system? [Build]
4. How do you work with infrastructure and configuration? [Build]
5. How do you work with monitoring and logs? [Operate]
6. How do you work when there is an incident? [Defend]
7. How do you work with disaster recovery, data restoration, purge, retention? [Operate, Defend]
8. How do you manage accounts and access control? [Operate]
9. How do you work with management of devices in development and operations? [Operate]

Note that the order is not a priority, just a way to structure an interview to cover most CIS Controls Safeguards during an interview.

How do you go from commit to deployed code in production?

There are many ways a team can design a deployment pipeline. Team size and composition, combined with architecture, source code organization and system size place different requirements on branch models and deployment. There is no one-size-fits-all.

We are looking for a structured approach where everyone in the team works and agrees on a model. The antithesis is a team where most developers work one way, but a few “heroes” are free to work outside the model. A lightweight approach can be a README in the source code repository that outlines the way the team works, in combination with a backlog (in typical open-source fashion.)

Make sure that the team has considered the following:

• Are the commits reviewed by more than the author? Is the review enforced?
• Does a pull request link to a work item? Is it required?
• Do you run automated tests as part of the CD/CI pipeline?
• What kind of scanners do you run?
• Are all changes built and deployed to all environments from a CI/CD pipeline?
• Are the artifacts unchanged between test and deploy?
• Are commit messages descriptive, or optional?
• How do you test your application?
• How quickly can you change the system while maintaining low risk?
• Are all software components under version control?

How do you understand your system?

One of the most important parts of this section is how the team works with requirements. We recognize the conflict between innovation and control. Features that are not specified risk affecting the security of the application. At the same time, restricting the team’s freedom to implement new ideas without approved requirements limits innovation. Each organization and team need to establish a balance, and we need to verify that they meet this agreed balance.

We are looking for an established security baseline. The team needs to share an understanding of quality and security for their application.

It is important that the team understands what part of the infrastructure it is responsible for, compared to the cloud provider.

Make sure that the team has considered the following:

• Where do requirements come from?
• Are security considerations part of requirements?
• Does the team share a documented understanding of the system?
• Is ownership of all infrastructure and applications well understood?
• Do all commits originate from a unique work item?
• Which reference architectures are used for technical guidance?

How do you work with maintaining your system?

We are looking for information about how the team deals with existing code and completed features. For any larger or older system still under development, there are typically parts that are not worked on often. How does the team deal with security for those parts?

Hardening of infrastructure needs to be revisited since recommendations and BCP change over time.

Make sure that the team has considered the following:

• How do you manage updating and patching operating systems and packages?
• How do you manage life cycle of libraries and third-party SaaS?
• How do you monitor changes in regulations?
• How you deal with key personnel and resource limitations?
• Does the team have role specific security training?
• Is system design and data classifications up to date?

How do you work with infrastructure and configuration?

We are focusing on how changes in infrastructure and configuration are managed. Any environment or device that contains production data must be treated as a production environment with the same level of protection and monitoring.

Make sure that the team has considered the following:

• Is production separated from all other environments?
• Is production data saved anywhere else than production?
• Is there a process to find unauthorized resources?
• Does the infrastructure follow secure practices?
• How do you work with secrets? (Managed identity and rotation).
• Is all data encrypted at rest?

How do you work with monitoring and logs?

We are looking for information about the quality of the logs. Can the logs be used to detect ongoing attacks or used as an audit tool? Are there automated scans looking for anomalies in the logs?

Make sure that the team has considered the following:

• Are logs centralized, reviewed, and retained for a minimum of 90 days?
• Are alarms automated, periodically tuned, and updated?
• Can you detect system misuse or a penetration test?
• Can you detect unauthorized resources in the application network?
• Is access to logs managed based on information classification?

How do you work when there is an incident?

Incident handling may be handled at the company level. We are looking for information about whether the team bears the responsibility of incident handling.

We are also looking for how the team communicates during an incident. Can the team effectively assemble enough resources in a timely manner? Do the team know who to contact?

Make sure that the team has considered the following:

• Is there a plan for incident handling?
• Dose the team know what to do when there is an incident?

How do you work with disaster recovery, data restoration, purge, and retention?

We are looking for information about how the team optimizes its “time to recover” and how data can be restored if an incident occurs. We also look at how the data can be purged, especially if the data stored is subject to the GDPR regulation.

Make sure that the team has considered the following:

• How is runtime data and configuration backed up?
• How often is the disaster recovery process tested?
• What is the process for purging data?

How do you manage accounts and access control?

Here are looking for a documented on and off boarding process, both for team members and for integration clients. How well is this process is executed? We also look at how developers authenticated to different system and how the access control is managed.

Make sure that the team has considered the following:

• How is identity and access controlled and maintained?
• Verify that there is a documented on- and off-boarding process
• How do you work with accounts and authentication (for all things)
• How do you maintain the privileges of members and integration clients?

How do you work with management of devices in development and operations?

We look to check if device management is part of the IT policies and, therefore, out of scope. However, some questions about how the policy is followed can be used to emphasize common security problems and give an indication of the cyber hygiene within the team.

Make sure that the team has considered the following:

• Is only the minimal amount of browser plugin used?
• How do you evaluate software that you install on devices?
• Is disk encryption enabled on all devices and storage?
• Are the devices used for other purposes not related to the project?