Omegapoint security review questionnaire

6 October 2023

This post defines a set of questions Omegapoint uses for security reviews. The purpose is to be able to cover many aspects of security for a cloud-native DevOps team, during a two-hour interview with the team.

The questions are based on experience from both building and reviewing systems using CIS Controls v8. Each main question has a set of sub-questions with mappings to verifications in Omegapoints verifications of CIS Controls v8 for cloud-native DevOps teams. But it can, of course, be adapted to other kinds of teams as well.

The goal is that with these questions the team should be focused on building secure systems, not compliance. Note that we do not devalue compliance requirements. The point we are making is that the primary focus for the team should be on building secure high-quality systems, not meeting the minimal set of compliance requirements.

Before using the set of questions below it is important to first do a threat model exercise which, for the given scope (team and system, not all systems for a customer), will help the team:

Such a threat model, together with system documentation, should answer the following questions:

The threat model will cover parts of CIS v8 Control 12, and 16. If no threat model exercise has been performed, you will need to ask these questions as well.

The following sections will provide details for each of the main questions:

  1. How do you go from commit to deployed code in production?
  2. How do you work with threat modeling and penetration tests?
  3. How do you maintain a shared understanding of the system?
  4. How do you work with maintaining your system?
  5. How do you work with infrastructure and configuration?
  6. How do you work with monitoring and logs?
  7. How do you work when there is an incident?
  8. How do you work with disaster recovery, data restoration, purge, and retention?
  9. How do you manage accounts and access control?
  10. How do you work with the management of devices in development and operations?

Note that the order is not a priority, just a way to structure an interview to cover most CIS Controls Safeguards during an interview.

How do you go from commit to deployed code in production?

There are many ways a team can design a deployment pipeline. Team size and composition, combined with architecture, source code organization and system size place different requirements on branch models and deployment. There is no one-size-fits-all.

We are looking for a structured approach where everyone in the team works and agrees on a model. A lightweight approach to documentation can be a README file in the source code repository that outlines how the team works in combination with a backlog (in a typical open-source fashion.)

Are the commits reviewed by more than the author? Is the review enforced?
Verifications: V1.B, V2.B
Does a pull request link to a work item? Is it required?
Verifications: V1.C, V2.C
Do you run automated tests as part of the CD/CI pipeline?
Verifications: V16.E
What kind of scanners do you run?
Verifications: V2.E, V7.B, V7.C
How are changes build and deployed to the different environments?
Verifications: V1.D, V2.D, V16.E
Are the artifacts unchanged between test and deploy
Verifications: V16.E
Are commit messages descriptive, or optional?
Verifications: V1.C, V2.C
How do you test your application?
Verifications: V16.E, V18.A
How quickly can you change the system while maintaining low risk?
Verifications: V7.D
Are all software components under version control?
Verifications: V1.A, V2.A

How do you work with threat modeling and penetration tests?

Continuously using threat modeling and penetration test is vitaly important for maintaining a high-security posture.

Threat modeling can be performed on different levels, in example for the entire organization or technically detailed for a given system scope together with penetration testing. Preferably it is also a part of the team´s way of working, where the team has a lightweigth risk-based approach to changes in the system, asking the following questions (from e g https://owasp.org/www-community/Threat_Modeling):

Note that threat modeling is closely related to the verifications V3.A and V12.A, as it should have affect on system design and implementation.

Also note that even if a recurring, structured approach to penetration testing is rarely handled by the team itself, it is important that the team understands strategies for penetration testing, how results concerns them, and how these activities relates to threat modeling.

Make sure that the team has considered the following:

Does threat modeling affect your system design?
Verifications: V16.D
Do you have a risk-based approach to changes in the system, as part of way-of-working and deployment procedures?
Verifications: V16.D
Do you perform penetration tests periodically and on significant changes?
Verifications: V18.A
Do you work with both internal security tests and external independent penetration testing?
Verifications: V18.A
Do penetration tests include both infrastructure, networking and application security?
Verifications: V18.B
How do you address the results? Are findings traceable to backlogs? Have high risk findings been remediated?
Verifications: V18.C

How do you maintain a shared understanding of the system?

One of the most important parts of this section is how the team works with requirements. We recognize the conflict between innovation and control. Features that are not specified risk affecting the security of the application. At the same time, restricting the team’s freedom to implement new ideas without approved requirements limits innovation. Based on the risk appetite of each organization and team, there is a need to establish a balance between innovation and control. This section focuses on verifying this agreed balance in an established security baseline. The team needs to share an understanding of quality and security for their application.

Make sure that the team has considered the following:

Where do requirements come from?
Verifications: V16.A
Are security considerations part of requirements?
Verifications: V12.D, V16.A, V16.E
Does the team share a documented understanding of the system?
Verifications: V2.G, V8.I, V16.A, V16.B
Is ownership of all infrastructure and applications well understood?
Verifications: V1.F, V2.G, V8.I
Do all commits originate from a unique work item?
Verifications: V16.A
Which reference architectures are used for technical guidance?
Verifications: V16.E

How do you work with maintaining your system?

We are looking for information about how the team deals with existing code and completed features. For any larger or older system still under development, there are typically parts that are not worked on often. How does the team deal with security for those parts?

Hardening of infrastructure needs to be revisited since recommendations and BCP change over time.

Make sure that the team has considered the following:

How do you manage updating and patching operating systems and packages?
Verifications: V7.A, V7.B, V7.C
How do you manage life cycle of libraries and third-party SaaS?
Verifications: V2.F, V15.A
How do you monitor changes in regulations?
Verifications: V14.A, V16.A
How you deal with key personnel and resource limitations?
Verifications: ISO 27002:2022 8.6
Does the team have role specific security training?
Verifications: V10.A, V14.A, V16.C
Is system design and data classifications up to date?
Verifications: V3.A, V12.B

How do you work with infrastructure and configuration?

We are focusing on how changes in infrastructure and configuration are managed. Any environment or device that contains production data must be treated as a production environment with the same level of protection and monitoring.

Make sure that the team has considered the following:

Is production separated from all other environments?
Verifications: V1.E, V3.H, V4.C
Is production data saved anywhere else than production?
Verifications: V3.I
Is there a process to find unauthorized resources?
Verifications: V1.F
Does the infrastructure follow secure practices?
Verifications: V3.E, V3.F, V4.D, V4.E, V4.F, V4.G, V7.C, V12.A, V12.E, V13.A
How do you work with secrets? (Managed identity and rotation).
Verifications: V4.B, V5.F
How do you work with and access data from prod environments
Verifications: V12.C

How do you work with monitoring and logs?

We are looking for information about the quality of the logs. Can the logs be used to detect ongoing attacks or used as an audit tool? Are there automated scans looking for anomalies in the logs?

Make sure that the team has considered the following:

Are logs centralized, reviewed, and retained for a minimum of 90 days?
Verifications: V3.G, V8.A, V8.C, V8.D, V8.E, V13.A
Are alarms automated, periodically tuned, and updated?
Verifications: V3.G, V3.K, V8.G, V8.H, V13.B, V13.E
Can you detect system misuse or a penetration test?
Verifications: V8.B, V8.F
Can you detect unauthorized resources in the application network?
Verifications: V1.F
Is access to logs managed based on information classification?
Verifications: V5.A, V5.C, V6.B, V6.C, V6.H

How do you work when there is an incident?

Incident handling may be handled at the company level. We are looking for information about whether the team bears the responsibility of incident handling.

We are also looking for how the team communicates during an incident. Can the team effectively assemble enough resources in a timely manner? Do the team know who to contact?

Make sure that the team has considered the following:

Is there a plan for incident handling?
Verifications: V17.A, V17.B
Does the team know what to do when there is an incident?
Verifications: V16.B, V17.B, V17.A
Does the team know what to do after a incident?
Verifications: V17.A

How do you work with disaster recovery, data restoration, purge, and retention?

We are looking for information about how the team optimizes its “time to recover” and how data can be restored if an incident occurs. We also look at how the data can be purged, especially if the data stored is subject to the GDPR regulation.

Make sure that the team has considered the following:

How is data and configuration backed up?
Verifications: V11.A, V11.B, V11.C
How often is the disaster recovery process tested?
Verifications: V11.D, V11.E
What is the process for purging data?
Verifications: V3.C

How do you manage accounts and access control?

Here are looking for a documented on and off boarding process, both for team members and for integration clients. How well is this process is executed? We also look at how developers authenticated to different system and how the access control is managed.

Make sure that the team has considered the following:

How is identity and access controlled and maintained?
Verifications: V3.A, V3.J, V4.A, V6.H, V6.J
Verify that there is a documented on- and off-boarding process
Verifications: V3.B, V6.B, V6.C, V6.D, V6.E, V6.G
How do you work with accounts and authentication (for all things)
Verifications: V5.A, V5.B, V5.C, V5.D, V5.E, V5.F, V6.F
How do you maintain the privileges of members and integration clients?
Verifications: V6.A, V6.H, V6.I

How do you work with the management of devices in development and operations?

We look to check if device management is part of the IT policies and, therefore, out of scope. However, some questions about how the policy is followed can be used to emphasize common security problems and give an indication of the cyber hygiene within the team.

Make sure that the team has considered the following:

Is only the minimal amount of browser plugin used?
Verifications: V9.A, V9.B
How do you evaluate software that you install on devices?
Verifications: V9.B
Is disk encryption enabled on all devices and storage?
Verifications: V3.D, V9.A
Are the devices used for other purposes not related to the project?
Verifications: V9.A