Rethinking DevOps as DevSecOps
Thursday, October 12, 2017
|
Akshay Aggarwal |
DevOps and DevSecOps tips for developers to improve security for their various deployments.
If you’re not already thinking right now that your DevOps teams should be run like a DevSecOps team, you may already be in a world of hurt. Time to wake up! As the adoption of APIs continues to grow, so do the risks to organizations that don’t actively test the security of their solutions.
Modern Agile development frameworks have changed the way engineering teams produce products. Under these frameworks, products receive small, frequent updates daily or weekly rather than major quarterly or annual product releases. These frameworks also rely on continuous integration (CI) systems to facilitate each build. While the product’s efficacy and flexibility are improved by these small releases, each new build introduces the potential for security vulnerabilities.
At the time of this writing there are more than 18,000 public APIs available. Continued growth and adoption of APIs is expected over the next decade as APIs power the backends of many of the most critical services.
Unfortunately the security tools used to test APIs are manual, slow to deploy, occur late in the development process, and rely on experienced penetration testers. Indeed, taking just one quick look at the current landscape of security testing tools, several gaps clearly emerge:
Many existing solutions are generalist security tools - not designed to perform security testing on modern APIs. These web-scanner style tools work by pointing to a single service, API, or endpoint. The tool then crawls that service and captures the traffic that is sent between the API and the client. By analyzing this data, the tool returns a pass or fails result, and while vulnerabilities need to be individually and manually located and verified by the penetration tester before it can be sent to a developer.
Older point-and-shoot tools also require users to manually configure the tool for each API endpoint before they can be tested. This laborious configuration step wastes time and leads to mistakes in testing; plus, these tools can only perform one test at a time. They also do not support modern authentication schemes used by most APIs and applications. This gap in support leads to poor code coverage and vulnerabilities.
Security tools should work with the tools developers already use to create and manage their products. Current security tools lack basic integrations, such as the ability to capture, analyze, and store log files, integrate with CI build systems, or send fault findings to common bug tracking software solutions. Intended to be used by security experts and pen testers, they also require manual efforts to test for, verify, and log vulnerabilities, which is very costly.
Because of the difficulty developer teams face in integrating existing security tools into their workflows, testing of products is often performed by security professionals late in the development process. Bugs are much more expensive to fix the later they are found. The time and cost to manually test products (see above) frequently encourages testing to occur only once per major development cycle rather than for every new build of the product. This results in intermediate builds of products being put into production without proper testing.
Existing security tools do not have the ability to stop builds with known vulnerabilities from being released to a production environment. Additionally, the time it takes to find and fix all vulnerabilities when they are discovered late can cause significant ship date delays.
Instead, a more efficient and streamlined approach to security would be to take what we call a DevSecOps approach, and automate API security testing around the engineering team’s current workflow to more closely and effectively navigate the complexity throughout the entire development lifecycle.
1. API Specific Security Testing
Your team should be testing against vulnerability lists that highlight new and emerging vulnerabilities, such as the OWASP Top 10. These typically surface during complex interactions between multiple APIs and services.
2. Automated Test Case Generation
Your engineering teams already have their automation or unit tests to ensure that your products behave as expected. These unit tests are designed to provide good code coverage and pass through the product’s authentication schemes. Thus, you should be configuring automation testing frameworks that enable the conversion of your current unit tests into security tests. Leveraging already created unit tests moves security testing to “the left” or earlier in the development lifecycle. Bugs fixes here are cheaper and easier.
3. Don’t Break Developer Workflow
Continuous integration is a software engineering practice in which isolated changes are immediately tested and reported on when they are added to a larger code base. You should run your security checks as a step within your current developer build pipeline. When faults are discovered, vulnerable builds should be automatically prevented from being put into production. Here, all interactions should occur from within the tooling your developers are already familiar with. You then don’t have to invest time and resources altering your team’s workflows or teaching your engineers to use new security tools. Security is baked into every build automatically.
4. Fuzzing
This is when your team “mutates” valid messages which can uncover additional security vulnerabilities that are not covered when doing your automated OWASP testing. This step is critical to discovering those company destroying 0-days.
5. Testing Profiles
Speed and shipping deadlines are critical for your teams that push builds multiple times a day. Use tools that allow users to balance security testing coverage with ship deadline requirements. Each testing profile should be configurable, allowing your teams to include or exclude certain checks or modes of testing.
6. Fault Results
Findings should include the information required for your developer to fix the issue quickly. All fault findings should be viewable from within the CI or bug tracking system the engineering team uses.
7. False Positives
Finally, one common shortfall of existing automated security testing solutions is management of false positives. It takes considerable time for your team to manually sort and determine which faults are valid bugs and which can be safely ignored. One option is to customize how faults are managed so that false positives do not slow development. When a failure is determined to be a false positive or a team decides not to fix an issue, it can be added to a list of automatically ignored failures. Future builds will still test for the fault, but will not report it in ticket management systems or block builds from deploying.
To better compete and head off major security concerns it is paramount that organizations deploy automated, scalable processes for security testing of their APIs. Key to that success is ensuring security is properly tuned along the entire development lifecycle. This requires new thinking and much closer testing that ultimately turns DevOps teams into DevSecOps teams.
This content is made possible by a guest author, or sponsor; it is not written by and does not necessarily reflect the views of App Developer Magazine's editorial staff.
Modern Agile development frameworks have changed the way engineering teams produce products. Under these frameworks, products receive small, frequent updates daily or weekly rather than major quarterly or annual product releases. These frameworks also rely on continuous integration (CI) systems to facilitate each build. While the product’s efficacy and flexibility are improved by these small releases, each new build introduces the potential for security vulnerabilities.
At the time of this writing there are more than 18,000 public APIs available. Continued growth and adoption of APIs is expected over the next decade as APIs power the backends of many of the most critical services.
Examples of critical services API's power include:
- Mobile Applications
- Web Applications
- Desktop Applications
- Browser Applications
- Microservices
- IoT
- Embedded Devices
Unfortunately the security tools used to test APIs are manual, slow to deploy, occur late in the development process, and rely on experienced penetration testers. Indeed, taking just one quick look at the current landscape of security testing tools, several gaps clearly emerge:
API Specific Testing
Many existing solutions are generalist security tools - not designed to perform security testing on modern APIs. These web-scanner style tools work by pointing to a single service, API, or endpoint. The tool then crawls that service and captures the traffic that is sent between the API and the client. By analyzing this data, the tool returns a pass or fails result, and while vulnerabilities need to be individually and manually located and verified by the penetration tester before it can be sent to a developer.
Older point-and-shoot tools also require users to manually configure the tool for each API endpoint before they can be tested. This laborious configuration step wastes time and leads to mistakes in testing; plus, these tools can only perform one test at a time. They also do not support modern authentication schemes used by most APIs and applications. This gap in support leads to poor code coverage and vulnerabilities.
Limited Integration
Security tools should work with the tools developers already use to create and manage their products. Current security tools lack basic integrations, such as the ability to capture, analyze, and store log files, integrate with CI build systems, or send fault findings to common bug tracking software solutions. Intended to be used by security experts and pen testers, they also require manual efforts to test for, verify, and log vulnerabilities, which is very costly.
End of Development
Because of the difficulty developer teams face in integrating existing security tools into their workflows, testing of products is often performed by security professionals late in the development process. Bugs are much more expensive to fix the later they are found. The time and cost to manually test products (see above) frequently encourages testing to occur only once per major development cycle rather than for every new build of the product. This results in intermediate builds of products being put into production without proper testing.
Preventing Releases
Existing security tools do not have the ability to stop builds with known vulnerabilities from being released to a production environment. Additionally, the time it takes to find and fix all vulnerabilities when they are discovered late can cause significant ship date delays.
Instead, a more efficient and streamlined approach to security would be to take what we call a DevSecOps approach, and automate API security testing around the engineering team’s current workflow to more closely and effectively navigate the complexity throughout the entire development lifecycle.
In this DevSecOps scenario the process is wrapped around these 7 key fundamentals:
1. API Specific Security Testing
Your team should be testing against vulnerability lists that highlight new and emerging vulnerabilities, such as the OWASP Top 10. These typically surface during complex interactions between multiple APIs and services.
2. Automated Test Case Generation
Your engineering teams already have their automation or unit tests to ensure that your products behave as expected. These unit tests are designed to provide good code coverage and pass through the product’s authentication schemes. Thus, you should be configuring automation testing frameworks that enable the conversion of your current unit tests into security tests. Leveraging already created unit tests moves security testing to “the left” or earlier in the development lifecycle. Bugs fixes here are cheaper and easier.
3. Don’t Break Developer Workflow
Continuous integration is a software engineering practice in which isolated changes are immediately tested and reported on when they are added to a larger code base. You should run your security checks as a step within your current developer build pipeline. When faults are discovered, vulnerable builds should be automatically prevented from being put into production. Here, all interactions should occur from within the tooling your developers are already familiar with. You then don’t have to invest time and resources altering your team’s workflows or teaching your engineers to use new security tools. Security is baked into every build automatically.
4. Fuzzing
This is when your team “mutates” valid messages which can uncover additional security vulnerabilities that are not covered when doing your automated OWASP testing. This step is critical to discovering those company destroying 0-days.
5. Testing Profiles
Speed and shipping deadlines are critical for your teams that push builds multiple times a day. Use tools that allow users to balance security testing coverage with ship deadline requirements. Each testing profile should be configurable, allowing your teams to include or exclude certain checks or modes of testing.
6. Fault Results
Findings should include the information required for your developer to fix the issue quickly. All fault findings should be viewable from within the CI or bug tracking system the engineering team uses.
7. False Positives
Finally, one common shortfall of existing automated security testing solutions is management of false positives. It takes considerable time for your team to manually sort and determine which faults are valid bugs and which can be safely ignored. One option is to customize how faults are managed so that false positives do not slow development. When a failure is determined to be a false positive or a team decides not to fix an issue, it can be added to a list of automatically ignored failures. Future builds will still test for the fault, but will not report it in ticket management systems or block builds from deploying.
Conclusion
To better compete and head off major security concerns it is paramount that organizations deploy automated, scalable processes for security testing of their APIs. Key to that success is ensuring security is properly tuned along the entire development lifecycle. This requires new thinking and much closer testing that ultimately turns DevOps teams into DevSecOps teams.
This content is made possible by a guest author, or sponsor; it is not written by and does not necessarily reflect the views of App Developer Magazine's editorial staff.
Become a subscriber of App Developer Magazine for just $5.99 a month and take advantage of all these perks.
MEMBERS GET ACCESS TO
- - Exclusive content from leaders in the industry
- - Q&A articles from industry leaders
- - Tips and tricks from the most successful developers weekly
- - Monthly issues, including all 90+ back-issues since 2012
- - Event discounts and early-bird signups
- - Gain insight from top achievers in the app store
- - Learn what tools to use, what SDK's to use, and more
Subscribe here
