Improve DevOps processes with API catalog

One of the biggest trends in DevOps is the “shift left” approach when it comes to security, so much so that security conferences now host developer days, developer conferences host security days, and the two have melded into DevSecOps. But pragmatically, how do you implement security earlier into your development cycles? According to CloudVector VP of Engineering and Threat Research Ravi Balupari, one of the most important tools for application security testing and quality assurance is an API catalog. In this Q&A, I sat down with Ravi to discuss API catalogs, and how developers can use them to improve their DevOps processes.

ADM: What is an API catalog? Why do you need one? Why should a developer care?

Balupari: An API catalog is a living document of every enterprise API that is consumed, exposed or otherwise integrated into its application stack, which includes the specifications of each API, which may include data type, parameters or access control. Maintaining an enterprise API catalog is considered a best practice for DevOps because of the benefits it imparts through the application development cycle. Comparing and contrasting an API catalog from development, testing and production environments can identify misaligned priorities, broken quality assurance processes, and incomplete or missing data.

ADM: What is the difference between an API catalog and an API specification or an API blueprint?

Balupari: Actually, there is very little difference between them, which is why there is a need for education and understanding among developers about this vocabulary. To me, an API catalog is a collection of information about APIs, an inventory of the APIs, and the blueprint for each API. There is also meta-information about the creator, the date of it was last modified, which application it belongs to, and so forth. An API blueprint and an API specification are exactly the same. An API blueprint or specification defines how an API is designed and should be used. 

ADM: How do you generate an API catalog? What are the common practices organizations adopt to maintain their API catalog? What can be done to improve it?

Balupari: At a high level, there are only two ways to create an API catalog, manually importing and documenting into a cataloging system, which may be an internal Wiki, a shared spreadsheet or an external solution like Mulesoft Anypoint Exchange. Some people may even be capturing this as a plaintext repository like GitHub or sharing on Slack, which we don’t recommend. So, the developers are capturing these blueprints/specs, including the parameters and so forth, back into the API catalog, in whichever form it exists. But then you also need to realize that every time you update your API, you also need to update its entry into the API catalog.

API gateways provide some storage of API specs, and some refer to discovery, but they are not automatically generating the API catalog. However, CloudVector recently launched API Shark, which is a free API observability tool, which does enable the automatic and continuous discovery of APIs to help generate an API catalog.

ADM: What sort of meta data should an API specification include?

Balupari: It typically captures information about the method (Get, Post,etc.), the endpoint URL, the parameters passed in the request, and the information of the data received in the response. The parameters are specified in detail and capture information such as this is required or optional, the data type, the bounds for the data type, the transport protocol mechanism, the authentication, and authorization mechanism, and a brief description about the data carried within the parameter.

ADM: How can you integrate API catalogs into your CI/CD process?

Balupari: The best practice is that developers should move their applications through dev, test and run. The risk of pushing to production without adequate testing is broken systems and downtime. Starting with dev provides your baseline. There is a prevalent way of thinking in security that you “trust, but verify” so even in DevOps, you should trust your API catalog in testing environments, but you still can’t assume it is the golden image. The recommended best practice is to verify the API catalog throughout the CI/CD process.

ADM: How can you use an API catalog in a dev environment?

Balupari: When you create your API catalog in a dev environment, this is considered your “golden image,” which becomes the basis. It is no wonder that so many developers refer to their API specs as API blueprints because they serve as the foundation of their application development.

ADM: How can you use an API catalog in a test environment?

Balupari: When you move into a test environment you introduce quality assurance, regression, performance and interaction use cases. An API catalog should be maintained up-to-date and compared against its development catalog. Contrasting these catalogs can help identify APIs that were listed in the dev catalog, but not the test catalog, which means that quality assurance missed testing that API. Likewise, if you find an API in the test catalog that was not in the dev catalog that means the dev catalog was incomplete and you cannot assume it was a “golden image.”

ADM: How can you use an API catalog in a production environment?

Balupari: When you move into a production environment, your API catalog should reflect real-world application access. Contrasting this production catalog against your dev and test catalogs can help expose additional issues and areas for improvement. For example, an API listed in the test catalog that is not in production could help identify an API that is not being used, which suggests that QA was not testing for the real-world use case--you could reassign your development cycles elsewhere. It can also be used to identify risk, as an API listed in production, but not in the test catalog has not gone through the required QA testing and application security processes--this is a Shadow API.

ADM: How can you use an API catalog for security and risk assessment?

Balupari: Security teams will appreciate DevOps for providing an API catalog because they can use it to assess risk and inform better application security testing. Additionally, identifying an API in the Production catalog, but not in the Test catalog can be useful in identifying threat actors searching for vulnerabilities to exploit an unpublished API.

About Ravi Balupari

Ravi Balupari is co-founder and vice president of engineering and threat research, where he leads the development of its API Threat Protection platform and the creation of special projects such as API Shark, the free API observability tool. Previously, Balupari was head of threat research at Netskope, and spent 10 years as a security researcher with McAfee.