Solving the challenges of database DevOps

As companies look to the promises of DevOps to improve their applications' quality and delivery, obstacles still exist with relational databases that hinder adoption of this collaborative process. In this interview, Perry Krug, Principal Architect of Strategic Accounts at Couchbase, shares insight into why relational database management systems impede database DevOps, and offers solutions that can help simplify the complexities of database change management.

We sat down and had a chat with Perry Krug, Principal Architect at Couchbase focused on strategic customers, to learn more about DevOps team and the database challenges that they have.

ADM: Tell us about your perspective on the status of database DevOps. Is the adoption rate of applying DevOps principles to an organization's databases widespread or still in its infancy?

 
Krug: New technologies including the cloud, mobile and big data have created vast opportunities in the realm of customer experiences. As a result of rising customer expectations, customer experience is now a competitive advantage that is at the forefront of C-level concerns. In order to provide the best customer experiences, businesses must be able to leverage data to innovate faster and more efficiently. 
 
DevOps adoption is growing rapidly, but is still in its infancy. As customer experience becomes more focused on interaction and engagement, databases need to be built for handling more than simple transactions. Additionally, customer needs are always changing and the volume of data businesses are managing is continually increasing. Therefore, there's a growing need for database changes to be as simple as code changes.
 
The ideal model is for database administrators to be an integral part of both Development and Operations teams. DevOps is about integrating the technical advances of agile development such as continuous integration, automation and version control into an operations context, at the same time as retaining the rigor and discipline of the lights-on mentality.
 
The new enterprise technology architecture needs to be far more agile than ever before, and requires an approach to real time data management that can accommodate unprecedented levels of scale, speed, and data variability. Relational databases are unable to meet these new requirements, and enterprises are therefore turning to NoSQL database technology.
 

ADM: What are the benefits for organizations in using the DevOps approach to their database development and management?

 
Krug: Database administrators should no longer have to write migration scripts by hand or keep an audit of which scripts have been run and which ones have not. They should be able to make whatever changes they choose to a development database and then "check it in" like with code.
 
Increasing the speed of delivery of database changes and freeing developers up to do more value added work are the key drivers for automating the delivery of database changes. As processes become automated, developers have more time to focus on more creative processes that will help build more engaging and responsive applications.
 

ADM: What do you see as the key barriers to adoption of a DevOps approach to databases?

 
Krug: One of the greatest challenges with integrating database changes into a DevOps process is synchronizing application and database changes. Another major drawback identified with traditional siloed database development is the increased risk of failed deployments or downtime when introducing changes. This is closely followed by slow development and release cycles and the inability to respond quickly to changing business requirements.

ADM: Do the challenges of database DevOps apply to SQL server only or are these issues prevalent with other relational databases?

Krug: These issues are certainly not unique to SQL Server and exist across many if not all relational databases.  The challenges have less to do with a particular technology, and more to do with a mismatch between what relational databases were designed for and what they are being stretched into.  When relational technology was in its infancy, there were only a few hundred  users accessing monolithic applications that were expected to change once or twice a year.  Fast forward to today where even the most traditional organizations have consumer-facing applications. These apps reach thousands if not millions of users who expect the same speed and efficiency of major applications such as  Facebook or Google or Amazon.  While large scale is certainly a reality for many applications, the more challenging requirement is to constantly evolve and improve that application...in some cases many times a day! Relational technology simply wasn't designed with these needs in mind and so a new breed of database is needed.  In the same breath, however, relational technology is very good at what it does, simplified transactions, and so will continue to have a place in many organizations, often alongside NoSQL and newer technologies. Application developers should be choosing the right tool for the specific job.
 

ADM: Tell us more about why relational databases are not well suited for a DevOps approach. Why do RDBMs impede DevOps?

Krug: Relational databases were born in the era of mainframes and business applications - long before the Internet, the cloud, big data and mobile. They were built in a time when businesses were utilizing the technology for primarily transactional purposes so they are not built for rapid change or innovation. These databases were engineered to run on a single server - the bigger, the better. The only way to increase the capacity of these databases was to upgrade the servers - processors, memory, and storage - to scale up.

A core principle of agile development is adapting to evolving application requirements: when the requirements change, the data model also changes. This is a problem for relational databases because the data model is fixed and defined by a static schema. So in order to change the data model, developers have to modify the schema, or worse, request a "schema change" from the database administrators. This slows down or stops development, not only because it is a manual, time-consuming process, but  also because it impacts other applications and services. Databases that support web, mobile and IoT applications must be able to operate at any scale. While it is possible to scale a relational database, doing so is typically complex, expensive and not reliable.
 

ADM: Why is NoSQL a better choice for database DevOps?

Krug: As enterprises embrace digital transformation - enabled by cloud, mobile and big data technologies - developers and operation teams must build and maintain web, mobile and Internet of Things (IoT) applications more quickly and at scale. NoSQL is increasingly the preferred database technology to power today's modern applications.

A NoSQL database supports agile development, because it is schema-less and does not statically define how data must be modeled. Instead, it defers to the applications and services, and thus to the developers as to how data should be modeled.
 
NoSQL databases allow developers to shorten production processes and scale processes more efficiently.

Notable NoSQL database abilities include:

• Reading and writing data formatted in JSON - which is the de facto standard for consuming and producing data for web, mobile, and IoT applications
• Deploying no more hardware than is required to meet the current load
• Leveraging less expensive hardware and/or cloud infrastructure
• Scaling on-demand and without downtime

 
NoSQL capabilities allow developers to provide an engagement database, which allows developers to focus their time and efforts on how to utilize existing data sets to address customers' interests and needs. The ability to exploit change allows organizations to build richer and ongoing engagements with their customers.

ADM: How does NoSQL simplify the complexities of database change management and address the other challenges you previously mentioned?

 
Krug: NoSQL simplifies database DevOps with:
 
- Schema flexibility: Developers need a single database that can store rapidly changing structured, semi-structured and unstructured data. NoSQL document database offers schema flexibility by allowing developers to operate directly on JSON data and derive meaning out of it.

- No impedance mismatch: With no object-relational mapping for the application, there is no impedance mismatch between domain classes and database structure. Only the application code needs to be updated and no coordination is required with the schema changes.

- Scalability:  One of the drawbacks of relational databases is the inability to adapt to changing business requirements. This highlights scalability as a major DevOps challenge. If the volume of data, the number of queries, or the types of indexes required to support the application changes the database needs to change to accommodate those changes. Not in weeks or months, but today! NoSQL databases run on commodity hardware and have a scale-out architecture as opposed to scale-up with RDBMS. Sharding can help with scalability in RDBMS but that's an extra complexity that now needs to be dealt with. In the era of increasing digital connectivity, applications need to be able to rapidly scale to support high user activity. NoSQL ensures each user experiences seamless performance and reliability.

About Perry Krug

Perry Krug is a Principal Architect at Couchbase focused on strategic customers. He has been with Couchbase for over 6 years and has been working with high-performance caching and database systems for over 10 years.