Table of Contents
DevOps signifies close collaboration between the development and operations teams. Though the term is common among IT organizations today, it was an unchartered territory back in 2009. Flickr introduced the concept at OReilly Velocity with a presentation titled: “10+ Deploys Per Day: Dev and Ops Cooperation at Flickr.”
The interesting thing about this presentation was the two presenters — John Allspaw, the Vice President of Technical Operations, and Paul Hammond, the Director of Engineering. This was the first time that the people from development came together with those in operations to promote the core idea behind DevOps: Release Early, Release Often!
DevOps has grown leaps and bounds since then. Many businesses implementing DevOps solutions are nailing exemplary deploy frequencies/day and have become living examples of how its implementation can impact efficiency and productivity. Businesses well-known for implementing DevOps include Amazon with 23,000 deploys/day, Google with 5500 deploys/day, Netflix with 500 deploys/day, and the list goes on. That is the power of an entrenched DevOps culture!
Many businesses are following their lead and moving towards DevOps transformation. The Puppet’s State of DevOps Report is proof of the wide adoption of DevOps. According to the report:
- 90% of respondents with highly evolved DevOps practices report their team has automated most repetitive tasks.
- 97% of respondents with highly evolved DevOps practices agree that automation improves the quality of their work.
- 62% of organizations stuck in mid-evolution report high levels of automation.
This guide discusses DevOps from the ground up and how this evolution can help organizations scale up their software development and deployment processes. Let’s get started.
What is DevOps?
DevOps is a portmanteau for Development and Operations, which comprises practices, culture, and tools that help increase software development velocity to ensure faster time to market. The primary objective of DevOps is to break the silos that exist between the development and the operations teams.
DevOps has its roots entrenched in the Agile development methodology.
83 percent of IT decision-makers report their organizations are implementing DevOps practices to unlock higher business value through better quality software, faster delivery times, more secure systems, and the codification of principles.
— Puppet
For a clearer perspective, here is a breakdown of the responsibilities of the “development” and the “operations” team in the software development process:
| What Does Development in DevOps Mean? | What Does Ops in DevOps Mean? |
|---|---|
| The major objective of a development team is to convert ideas into a product that solves end-user’s problems. Here are some responsibilities they carry on their shoulders: | The major objective of the operations team is to deploy the product and manage it throughout the product’s life. Here are some responsibilities they carry on their shoulders: |
Requirements gathering |
Renting out server space |
Design |
Configuration of server and storage |
Development |
Managing outages |
Testing |
Managing backups and security |
Product backlog refinement |
Deployment and maintenance |
Ensuring continuous delivery |
DevOps vs Agile
Agile focuses on the development aspect of the software that covers the primary phases of the software development lifecycle (i.e., requirements analysis, design, code, test, deploy, and management).
The shortcoming of this approach was that it could not nicely integrate the operations team into the process. Thus, DevOps was introduced to fill the gap. In DevOps, development and operations work in tandem for on-time software delivery.
DevOps inherits the principles and values of the Agile Manifesto and extends the focus of software development to include automation and collaboration (between development and operations).
Refer to the following comparison chart for a better understanding of the differences between DevOps and Agile:
| Parameter | Agile | DevOps |
|---|---|---|
| Focus | Development and project management | Development and operations that includes integration, delivery, and deployment |
| Part of | Software/product development process | More of a cultural shift |
| Practices/ Frameworks |
Scrum Kanban DSDM Lean Crystal Feature-Driven Development |
Continuous integration Continuous delivery Continuous deployment |
| Characteristics |
Incremental and iterative development Focus on meeting the needs of the customers Development team productivity |
Cross-functional collaboration Automation CI/CD Pipelines |
Why DevOps?
DevOps addresses the shortcomings of Agile (i.e., the incremental and iterative development did not lead to the iterative and quick deployments).
Agile only focused on faster development, which was not enough. Faster deployment should have followed faster development to meet the customer demands promptly.
DevOps focuses on faster development, faster deployment and promotes collaboration between the development and operations teams.
DevOps = Development + Operations
DevOps = Communication + Collaboration
DevOps = Faster Development + Faster Deployment
The Three DevOps Principles
The following DevOps principles are inspired by “The Phoenix Project,” a fictional novel around IT and DevOps.
- Systems Thinking There needs to be a continuous and constant flow between the development and the operations teams. The key to achieving this flow is to implement a continuous integration/continuous delivery/continuous deployment (CI/CD) pipeline, limiting WIPs (work in progress), and introducing automation.
- Promote Feedback Loops Fast feedback is equal to quick learnings. The focus of the DevOps team should be to identify faults as soon as possible. The best practices for promoting quick feedback include — introducing automation test suites, employing tools for detecting a failure in the deployment pipelines, and monitoring continuously.
- Continuous Experimentation and Learning Continuous learning is a primary parameter that defines the success of DevOps initiatives. The best practices for fostering a continuous learning culture include — building trust among teams, effective communication and collaboration, and allowing Devs and Ops teams to work on non-functional requirements.
DevOps Framework: CALMS
CALMS is a DevOps framework introduced by Jez Humble, the co-author of “The DevOps Handbook.” CALMS stands for — Culture, Automation, Lean, Measurement, and Sharing.
This framework measures an organization’s ability to adapt to the DevOps culture and its implementation success.
Here’s what CALMS framework signifies:
- Culture DevOps is a cultural change rather than an adoption process. The major step towards this cultural change comes from communication and collaboration between the development and operations teams.
- Automation Automation aims to eliminate the burden of manual testing and repeatable tasks. To implement DevOps — automating the build, test, and deploy (i.e., the CI/CD pipelines) is the first step forward.
- Lean Lean is a management approach that aims to deliver value through eliminating waste and showcasing delivery speed. In context to DevOps, continuous improvement (in the form of retrospectives and A/B testing) and embracing failure are add-on lessons inspired by Lean practices.
- Measurement
Measurement of success forms the foundation for continuous improvement. When implementing DevOps, the team needs to measure:
- The lead time between development and deployment
- The frequency at which bugs are reported
- The turnaround time to recover from failure
- The retention/churn rate for the software
- Sharing Sharing roles and responsibilities is a deeply entrenched value of the DevOps culture. The basic idea behind sharing is to promote an organizational setup where Dev thinks like Ops and vice-versa. The pairing between the two teams should be such that they understand each other’s roles well and work collaboratively towards a common goal.
Benefits of DevOps
The primary advantages of DevOps at the organizational level include:
- Speed DevOps helps introduce automation to the integration, delivery, and deployment of the software. With the aid of the CI/CD pipelines, the software can be released to the production environment multiple times a day.
- Scalability Automation plays a significant role in managing the changing software requirements and ensuring that the system can be scaled efficiently and with speed.
- Promotes Cross-Functional Setups DevOps breaks the silos that exist between the development and the operations teams. When an organization makes this cultural shift, the two teams work closely and maintain ongoing communication to manage their shared responsibilities collaboratively.
- Reduced Failure Rate
Software changes can sometimes lead to failed integrations, thus causing compilation issues in the software. However, DevOps introduces automation to the development and deployment pipelines, wherein chances of change failure get significantly reduced.
Google’s DevOps report suggests that DevOps leads to a seven times lower change failure rate (i.e., changes are 1/7 less likely to fail).
- Offers Competitive Advantage
Continuous value delivery on a regular basis is imperative for any software’s success. DevOps leads to faster code deployments, which, in turn, helps improve the business’ value proposition and also offers a competitive advantage.
For instance, Amazon deploys code every 11.7 seconds with DevOps, which is a significant factor for it being a market player.
DevOps Practices
The commonly used practices that are categorized under DevOps include:
- Continuous Integration (CI)
As a part of continuous integration, the developers regularly put the code changes into a central repository. Automated tests are run on the code to identify bugs and other runtime errors.
Continuous integration aims to shorten the test loops so that the software changes can be updated without delays.
- Continuous Delivery (CD)
As a part of continuous delivery, the code changes are automatically built, tested, and prepared for deployment in the live environment. Developers are notified of bugs, and the code is automatically retested, and this loop continues until the code is ready for deployment.
The objective of continuous delivery is to validate that the code is bug-free and is ready to be pushed to a live environment.
- Continuous Deployment (CD)
As a part of continuous deployment, the new code changes are automatically released to a live environment. That is, the updates and upgrades are available to the end-users.
Continuous deployment aims to ensure that the code changes are deployed to a live environment quickly and securely.
- Microservices Architecture
Microservices architecture is a design approach to build applications in the form of loosely coupled, independently existing services. This defies the monolith approach to development, where a malfunction to a feature results in downtime of the entire application.
Microservices architecture is a powerful technology for developing applications with API capability. Netflix is a prominent example of a service that is built using a microservices architecture.
Microservices Architecture: Loosely coupled code blocks + Respective databases = Single application
- Infrastructure as a Code Infrastructure as a code is a practice where the infrastructure needs are managed and provisioned through code, not manual hardware configurations. In other words, engineers can interact with the infrastructure using application code similar to how they do with software.
- Version Control Version control refers to the practice of managing the code in patches or versions. This makes it easy to write and track code changes. Moreover, multiple developers can collaboratively work on a single version of the code, which helps create quality code through work division.
DevOps Lifecycle: Explained
The DevOps lifecycle is nearly similar to the Agile software development lifecycle, but with additional emphasis on the operations stage.
Here is an overview of each of these stages:
- Requirements Analysis
The functional requirements of the software are discussed and prioritized. The team creates a product backlog where they priority-arrange all the features of the software.
- Design
The design team creates the prototype that, in turn, helps visualize the UI/UX flow of the product. The focus of the design phase is to create a great user experience.
- Develop The developers write code at this stage to convert the idea into a workable product. The development team works in time-boxed sprints to develop the user stories on an incremental basis.
- Testing and Continuous Integration As a part of the DevOps process, testing is performed by leveraging automation. Continuous integration is also performed at this stage where the developers put their code into a central repository for running automated tests.
- Continuous Delivery The code is automatically built, tested, and prepared for deployment. The bugs (if any) are automatically reported to the development team, which then reworks the code and, again, puts it into the central repository.
- Continuous Deployment The code is pushed into the live environment consistently and reliably. That is, the product or the changes are made available to the end-users. Continuous deployment usually happens several times a day once the code passes from all approvals and quality checks gates.
- Operate The operate phase involves maintaining and continuous improvement of the code. This is done to fix bugs, improve performance, or introduce new features. The product needs to maintain a shipping cadence to offer continuous value to the customers. DevOps plays a significant role here as it aims to speed up the integration, delivery, and deployment process without compromising quality.
How to Build a DevOps Culture?
For implementing DevOps at the organizational level, introducing a cultural shift is the first step forward. This shift is about inculcating continuous collaboration and communication between the development and the operations teams.
The best practices to instill a DevOps culture include:
- Upskilling and Reskilling
To embrace the DevOps culture, upskilling and reskilling are highly essential. This practice includes training the existing development teams to adopt the DevOps best practices.
The key strategy to implement DevOps starts with communication and collaboration between the two teams, where they will share roles and responsibilities.
The set of shared roles and responsibilities in a DevOps setup include:- Joint planning for product development and infrastructure deployment
- Build a CI/CD pipeline
- Prioritizing automated testing to ensure timely bug fixtures, redeployment, and maximized delivery speed
- Building an automation-led culture to cut down on the traditional approval and sign-off practice
- Immediate response management when tickets are raised
- Shared Accountability
In a traditional setup, the development team is accountable for the code, whereas the operations team will be accountable for timely and seamless deployments. However, in a DevOps setup, shared accountability is given priority.
A developer is responsible for writing quality code, fixing bugs, and ensuring high code performance in the operate stage. On the other hand, the operation team is responsible for planning and ensuring all governance, security, and compliance policies are managed across the development stage. - Building Strong Belief in Retrospectives
Any DevOps team needs to be proactive when it comes to learning from failures. This is why running retrospectives should be a standard practice to follow.
A retrospective is a popular Agile practice where the development team arranges meetups to discuss past projects. The learnings are shared and recorded so that the team can avoid the past and deduce better ways to develop and operate. - Follow Lean Management Practices
Lean management practices aim to deliver value to the customers, eliminate waste, and promote continuous improvement. It does so by introducing a shared responsibility and culture at the organizational level.
The benefits of incorporating lean management practices include:- Increased focus on reducing release cycles
- Better team productivity
- Implementing smarter and speed-oriented processes
- Better utilization of resources
DevOps Implementation Challenges
Implementing DevOps can be challenging at first. Some of the common DevOps challenges include:
- Adapting to the new job roles
- Expenses related to training, tools, and other resources
- Automation process risk
- Scaling DevOps across teams and locations
- Keeping up with the DevOps protocols
- Identifying and eliminating the DevOps implementation bottlenecks
The DevOps Tech Stack
Leveraging the right tools is essential for DevOps success. The following table lists the commonly used DevOps tech stacks.
| Purpose | Tools |
|---|---|
| Project Management Tools |
Jira Kanban ClickUp |
| Source Code Management |
Git GitHub |
| Artifact Management |
JFrog Artifactory Nexus Artifactory |
| CI/CD Enablers |
Jenkins Tekton Pipelines CircleCI |
| Containerization |
Docker Red Hat OpenShift Amazon Elastic Kubernetes Service |
| Configuration Management |
Puppet Chef |
| Cloud Providers |
Amazon AWS Microsoft Azure Google Cloud Platform (GCP) |
| Monitoring Release Cycles |
New Relic One Sumo Logic Dynatrace |
DevOps Best Practices
The best practices to implement DevOps at full-scale include:
- Promote knowledge sharing, upskilling, and reskilling culture
- Choose the right stack of DevOps tools
- Ensure that the software meets the “architecturally significant requirements” (ASRs). The common parameters of ASRs include — deployability, testability, modifiability, and monitorability
- Emphasize efforts on test automation and CI/CD
- Break large tasks into smaller manageable tasks for trimming the release cycles
FAQs Related to DevOps
1. What is DevSecOps?
DevSecOps is a portmanteau for DevOps and Security. The DevSecOps aims to enable security across the DevOps environment using best practices, following standards, and leveraging the latest security-oriented technologies.
2. What are the KPIs for DevOps?
There are four key metrics to measure DevOps success. These metrics have been introduced by DORA and are described in the following bullet points.
- Lead Time for Changes: Measure of time spent between the code testing and code production
- Deployment Frequency: Measure of how often the code is pushed into production
- Time to Failure Recovery: Measure of the time it takes to restore and recover service after a failure has been detected
- Change Failure Rate: Measures the frequency of failures detected in the deployment pipeline
Conclusion
DevOps is a cultural shift that aims to reduce software development and deployment cycles. The implementation of DevOps at the organizational level helps break the silos between the development and operations teams.
To develop and deploy software and improve time-to-market, DevOps is a go-to practice. This write-up covers everything around DevOps and how it is essential to embrace the change where faster release cycles are a priority.
In the right words — Develop fast, ship faster!
