What is DevOps? A Tutorial and Training. DevOps Explained

In this session today, we will review DevOps, key practices and tools, and steps that one can take to transition to a DevOps environment. Before, I begin, please ensure to subscribe to my YouTube channel and you will be notified of more learning videos on the latest topics and trends in the area of digital and cloud computing.

Definition of DevOps

So, first let’s get to the definition of DevOps. What is DevOps? Let’s review the concept from a number of dimensions.

• DevOps refers to a collection of practices and a general philosophy in the area of software development, the overall goal of which is to constantly deliver and deploy high quality software at high velocities. So, it’s not one practice or a methodology but usually a collection of various practices and methodologies.
• DevOps refers to the concept where software developers and Ops staff collaborate throughout the software development and deployment lifecycle to ensure the delivery of quality code to production. So, it eliminates the silo mentality and the finger pointing that has existed in IT environments for the past many years. This elimination of silos of software development and operations has allowed software developers to understand the complications inherent in running software that they develop in the operations environment making them sensitive to stability and reliability issues that are important for productions and operations environments. Likewise, operations staff and engineers and system administrators understand the complexities of the software build process making them less critical of “those software developers.” So, a DevOps culture thus instills teamwork to solve issues, empowers teams to take critical decisions – all the while keeping teams focused on the ultimate business outcome, which is to deploy and run quality software that would delight customers. So, as DevOps brings development and operations teams together, the scope of DevOps usually encompasses both software development and infrastructure management processes.
• In an ideal environment, DevOps practices integrate with Agile Software Development Methodologies. Here, we need to understand the relationship between Agile software development and DevOps. To put it simply, Agile software development enables production of quality software quickly. However, even if that’s done there is no way to ensure the integration, testing and deployment of that code rapidly. That’s where DevOps practices take over. So, Agile Software Development approaches together with DevOps enable a fast and rapid software design, development, testing, and deployment of quality software products to production that in turn can have a direct bearing on customer experience and satisfaction.
• DevOps is also synonymous with the many tools that are used in software delivery and deployment because a key tenet of DevOps has to do with the automation of the various pipelines that integrate software development, testing, deployment in production and monitoring. We will cover some of these tools and their functionality a little later in this presentation.

 

Business Benefits of DevOps

Next, let’s review the overall business benefits of instituting DevOps principles and practices. Most organizations who have successfully implemented DevOps report the following:

• An accelerated delivery and deployment process – As DevOps brings down the silos between organizations by getting teams in software development and production or operations to collaborate closely throughout the software development and deployment lifecycle, organizations observe a considerable increase in the velocity of quality software development to production. This helps the organization to serve its customers faster and to innovate and test its innovations at a rapid pace. This is a departure from the traditional practices where production staff would become a bottleneck in the deployment of developed software to production to ensure stability of operations. However, as teams in DevOps work as part of one team throughout the overall process, any issues related to software development or operations are addressed during the overall development lifecycle facilitating a fast delivery to production and operations.
• Higher frequencies of software releases – DevOps practices such as CI and CD along with automation of the pipelines from software development to deployment enable organizations to release software constantly to production or at a minimum get it ready for deployment. Depending on the size of the organizations and the scale of their software development, many have reported that their releases have gone up by 50 to 100 times. For example, in one of the latest DevOps conferences, Netflix reported to have been doing thousands of releases on a daily basis. That’s an astounding increase from earlier practices which allowed for much fewer number of releases per week.
• Automation of repetitive tasks – As we discussed in the earlier point, such a high increase in the frequency of software releases isn’t possible unless various facets of the overall software delivery and deployment pipeline are fully automated. The various steps in this whole pipeline can include code development, integration, testing, security, validation, deployment, monitoring, etc. a number of which can be automated in an organization that institutes DevOps.
• Stability, security and reliability of deployed software – Automation can ensure that various policies and best practices are reflected in code or scripts minimizing human errors and thus increasing compliance with an organization’s policies thus ensuring stability, security and reliability of a production and operations environment.
• Better predictability of software release cycles – Due to the automation of the various facets of the overall lifecycle, the organization expects to get better predictability on when certain business functionality can be deployed into production and thus can plan accordingly.
• Fewer errors in delivered and deployed code – Automation and the constant practices of testing and integration ensure that developed code has fewer errors when run in production.

 

Popular DevOps Practices

Now we will cover some of the popular DevOps practices. Although there a number of DevOps practices to help organizations achieve the overall goal of rapidly delivering quality software to production and to enable its automated monitoring, in the following we will cover the 4 popular practices that are at the foundation of any organization using DevOps.

Continuous integration refers to the practice of developers integrating their newly developed or modified code with the code baseline checked in by others continuously. The keyword here is ‘continuous’ and that’s to ensure that any defects that may surface during integration are surfaced as early as possible rather than waiting to integrate later in the process. So, as multiple developers produce and update code, they are constantly integrating with the main baseline to prevent discovering larger integration problems later. This practice, therefore, removes potential hurdles from the process and speeds delivery of software to the production environment.

Continuous delivery ensures that software is constantly readied for release by automating the steps highlighted in continuous integration along with other steps of unit testing, load testing, integration testing, API reliability testing, etc. This helps developers discover any issues pre-emptively rather than discovering at a later stage. Whether that actually gets released to production or not depends on other factors including prioritization by the product owner of the various business functions. The deployment to production therefore waits for a manual approval trigger. Through the practice of continuous delivery, high quality software is ready to be deployed quickly to production reducing the risk of suboptimal code being released to production while ensuring speed and fast time to market.

Continuous deployment is similar to the continuous delivery process except that delivery is automated all the way to production and not merely to a staging environment. In general, unless you are confident of auto deployment, this practice is not recommended. Usually, someone does a final manual check of other dependencies before code is deployed in production. Most organizations prefer to take the process all the way to continuous delivery and then wait for a manual check and validation of other dependencies. However, depending on your business situation and process and environment maturity, you can consider the instituting of this practice with care.

Infrastructure as code facilitates configuration of infrastructure components such as servers through code. In traditional environments, software developers manually provision and configure servers and apply patches to the multiple servers in various environments such as dev, test, pre-production, and production. As technological advances in cloud computing have allowed engineers to interface with infrastructure through APIs and code, engineers can provision and configure servers using software thus simplifying and accelerating the entire process.

 

So, when we look at these practices of team collaboration, and lean processes that fuse development, testing, deployment and monitoring processes, we see that these practices have come from the Agile and Lean approaches that started a few years ago. So, if you are an IT and technology executive, this should provide you some cues to ensure that regardless of your IT maturity, ensure that your teams understand the fundamentals of Agile and Lean as that knowledge can help your teams to formulate your organization’s processes.

DevOps Tools

Now we will look at DevOps tools. These tools span a number of areas including the building and compiling of software, testing, configuration management, application deployment, monitoring, version control, and others. Other tools are used in the areas of continuous deployment, continuous delivery, and continuous deployment. These tools together with the emergence of virtualization allows organizations to deploy digital services quickly to the business.

Basically, within DevOps, the tools are needed for a wide variety of activities and functions some of which are the following:

1. Building and provisioning of servers
2. Virtual infrastructure provisioning – This refers to using APIs and other tools to help you provision other parts of the infrastructure either in your cloud environment or public cloud environments such as Amazon’s AWS and others.
3. Building code – These tools compile code into executable components.
4. Maintain source code repositories
5. Configuration management – These tools facilitate configuration of development environment and servers.
6. Testing and automation –
7. Version control – This ensures that all code history is maintained in the repositories using version numbers. With numerous developers checking in and checking out code, tools can help track previous histories using automated versioning. Also, if there is ever a need to revert back to the previous version of functioning software in a production environment, these tools allow that very easily.
8. Pipeline orchestration – These tools orchestrate the entire process from the time software is ready for deployment all the way to deployment. There are other tools that provide complete visibility from the beginning to the end.
9. And so on.

 

Over the past few years, a number of tools have surfaced with multiple features that are too numerous to mention but here we will cover some with their primary features. With time, these tools mature and incorporate additional functionality. Here is an overview of some of those tools.

• Jenkins – It’s one of the most common and popular tool and addresses various facets of both Continuous integration and continuous delivery.
• Vagrant helps DevOps teams create and configure lightweight, development environments. This falls under the Infrastructure as Code and essentially lets developers create a single file for projects where they can describe the type of machine they want, the software that needs to be installed, and access rules for the machine. Vagrant then uses that to provision development environments.
• Splunk – This tool provides operational intelligence to the teams and is based on data analytics.
• Nagios – Monitors the infrastructure components such as applications, services, operating systems, network protocols, system metrics, and network infrastructure.
• Chef is another popular tool that turns infrastructure into code so that users easily and quickly can adapt to changing business needs.
• Docker – This is an open integrated tool that allows DevOps teams to build, ship, and run distributed containerized applications
• Artifactory is a universal code repository manager that supports software packages created in any language or technology.
• JIRA – This is one of the very popular tools used by Agile teams. This tool is used by DevOps teams for issue and project tracking.
• ProductionMap is another popular tool with advanced orchestrator and development features. This tool enables teams to develop and execute complex automation on a large scale of servers and hybrid technologies.
• Ansible is a DevOps tool for automating your entire application lifecycle. Ansible is designed for collaboration and makes it much easier for DevOps teams to scale automation, manage complex deployments, and speed productivity.

 

If you work with public cloud frameworks such as Amazon’s AWS or Micrsoft’s Azure, then they you will have to integrate with their specific tools and solutions. For example, in the AWS world, you have access to the following: There are others as well but we will cover the key ones here.

• AWS CodePipeline – AWS CodePipeline addresses both the continuous integration and continuous delivery practices and when configured properly according to your workflows, allow a complete smooth DevOps pipeline.
• AWS CodeBuild – This tool as is obvious from the name is used for building software and checking in repositories and to perform testing. It also ensures that one doesn’t have to worry about server provisioning, etc. as they are taking care of in the background.
• AWS CodeDeploy – This AWS service allows the deployment of code to production to AWS server instances.

 

DevOps Transitioning

Finally, we will discuss some of the steps that organizations can take to institute DevOps in their environments.

• Start Small – First start small. Start with a small project and get it through a continuous integration and delivery pipeline and then to deployment. So, essentially get your teams to understand the technicalities of instituting a DevOps pipeline from planning and development of code to deployment, monitoring and collecting feedback.
• Focus on the cultural aspects – Also, in parallel start to focus on the cultural aspects. That’s very important. DevOps is not merely about getting a bunch of tools and making them work. If the cultural aspects that we discussed earlier about collaboration, bringing down siloes, etc. are not taken care of, then the effort won’t yield fruitful results.
• Define a workflow specific to your environment – Next, as you start to mature gradually and start to piece together various tools to support the development and deployment processes in your environment, define a specific workflow or workflows that would be appropriate for your software environment. So, for example, you may have multiple and hybrid development environments related to Docker, Kubernetes, legacy applications, working in public cloud environments, and more. Ensure that your defined workflows will work to support all those scenarios.
• Select the right tools to define your workflow – Depending on the workflows that you define, you will need to then ensure that you pick the right tools that integrate tightly to form an integrated DevOps pipeline. That is essential to ensure maximum automation that at the end will help you achieve high velocity development, delivery, and deployment.
• Establish business level metrics and measure maturity over time – Finally, ensure that you institute the right metrics to ensure that you can measure your organization’s maturity over time in terms of delivering more software, quality of deployed software, and so on.

 

With this we come to the conclusion of this presentation. To learn more on other topics, ensure to subscribe to my YouTube channel where I post a number of best practices related to the area of digital transformation.