A Pipeline of Tasks Illustrating a Simple DevOps Chain
DevOps is the amalgalm of development activities and the business continuity provided by operations teams. It unites the efforts of what are still two teams, with different competencies, into one value chain. DevOps creates and sustains a unified process for transforming technical requirements into live code in production systems. In this article I walk through a very simple example to illustrate the key concepts and actors involved in the value chain, and how they are linked. The diagram that follows (below) provides a quick high-level view. This article is an adaptation for Jenkins based on a similar post for Maven. Many examples for DevOps are based on Maven. If you have little Maven experience or find yourself in an environment where Ant is already in use, this article will help you get a simple DevOps started quickly.
In the sections that follow, I will be talking about each of the pools of activity in the diagram above.Creating and Saving Artefacts
The first step of course is to create the artefacts that will power our target systems, whether DEV, TEST, UAT, or PROD. Software engineers are the primary actors in this scope. The term, software engineer, is used here to include developers, testers, DBAs and operations personnel. The term, resources, refers to any code, script, list, sheet, etc. that will either implement, support, or validate our target systems. Software engineers will create resources on their local machines and by saving them to a repository, trigger a chain of events that will take some of those resources into production systems.
Continuous Integration (CI)
As with the repository, there are many options; I used Jenkins. Jenkins is used to periodically validate the status quo (code base). It does this by a sequential process, executed on a schedule:
- Check out all updated artefacts
- Compile and/or validate the artefacts
- Package the compiled artefacts
The schedules for each of these steps differ, but there is a dependency from 3 to 2 and from 2 to 1. If step 1 fails, the others are aborted, and if step 2 fails, step 3 is aborted. The checkout is run every 15 minutes, the compile is run hourly and the package is run every other hour. Each of these three steps are executed for the artefact groups used: MDS, OSB, BPEL. If any of the jobs fail, the Jenkins administrator is notified by email with details about the job and optionally the log output.
Apart from the integration with Subversion via a plugin, the Jenkins service relies heavily on an Ant build file for running each of these jobs. The flow of data/control for each job is as follows:
- Jenkins job is started (manually or on schedule)
- Jenkins invokes Shell script
- Shell script invokes Ant
- Ant executes a task/target
Three separate Ant build files for each of the artefact groups: MDS, OSB, BPEL. This is for ease of maintenance, and because the names of the targets/tasks are the same: compile, clean, package, etc. However, all the build files share the same property file since many of the properties are common to all builds.
Configuration Management
Once the artefacts have been packaged, they are ready for deployment. Once again, there are a number of options. We could use Chef, Ansible, Jenkins, Puppet, Bamboo, etc.. Now, if you have to build the machine before you can deploy you are left with no choice but to use proper CD tools like Ansible, Chef, or Puppet. In this case the platform was already available and so I used Jenkins and Ant once again. A Jenkins job was created for deploying to target environments. The deploy job should be scheduled for DEV (every morning) but manual for all other environments. In addition, there should be offline governance controls over the manual deployment to non-DEV environments.
Smoke Testing
Once the artefacts are successfully deployed to the target environment, it is useful to test the services. There are quite a few options available for testing deployed services, including JMeter, SOAP-UI, etc. The testing could be carried out manually, managed offline, or integrated with Jenkins.
The automation of these essential tasks help to free developers from the bother of checking systems and code for breakages. Instead, the Jenkins services continuously build and check the source code to ensure that the baseline is stable.
(Link to archive with Jenkins, Ant and shell resources: jenkins-ant archive)
—
Oyewole, Olanrewaju J (Mr.)
Internet Technologies Ltd.
lanre@net-technologies.com
www.net-technologies.com
Mobile: +44 793 920 3120
