What is Continuous Integration?
Continuous Integration, as well as Continuous Delivery and Continuous Deployment (CI/CD), are concepts widely used in modern software development. It is a practice of automating the code quality checks in a project with multiple contributors, making it easier to identify and solve problems like code conflicts and regressions. Continuous Integration ensures that the newly added code doesn't break any tests by automatically triggering the full test suite, generating automatic builds when code is merged to a certain branch (i.e. production), and/or automatically deploying recent builds.
Luckily there are many tools out there that help implement CI/CD processes, offering a variety of different features for different types of projects.
CI/CD pipeline process:
It’s very important to understand the Pipeline process, which is a sequence of stages, with each stage described by a series of steps required to complete the stage.
This is a general pipeline to be followed on a standard CI/CD process:
VCS (git) push → Quality Checks (run tests and solve code conflicts) → Build app (bundle app into a zip) → Deployment to staging → Tests in staging → Deployment to production → Tests in Production
CI/CD tools categories:
Self-Hosted:
These tools are installed on-premises, meaning they are running on your workstation, datacenter, or in any part of your infrastructure. With self-hosted options, your team is responsible for installing and maintaining the tool, which may require more more complexity. However, this alternative is beneficial if you want to make special customization or if you need increased security to meet compliance standards (i.e. HIPAA compliance data).
Jenkins - This is one of the most popular tools for CI/CD since it is a free, open source solution. Jenkins is an automation framework that can be extended with over 1500 available plugins. It contains a Jenkinsfile based on the Groovy programming language, which describes the stages of a pipeline and actions needed to complete the stage. It also provides nice pipeline visualization via plugins like Blue Ocean that makes it easier to see the stages and their states.
Bamboo - Offered by Atlassian, the company responsible for developing Jira and bitbucket, this option is ideal for teams already using Jira or bitbucket given that Bamboo offers integration with those tools. This is a self-hosted product, and pricing is dependent on the number of jobs and remote agents. Bamboo pipeline can be configured using the web interface or with Spec files (Supporting yaml and java formats).
TeamCity - This option is offered by Jetbrains, the company responsible for products like Intellij and PyCharm. TeamCity offers integrations with IDEs developed by JetBrains and other editors like Visual Studio, allowing developers to trigger builds or analyze build results without leaving their IDE. TeamCity offers Free and Enterprise license options, and can be configured through the web interface or using config files (XML or Kotlin format)
| Name | Price | Parent Company | Config File | Notes |
|---|---|---|---|---|
| Jenkins | Free / Open Source | Cloud Bees | Via Web interface or jenkins file (Groovy language) | Plugin based (open source plugins) |
| Bamboo | Cost varies from $10 - thousands based on numbers of jobs and remote servers. | Atlassian | Via Web interface or spec files (YAML and java) | Plugin based (Free & Paid options) |
| TeamCity | Free and Enterprise licencing (based on number of agents) | JetBrains | Via Web interface or config files (XML/Kotlin) | You can trigger builds without leaving your IDE |
Software as a Service (Saas)
With SaaS tools, the vendor provides and maintains the tool, decreasing the team’s workload. This option is particularly useful for small companies that want to focus on their core business needs. There are many low cost or free tools available to manage your project, making it very easy to start. However, this option can become expensive at a larger scale.
Travis CI - This option is very popular among open source projects given that Travis CI offers free continuous integration services for projects hosted in GitHub. Travis CI also offers enterprise plans, but is always free for open source projects. Travis CI uses a yml configuration file called .travis.yml, allowing the configuration to be version controlled and flexible. Travis CI allows you to start services and connect to them similar to MySQL.
CircleCI - CircleCI uses a variety of Docker containers or virtual machines to run the build steps, and offers free performance and scale plans depending on the project needs. For mobile developers, it provides images with tools to build and test Android applications. CircleCI workflows are configured in a yml file called config.yml inside the .circleci directory and provides a useful command line tool to test configurations, run jobs locally, etc.
CodeShip - CodeShip offers two versions: Basic and Pro. The Basic version is easy to set-up using the build instructions through the web interface. The builds run on shared VMs with Docker containers pre-configured for common CI tasks and programming languages. With Codeship Pro, the build instructions are stored in a file named codeship.yml, and builds run on dedicated VMs. Both options can connect with popular repositories like GitHub, Bitbucket and GitLab.
| Name | Price | Config File | Notes |
|---|---|---|---|
| Travis CI | Starts with 100 free builds and continues to be free for open source projects. | .travis.yml | Allows integration and System testing.Only works with projects hosted on GitHub |
| CodeShip | Basic and Pro Licensing Basic: free up to 100 builds/month Pro: based on VM size and number of concurrent builds | Basic version - setup via web UI / Pro version - codeship.yml | Easy to use web interface with plenty of technologies to choose from. |
| Circle CI | Free version - One job, one container Paid Version - macOS development and increased job concurrency. | config.yml | Allows Orbs - Shareable configuration packages |
Cloud Service Providers
Big companies like Google, Microsoft, and Amazon offer cloud computing tools that make it very easy to start, maintain, and scale up or down based on demand, with improved availability, disaster recovery, and better security and reliability.
AWS CodePipeline: This is where developers define each stage of their pipeline and the actions that need to be taken on each stage. CodePipeline takes imports from Github, AWS CodeCommit, Amazon ECR, Amazon S3, etc. Each stage can call action providers like AWS services, third party actions or developers can define custom actions.
AWS CodeBuild: Working as an action provider for CodePipeline, AWS CodeBuild provides on-demand build services that can be used by CodePipeline to complete the builds required by the pipeline process. Uses a file named buildspec.yml where the build steps are defined (phases).
Azure Pipelines: Part of Azure’s DevOps product line, Azure Pipelines offers a complete set of hosted tools for application development with pipelines to build, test and deploy applications. One of the main advantages offered by Cloud Service Providers is the access to other resources in the cloud; Azure pipelines can interact with virtual machines, Kubernetes clusters or serverless applications. Azure Pipelines allow developers to build projects on a variety of platforms like Ubuntu Linux, MacOS and Windows, all hosted in the Azure cloud. Azure Pipelines control jobs with resources known as agents and tasks, and each project is configured by a YAML file located in the root directory named azure-pipelines.yml.
Google Cloud Platform Cloud Build: Provides a way for developers to build custom workflows to test applications and then deploy them to the Google Cloud Platform. Triggers can be added to the code repository to trigger the build when a commit is added. There are two options for the configuration file: cloudbuild.yaml or cloudbuild.json. The Yaml configuration file contains the description of all the steps that will be included in the build. Build steps are provided by Google and community contributions and developers can also create their own custom build steps.
| Name | Price | Configuration File | Notes |
|---|---|---|---|
| Amazon Web Services | No upfront cost, only pay for what you use, free 30 days after a pipeline is created | buildspec.yml | Two parts - CodePipeline and CodeBuild |
| Azure | Open source projects have unlimited build time and 10 parallel jobs for free | azure-pipelines.yml | Uses standard agents and task, standard and custom.Test matrices can be defined. |
| Google Cloud Platform (GPC) Cloud Build | Standard builds are given 120 free minutes each year and billed $0.003 per minute after that | cloudbuild.yaml / cloudbuild.json | Configuration file defined in steps. You can run multiple commands per build step. |
Conclusion
There are many options available to make the development process easy and reliable. The important thing is to fully understand the project and how we forecast its growth in the future.
Many platforms offer free or low cost options at the beginning, with the opportunity to expand in the future. It is important to conduct a cost/benefit analysis that helps prioritize goals and assess the short and long-term goals.
Header Photo by Ivan Slade on Unsplash
