Codeship: Deployment Pipelines and ParallelCI for Faster Results
Codeship is a hosted continuous delivery service that focuses on speed, reliability and simplicity. In this tutorial, I'll review the basics of Codeship from part one and then walk through deploying your code.
First, I'll show how to use Codeship to deploy to AWS CodeDeploy. Then, we'll explore Codeship's new ParallelCI, which speeds your tests and builds by delegating tasks in parallel to different virtual machines. I'll also introduce the Codeship API, useful if you'd like more automated control over Codeship's services.
Let's begin with a review of what Codeship does.
With Codeship, you configure the service to build, test and deploy your application from your source code manager to the staging and production platforms of your choice. The service offers a variety of powerful setup options. Currently, Codeship is integrated with popular source code managers GitHub and Bitbucket. It supports a variety of programming languages (Ruby on Rails, Node.js, PHP, Java, Go, Dart, etc.) and deployment platforms (Amazon Web Services, Heroku, Google App Engine, etc.)
The practice of continuous integration involves repeated automated testing and code merging with notifications to alert developers to problems in their code in real time. Codeship combines this with the practice of continuous delivery, which provides for regular code deployment once changes have passed automated tests.
Essentially, when you push your code to GitHub, Codeship rebuilds your application on its secure servers and runs your automated tests. If the tests fail, it notifies your development team via email or integration with popular services such as Slack, HipChat, etc. If your tests pass, Codeship follows your defined pipeline to move code to staging or production servers as specified.
Working with a service like Codeship can minimize fallout from bugs, speed their detection, and make your team more comfortable and proficient in releasing regular incremental improvements to your platform—hopefully improving your responsiveness to your customers and your market.
Codeship helps your team write and deploy their code with increased confidence.
Recently, Codeship announced an exciting new feature to speed your tests. It's called ParallelCI.
ParallelCI allows you to split your test commands across multiple build virtual machines (VMs) to speed up your build time. By running your tests in parallel, Codeship increases your team's productivity with faster builds. You can use ParallelCI to configure your project with up to ten parallel pipelines to gain as much as 10x speed increase for your test suite.
Configuring Your Deployment
Now, let's begin to explore Codeship's deployment features in more detail.
You choose what Codeship does with your code after your tests pass. For example, perhaps you update your staging server for further tests, or perhaps you roll everything out to your production servers. Codeship allows you to define sophisticated pipelines for testing and deployment.
By the way, Codeship also offers this free and very informative eBook: Efficiency in Development Workflows. You may wish to check it out along with the Codeship Blog, which keeps you up to date with new developments and best practices.
The Project Build Log
I'm assuming that you followed our first tutorial and have signed up and begun using Codeship.
If you visit your project's build log, you'll see a list of recent results. Click on the green Success button.
This will bring up the build result details and the Build options sidebar menu.
Here's the Build options menu. Click on Review Test Settings:
Here's you'll see the Project Settings sidebar menu on the left, which is where we'll find the Deployment options we need.
Here's a detailed view of the Project Settings menu. Click on Deployment:
Configure Your Deployments
Codeship provides ready-built deployment integration with many popular hosting solutions including Google AppEngine, Amazon Web Services CodeDeploy, Heroku, Engine Yard, Nodejitsu and more. Codeship is an official partner of AWS and CodeDeploy.
I'm going to walk you through two examples. First, I'll deploy my earlier PHP-based Yii2 Framework build to AWS with CodeDeploy. Then, I'll build the Ruby Discourse Project using ParallelCI to speed my build times.
Using Amazon CodeDeploy With Codeship
First, let's take a walk through Amazon AWS CodeDeploy:
I'll provide a basic walkthrough of AWS CodeDeploy, but my goal isn't to detail it here. Instead, my focus is to help you configure the Codeship connection to AWS CodeDeploy.
First, visit the AWS CodeDeploy Getting Started page and choose a Custom Deployment:
Then, create a new application. I've already created an EC2 instance and tagged it CodeDeployDemo.
Choose a Deployment Configuration. I've already set up a Service Role as well:
Now that your AWS CodeDeploy Application is set up, you need to configure your Deploy group for your instance.
This will differ depending on your application and S3 bucket location:
Then, let's return to Codeship. Enter your project and click Project Settings in the upper right. Then click Deployment:
Then, click the CodeDeploy button:
Fill in the AWS Code Deployment property settings with your Amazon access keys, secret key, and the CodeDeploy App Name and Deployment Group. Specify your S3 Bucket as well:
The next time that you push a commit, Codeship will build your app, test it, and initiate an AWS CodeDeploy operation.
Codeship also offers a number of video guides on their Vimeo channel that may interest you. For example:
- Deploying Ruby on Rails to Heroku
- Deploying a Node.js Application to Nodejitsu
- Deploying a Django Application to Google AppEngine
Here they are in order. Then we'll dive into using ParallelCI to speed our tests.
Deploying Ruby on Rails to Heroku
Here's a short guide to deploy a Ruby on Rails app from GitHub to Heroku, and the video is below:
Deploying a Node.js Application to Nodejitsu
Here's a short guide on How to deploy a Node.js app from GitHub to Nodejitsu, and the video is below:
Deploying a Django Application to Google AppEngine
Here's a short guide on How to set up Continuous Integration and Continuous Deployment for a Django Application from GitHub to App Engine, and the video is below:
Using ParallelCI With Codeship
I'm going to create a fork of the popular community discussion platform Discourse for Ruby on Rails. Then, I'll add a project at Codeship for my forked copy of Discourse.
Click on Environment and configure the environment variables for our Rails project:
Then we'll use the following setup commands to set up the build instructions:
bundle install --without development --deployment --retry=3 --jobs=3 bundle exec rake db:migrate
Then I'll push an example change to the ReadMe file to initiate a Codeship build.
Here are the results from a successful build:
Testing With a Single Pipeline
Now let's add the following tests to a single pipeline.
bundle exec rake qunit:test bundle exec rake plugin:spec bundle exec rspec --profile
The next time we commit, Codeship builds the code and runs the test. With one pipeline, it takes nearly ten minutes.
Now, let's switch to ParallelCI.
Testing With Multiple Pipelines
With ParallelCI, each project has multiple test pipelines that are run in parallel. Each of those pipelines runs in a virtual machine independently of the other. Codeship will first run your setup commands and then any arbitrary test commands you defined for this specific pipeline via the interface. To ease distinguishing different pipelines, you can provide a friendly name for each one.
Here's a video walkthrough with a different codebase of how we'll set up ParallelCI to speed these tests:
For my first test pipeline, I'm going to run the following tests:
bundle exec rake qunit:test bundle exec rake plugin:spec bundle exec rspec spec/controllers spec/models --profile
For my second test pipeline, I'll run the following:
rm -fr spec/controllers spec/models bundle exec rspec --profile
Then I'll push a new commit, and you can see ParallelCI running below with each pipeline's individual status log:
After only 7 minutes 26 seconds, the two-pipeline build completes:
It's a sample time saving of nearly 30%. It's a good example of how Codeship's ParallelCI can save your team time when running more sophisticated build and test sequences. Also be aware that Codeship allows for up to 20 parallel test pipelines — the speed gain here can be exponential.
Keep in mind, if you have a deployment configured for a specific branch and each test pipeline reports a successful run, your deployment pipeline will be run. You do not need to change anything if you use one of the integrated Codeship deployments, e.g. Heroku, AWS CodeDeploy, Google App Engine, etc.
Also note that as your build and deployment commands are run on multiple virtual machines, artifacts created during the test steps will not be available during the deployment. If you need artifacts from the previous steps, make sure to regenerate them during the deployment using a script deployment added before the actual deployment.
There's one more thing to mention, the Codeship API.
The Codeship API
If you want even more control over Codeship to fully script and automate your continuous delivery services, Codeship offers an API:
You can configure the builds on your project and restart them as needed.
I'd like to summarize what we've learned about Codeship:
- Codeship is a hosted Continuous Delivery service that focuses on speed, reliability and simplicity.
- It automatically tests and deploys your applications.
- It's integrated with all the relevant source code management systems, e.g. GitHub and Bitbucket
- It supports a variety of programming languages, e.g. Ruby on Rails, Node.js, PHP, Python, Go, Dart, etc.
- It's well integrated to a variety of popular deployment solutions, e.g. Heroku, AWS CodeDeploy, Google App Engine, etc.
- It provides Notifications via email and chat services like Slack, HipChat, etc.
- You get 100 builds/month and 5 private projects for free to get started easily and without risk. Open source projects can use Codeship for free. Educational projects receive a 50% discount.
- Codeship's newest feature ParallelCI lets you run builds in parallel, speeding up the test suite run time tremendously.
- If you'd like to automate Codeship, you can use its API.
Codeship is able to provide value to small teams with their free or basic plan, as well as very large companies, that need up to 20 parallel test pipelines, with their Enterprise plan. See the Codeship pricing page for more information. There's even a free 14 day trial for anyone to try out their 20 parallel pipeline testing configuration.
I hope you give Codeship a try. Share your experience in the comments. As always, please feel free to post your questions too. You can also reach me on Twitter @reifman or email me directly. Visit my Tuts+ instructor page to see other tutorials I've written.
- Codeship Website
- The Codeship Blog
- Codeship Documentation
- Codeship Video Guides
- Efficiency in Development Workflows (eBook)
Source: Tuts Plus