Use an AWS Serverless lambda to trigger a Jenkins build by a Github PR comment – Part 1

13 Nov

At Flipkart, I came across a workflow that involved Git and Jenkins integration – A comment that said ‘build’ on a pull request would trigger a Jenkins build job for that branch. I am aware of Github webhooks that integrate with Jenkins and can trigger a job when a branch is merged into the main branch, or when a new commit is pushed to a branch. However, using comments to drive builds was something new to me, and it always made me curious as to how it worked. So I set out on a path to figure it out for myself. Initially I devised a solution that would check all my commit messages and triggered the build if the last message contained the word ‘build’. However, the control would come into being only after the job was triggered and this meant wasting a job trigger. I needed an intermediate server that would listen to my Github webhook deliveries and then trigger the build for a particular branch if I commented ‘build’ on the PR for it. I came across AWS serverless, and it all fell into place.

This blogpost will try to describe the process, in hopes that it may help someone who is trying to do something similar. Mind you, I am using comments to trigger my builds, with this code/lambda, you can very well trigger builds again any Github event you wish.

Let’s build stuff!

The prerequisites:

  • A Git repository – Duh, obviously!
  • A Jenkins setup – I did this on my local machine, and exposed it using ngrok. But you can provide your own on premise Jenkins URL.
  • An AWS account – I am using a free tier account which I created for my learning.

Working knowledge of Git as well as AWS Serverless and lambdas is highly advisable. This is not an AWS lambda tutorial, but I will add some links as well as tips to get you started if you are new to Serverless. I have also included a few basic commands.

Serverless Setup

We will first begin by creating our AWS lambda that does all the heavy lifting. The following commands are for reference:

Install Serverless locally:

$ npm install -g serverless

Next we need to create an application that will hold our lambda function. Create a folder on your local machine where you’d like to keep your project and cd into it. Type the following command to begin:

$ serverless

This will present you the following options to choose from:

Serverless options

Select AWS – NodeJS – HTTP API, since that is what we want to build. Follow through the prompts for the user and the application to add this lambda to. The directory should look like this:

The serverless.yml and a handler file has been created for us

functions:
  hello:
    handler: handler.hello
    events:
      - httpApi:
          path: /
          method: get

Right now we have our root path mapped to the ‘hello’ handler from the handler.js file. We are going to change the function body later.

"use strict";

module.exports.hello = async (event) => {
  return {
    statusCode: 200,
    body: JSON.stringify(
      {
        message: "Go Serverless v2.0! Your function executed successfully!",
        input: event,
      },
      null,
      2
    ),
  };
};

To check if everything works, lets deploy the function:

$ serverless deploy

This will deploy the code to your AWS realm and return a URL or an endpoint for this API. Copy this URL somewhere, we will need it in the webhook setup stage. Hitting this URL in the browser will show the appropriate response returned by the handler, which in our case is simply the JSON object with the message, “Go Serverless v2.0! Your function executed successfully!”. You can install the serverless-offline plugin to test the function locally like so:

$ npm install -g serverless-offline

Then make an entry to the serverless.yml file:

plugins:
  serverless-offline

This will allow you to test the lambda offline before deploying by running:

$ serverless offline

The offline server runs at localhost:3000.

Now that our basic serverless set up is done, we can move ahead to setting up Git.

Github Setup

Create a branch for a project repository that you want to build using Jenkins. You can either do that locally or in your Github repository interface on the web. Create a pull request with this branch as the source. We now need to set up a webhook that will call our lambda function. Go to your repository on Github, and go to the Settings for it.

Go to the webhooks section, and click on ‘Add webhook’. This will open up the following screen:

We need to provide the URL for our lambda in the Payload URL field. Paste the URL from the serverless setup step we followed earlier, which was generated when you deployed the lambda to AWS. When the webhook is triggered, Github will deliver the payload to this URL. Keep the content type as form/urlencoded. Now we need to select the hook. Check ‘Let me select individual events’. From the list, select ‘Issue comments’ which corresponds to PR comments.

Save the hook set up. That’s it, our Git repository is set to push to our lambda.

We can now move on to our Jenkins setup. As mentioned earlier, Jenkins can be installed locally and exposed over the network via ngrok, or you can have a dedicated server for it. All we need is the URL to which we will make API requests.

Jenkins Setup

Log in to your Jenkins account, and create a ‘Freestyle project’.

(It is assumed that you have provided a Git executable path for Jenkins before this step.)

This will be our build job, you can name it anything. We need this name to trigger the build via the API call. Head to the Source Code Management section, and provide the URL of your Github repository in the input, as well as the branch name. The master branch is prefilled here, but in most scenarios, master is the default branch, and PRs are created from other branches against it. Jenkins provides it for other hooks such as merge, so when a branch is merged to master, the build should be triggered. Provide the branch name from the first Github step (or any branch that you wish to use for triggering the build and has a PR). You don’t need to provide credentials here if you have SSH set up for Git. Now we need to tell Jenkins to trigger the build by remote scripts, in the ‘Triggers’ section.

Next we need to add the actual Jenkins script or the build script. Here you can add the shell file with stages for the build, or any other script that does the actual work. The project I am using for this job is a Node JS application, or a StencilJS application to be precise. I have added a build step, which is to ‘execute shell’, and added the two required commands that will build my application. I have also provided a NodeJS runtime since Jenkins will require it for executing the above commands.

npm install
npm run build

Your project can have different steps depending on the stack/technology. With this, our Jenkins is set up is complete, and it is ready to receive calls from our lambda. The final step, or the meat of the matter remains, and that is our actual lambda code which will handle everything.

Let’s break this post here, and focus on the lambda in the next part of this post.

No comments yet

Leave a Reply