CI/CD #04. Jenkins: trigger a Freestyle project via Git webhook.

We’re setting up a Jenkins “Freestyle project”, using “Execute shell” build step, to: ⓵ clone a Git repo, ⓶ create a virtual environment, ⓷ run editable install, and finally, ⓸ run Pytest. And this Jenkins job can be triggered remotely when we push some file(s) onto the target Git repo.


CI/CD #04. Jenkins: trigger a Freestyle project via Git webhook.

We’re going to recreate the Freestyle project discussed in CI/CD #01. Jenkins: manually clone a Python GitHub repo and run Pytest, you don’t need to read this post, all necessary steps are described where appropriate.

❶ Log into Jenkins, click on + New Item. On the next page, under Enter an item name, enter app_demo – at runtime, Jenkins will create a directory with this name under Jenkins’ work directory, i.e. /var/lib/jenkins/workspace/app_demo.

And /var/lib/jenkins/workspace/app_demo is the value of the WORKSPACE Jenkins environment variable.

❷ Select Freestyle project, then click on the OK button to move to the Configuration page.

For Description, write something meaningful, e.g. CI/CD #04. Jenkins: trigger a Freestyle project via Git webhook.

The rest of the options are shown in the following two (2) screenshots:

Basically:

Repository URL: https://github.com/behai-nguyen/app-demo.git.
Branch Specifier (blank for 'any'): */main.
MOST IMPORTANTLY, see the second screenshot, under Build Triggers check GitHub hook trigger for GITScm polling. Without checking this option, Jenkins will not process the push events from Git, the on-screen explanation should sufficiently explain the purpose of this option.

Under Build Steps, pull down Add build step, and select Execute shell, then paste in this content:

PYENV_HOME=$WORKSPACE/venv

# Delete previously built virtualenv
if [ -d $PYENV_HOME ]; then
    rm -rf $PYENV_HOME
fi

# Create virtualenv and install necessary packages
virtualenv $PYENV_HOME
. $PYENV_HOME/bin/activate
$PYENV_HOME/bin/pip install -e .
$PYENV_HOME/bin/pytest

Finally, click on the Save button: we’re taken to the app_demo project page.

❸ To test that this project is working, I removed app_demo/ from /var/lib/jenkins/workspace/. Then clicked on ▷ Build Now to run the first build.

It ran successfully for me. And app_demo/ was created under /var/lib/jenkins/workspace/, so was the virtual directory app_demo/venv/.

Remove app_demo/ with:

jenkins@hp-pavilion-15:~/workspace$ rm -rf app_demo

❹ Next, we need to make our Jenkins server available publicly. For this, we’re using ngrok – this has been discussed previously in Install ngrok for Ubuntu 22.10 kinetic.

For Jenkins, the port is 8080:

$ ngrok http 8080

We’ll get something similar to the screenshot below:

In this instance, our Jenkins “public” URL is https://5b3c-58-109-142-244.ngrok-free.app. For Git payload URL (discussed later), we need to append github-webhook/.

– Please notice that it is github-webhook/: DO NOT FORGET the trailing /!

The complete Git payload URL is https://5b3c-58-109-142-244.ngrok-free.app/github-webhook/.

❺ The next step is to set up webhook for repo https://github.com/behai-nguyen/app-demo.git. A similar process has been described before, in Set up GitHub Webhook and test our server. Follow the steps described below.

⓵ Go to https://github.com/behai-nguyen/app-demo.git, or yours one. Click on Settings on top right hand corners, then click on Webhooks on the left hand side bar.

⓶ On the next screen, click on the Add webhook button on the top right hand corner.

⓷ On the next screen:

Payload URL *: https://5b3c-58-109-142-244.ngrok-free.app/github-webhook/ -- the URL we've discussed above.
Content type: application/json
I left everything else at their default values.

This screen should look like:

Click on the Add webhook button to continue. Git should send a ping event, we should receive it successfully:

❻ Test the webhook. Let’s ensure our Jenkins session is still active. Our project should show one successful build from the previous manual run. And there is no app_demo/ under /var/lib/jenkins/workspace/.

I added another marker to D:\app_demo\pytest.ini, check this one in:

It should automatically trigger another build. The ngrok console shows another event:

Git records another successful push event, details should match our commit:

Refresh Jenkins, it should show another successful build:

The detail of the second build:

Note the second screenshot: there was only one test module tests/functional/test_routes.py; and a total of 5 passed, 1 warning.

❼ Test the webhook. Remove app_demo/ from /var/lib/jenkins/workspace/:

jenkins@hp-pavilion-15:~/workspace$ rm -rf app_demo/

Add another test module D:\app_demo\tests\jenkins_demo\test_push_01.py, this module has only a single test method:

It should automatically trigger another build. Both Git and ngrok should show another successful push event. And Jenkins should show a third successful build:

The last screenshot shows the new test module, and the total tests increased by 1 (one), to six (6): 6 passed, 1 warning.

And of course, app_demo gets recreated, together with the virtual environment directory:

❽ Still testing the webhook. Leave everything as they are. Using GitHub UI, I edit https://github.com/behai-nguyen/app-demo/blob/main/tests/jenkins_demo/test_push_01.py, adding the comment # This is a manual test comment.; and commit the change:

Both Git and ngrok should show another successful push event. That makes four (4) push events so far. The following screenshot, Git shows the details of the push:

And Jenkins also shows a fourth (4th) successful build, whose details match the push event’s. There were no changes to code, tests should naturally pass and the total number of tests remains the same:

As the concluding remark to finish off this post. Are you wondering why I use Freestyle project to explore Git webhook?

I started this post using both Pipeline implementations discussed in CI/CD #03. Jenkins: using Pipeline and proper Bash script to run Pytest. I could not get Jenkins to trigger the build. I need to do more studies on this, and I will discuss my findings in later posts. And I still have not come across any documentation explaining why Freestyle project works in this instance. Studies and more studies 😂

Thank you for reading. I hope this post is informative. Stay safe as always.

✿✿✿

Feature image sources: