Whenever you find yourself in a repetitive task, try this: Step back and think about ways you can automate it. One tool CI/CD people like to use for automation is GitHub Actions.
GitHub Actions are widely used for running tests and testing on our code. But what happens when your automation code changes, and you want to commit those changes? If you’re thinking of running the script manually and opening a pull request, that’s an option. But here’s a better option: Automate the process by using GitHub Actions to open pull requests for you. This article shows you how, step by step.
At Aven, we’ve got an interesting problem: We keep expanding our list of cloud providers. This is good because it gives you the flexibility to choose whichever provider works for you. For us, the maintainers of the Dev Portal documentation, it is a challenge to ensure that the latest information about our cloud providers is available to our readers.
You can find a list of Aven’s cloud providers here [Developer.aiven.io], which is also the portal where we host the developer documents. To quickly retrieve the list of currently available cloud providers, my colleague Lorna Mitchell wrote a Python script that pulls our cloud list from the Aven API and generates the documentation, which is really cool!
It was great to have the script. Now, by running the script, we can quickly generate new documents from Aven API data. However, it did not completely automate the process. Getting merged changes regularly requires a number of manual actions: run scripts, check if new changes are to be made, create pull requests, etc.
This is a good example where you can configure a GitHub Actions Workflow to run scripts periodically and open pull requests when changes are detected. So if you have a similar challenge, below you will find how to add an automation flow step by step.
Configure GitHub Actions Workflow
Time for some action – or better, GitHub Actions! You can follow these steps to configure your GitHub Actions workflow:
1) check script
Let’s make sure the script works correctly before using it for automation. So, here you can run your script twice and check that it is working as it is supposed to.
To make the process easier, you can add commands to run scripts in the Makefile. Makefiles are a helpful way to organize code compilation, and it helps to run long commands in a shorter version using
make command. Following on from our example, here’s what we add to the Makefile:
``` SOURCEDIR = . # (Re)Generate cloud listing cloud-list: python "$(SOURCEDIR)/scripts/aiven/clouds.py" "includes/clouds-list.rst" ```
Now we can just run the script with:
``` make cloud-list ```
This step makes sure that the script you use to change the code produces the changes you want to make in the pull request. Mistakes happen. If your script is not working as expected, you may need to debug your code before proceeding. If the script is working, you can go ahead and create a GitHub workflow.
2) Create a GitHub Workflow
In this step, you need to add a YAML file to define the workflow configuration. First, make sure you’ve created a
.github/workflows/ folder; If not, you can go ahead and create the directory. Then, create a YAML file to define the workflow configuration, for example,
.github/workflows/cloud-list.yaml, In this file, you can define the name of the workflow. In our example, we start our workflow by giving it a name:
``` name: Cloud - Create PR to update available list ```
Choose a name for the file and for the workflow that relates to the changes you are bringing to your code. It can help to identify your code and your workflow. Check out this quick guide from GitHub on how to create your first workflow.
3) Choose a trigger
GitHub Actions enables you to trigger your workflow to run based on a certain event or events. In the example, the task needs to run periodically. For that, configure it to run
on schedule According to cron job.
Cron syntax can be confusing sometimes. To double check your cron syntax, you can use an online and free tool called crontab guru. Moreover, the page has many examples which can help to understand the syntax better.
For example, this workflow is configured to run every Tuesday at 6 AM UTC (8 AM CEST):
``` on: schedule: - cron: "0 6 * * 2" ```
On GitHub Actions, the time is based on UTC, so you may want to consider this during setup.
Comment: On GitHub Actions, you can run scheduled workflows once every 5 minutes.
If you want to run your job in each pull request or have a different use case, you can view all available triggers to run your workflow on the GitHub documentation.
3) Install dependencies and run script
A GitHub workflow consists of one or more tasks. A job is a set of steps executed in a certain environment which is defined as:
runs-on, In the example below,
CloudList The job runs on the latest ubuntu image. After defining the environment, you need to do
checkout From a branch, for example the main branch, and decide on the Python version. Lastly, you need to add a step to install dependencies and run your scripts.
``` jobs: CloudList: runs-on: ubuntu-latest steps: - name: Checkout the repo uses: actions/checkout@v2 - name: Set up Python 3.8 uses: actions/setup-python@v2 with: python-version: "3.8" - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt - name: Update available cloud list run: make cloud-list ```
This makes the job already run periodically and generate the file. However, the end goal is to open a pull request so that the maintainers can examine the changes and merge the code. So let’s see how you can add a pull request step to your workflow.
4) Create pull request
You can create a pull request by using the Create Pull Request action in our workflow. A nice feature here is that you can add some customizations related to your pull request that can help with the pull request review process, including a label and a defined branch name, for example:
``` - name: Create Pull Request uses: peter-evans/create-pull-request@v4 with: commit-message: Cloud - Update available list title: Cloud - Update available list body: Cloud - Update available list base: main labels: automated-pr, Clouds & Regions branch: cloud-update-advanced-params delete-branch: true ```
The workflow runs the specified task as per the schedule even if the tasks are not merged.
In addition to running a workflow from time to time, you can combine this with the option to trigger your workflow manually, which can help you test it. You can check out this GitHub Actions manual for that.
When the action takes place, the shiny new pull request looks like this:
So every week, the workflow goes on and always keeps our Aven cloud inventory updated for our dear users. I
As DevPortal maintainers, we are happy to use GitHub Actions in our CI/CD process to always keep our Aven Cloud inventory up-to-date.
In this article, we reviewed the workflow code one segment at a time. The complete script looks like this:
name: Cloud - Create PR to update available list on: schedule: - cron: "0 6 * * 2" jobs: CloudList: runs-on: ubuntu-latest steps: - name: Checkout the repo uses: actions/checkout@v2 - name: Set up Python 3.8 uses: actions/setup-python@v2 with: python-version: "3.8" - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt - name: Update available cloud list run: make cloud-list - name: Create Pull Request uses: peter-evans/create-pull-request@v4 with: commit-message: Cloud - Update available list title: Cloud - Update available list body: Cloud - Update available list base: main labels: automated-pr, Clouds & Regions branch: cloud-update-advanced-params delete-branch: true
In addition to running a workflow from time to time, you can also add the option to trigger your workflow manually. You can check out this GitHub Actions manual with all the information.
If that’s what you have in mind for your pull request, you can see different examples on how to customize
create-pull-request Actions to fit your use case.
You can find more examples here to learn how to use GitHub Actions to open pull requests for you. Have fun automating your pull requests ️!