Hello beautiful people of the internet ☘️
Guess who decided to break the hiatus with some insightful blog. Hope you find value in this piece. Your feedback is appreciated.

Intro to GitHub Actions

I assume you already have some knowledge about GitHub actions, and in case if you don't : This is an automation tool provided by GitHub*(by Microsoft) -* as most of us have our code stored up in GitHub repos, many people use it for CI/CD pipelines.

Not just CI/CD, you can do a lot of thing with GitHub Actions, possibillites are limitless and depends on your creativity how you use it.

By Default, GitHub runs your workflow on their own servers aka runners. You can also host your CI/CD workers called self-hosted runners.

How does it work ?

Well, it's simple - you have to define a workflow file in YAML following the syntax and place this file in .github/workflows/ directory. You also have to define a trigger in this workflow and whenever that trigger event occurs, boom! your workflow is started. This is fun right ?

P.S. - I am using GitHub Actions from past couple of years and yes, and it still surprises me with some bugs (or features maybe?) but lately I have learned how to deal with it.

Types of Custom GitHub Action

From the docs https://docs.github.com/en/actions/creating-actions/about-custom-actions.

Docker - Docs - In this, you can containerize your action so it can run anywhere. We will be using this approach to create our custom action.
Javascript - Docs - GitHub runners already have node installed, so you can write your custom action in javascript to run it directly on the runner during workflow execution. It's fastest among other approaches.
Composite -Docs- In this, you combine multiple steps in a GitHub Action, and later you can use them as a composite workflow. I think it's superseded by Reusable Workflows and is a better alternative.

Enough with the types, let's dig into creating one.

Building a Custom GitHub Action from scratch

Feeling Good Love GIF by The Bachelor

for the demo we will build a custom action whose job would be -

To detect if a Container image with given tag exists in a container registry (DockerHub or AWS ECR) or not, if exists, output some flag so build/push step can be skipped.

sometimes when we run the same CI/CD pipeline again (for whatever reason), we would like to skip the build/push stage so we can save some time as that image is already build and pushed, so why do it again ?

Cool, Assuming you understood the problem statement, So now we know what our custom GitHub Action will solve.

Let's Start.

Step 0 : Create and Init an empty git repo

mkdir container-image-check-custom-action
cd container-image-check-custom-action
git init

Step 1 : Write your script which solves your problem statement

We don't want to go deep into the development of below script but want to make sure it solves our problem statement.

Why Everyone Needs to KISS | Force 5

let's write our script

script.sh

#!/bin/bash

CONTAINER_REPO_NAME="$2"
CONTAINER_IMAGE_TAG="$3"

# in case if it is a Docker image
DOCKER_HUB_USERNAME="$4"
DOCKER_HUB_PAT="$5"

function check_ecr_image_tag_if_exists() {

    echo "Started Image & Tag checking for repo $CONTAINER_REPO_NAME for tag $IMAGE_TAG"

    IMAGE_META="$(aws ecr describe-images --repository-name=$CONTAINER_REPO_NAME --image-ids=imageTag=$CONTAINER_IMAGE_TAG 2>/dev/null)"

    if [[ $? == 0 ]]; then
        IMAGE_TAGS="$(echo ${IMAGE_META} | jq '.imageDetails[0].imageTags[0]' -r)"
        echo $IMAGE_TAGS
        echo "image_exists=true" >>$GITHUB_OUTPUT
        echo "Image $1:$2 exists on ecr."
        echo $IMAGE_META
    else
        echo "$1:$2 does not exist on ecr."
        echo "image_exists=false" >>$GITHUB_OUTPUT
    fi

}

function check_docker_image_tag_if_exists() {

    echo "Started Image & Tag checking for repo $CONTAINER_REPO_NAME by user $DOCKER_HUB_USERNAME for tag $CONTAINER_IMAGE_TAG"

    TOKEN=$(curl -sSLd "username=${DOCKER_HUB_USERNAME}&password=${DOCKER_HUB_PAT}" https://hub.docker.com/v2/users/login | jq -r ".token")
    REPO_RESPONSE=$(curl -sH "Authorization: JWT $TOKEN" "https://hub.docker.com/v2/repositories/${DOCKER_HUB_USERNAME}/${CONTAINER_REPO_NAME}/tags/${CONTAINER_IMAGE_TAG}/")

    echo
    echo Response is:
    # echo $REPO_RESPONSE | jq .
    echo
    echo Tag status is:
    TAG_STATUS=$(echo $REPO_RESPONSE | jq .tag_status | tr -d '"')
    echo $TAG_STATUS
    echo

    if [[ $TAG_STATUS == *"active"* ]]; then
        echo Docker Image $CONTAINER_REPO_NAME exists with Tag $CONTAINER_IMAGE_TAG
        echo "image_exists=true" >>$GITHUB_OUTPUT
    else
        echo "Docker Image Does Not Exist"
        echo "image_exists=false" >>$GITHUB_OUTPUT
    fi
}

# -------------------------------------------------------------------------

if [[ $1 == "ecr" ]]; then
    echo "got ecr: $1"
    check_ecr_image_tag_if_exists
elif [[ $1 == "dockerhub" ]]; then
    echo "got dockerhub: $1"
    check_docker_image_tag_if_exists
else
    echo "Unsupported Registry: $1"
    exit 1
fi

Flow of the script

The script when executed, will determine if a Docker Hub or ECR image exists or not based on the given repo name and tag.
The first argument is used for setting the target registry type - ecr or dockerhub
The Second and third argument is for name of the container repo and the image tag, respectively.
For Docker Hub, you need to provide your Docker Hub username and Docker Hub Personal Access Token, that's what 4th and 5th args are for.
Based on the run, if image exists or not, it will update the GitHub Actions output variable by updating GITHUB_OUTPUT file. (Read - Setting Output variables)

Example - ECR

bash container-image-check-custom-action/script.sh ecr python-server v1

Example - Docker Hub

bash container-image-check-custom-action/script.sh dockerhub python-server v1 docker_user dockerhub_my_personal_access_token_1234

Step 2 : Test Locally

Using above commands, first of all check in your local env that is this script is working as expected or not.

ECR:
args = Repo and tag

DockerHub:
notice all the args

Step 3.1 : Build Docker Image to Test

As we are using docker type of our custom action, we should build the docker image and test is locally before pushing.

As this is going to run as a container, we should verify it locally before pushing, so we can avoid excuses like :

rant - Always works on my machine - devRant

Dockerfile

FROM amazon/aws-cli:2.15.40
RUN yum update && yum install -y jq
WORKDIR /scripts
COPY . .
RUN chmod +x /scripts/docker-entrypoint.sh
ENTRYPOINT ["/scripts/docker-entrypoint.sh"]

now, build a docker image out of this

docker build . -t container-image-check-custom-action:v0

Step 3.2: Test/Run Using Docker

docker run -it container-image-check-custom-action:v0 dockerhub shorty latest k4kratik dckr_pat_1234

we can see it give us outputs as expected. (Yes, I know. I just tested it for DockerHub and not for AWS ECR.)

GIF by Giphy QA

Step 4 : Define the Action definition in action.yaml

Now, we will create action.yaml so we can specify GitHub how to use our script in other workflows as a custom action.

name: Container Image Existence Checker

description: Check if a given ECR or Docker Hub image exists or not in the registry.

inputs:
  type:
    required: true
    default: "ecr"
    description: Type of the registry, allowed values are "ecr" and "dockerhub"
  container_repo_name:
    required: true
    description: Name of the Container Repository
  image_tag:
    required: true
    description: Image Tag for which you want to check.
  dockerhub_username:
    required: false
    description: Docker Hub username for authentication for private repositories.
  dockerhub_token:
    required: false
    description: Docker Hub Personal Access Token for authentication for private repositories.

outputs:
  image_exists:
    description: "A boolean value to indicate if the image exists"

runs:
  using: "docker"
  image: "Dockerfile"
  args:
    - ${{ inputs.type }}
    - ${{ inputs.container_repo_name }}
    - ${{ inputs.image_tag }}
    - ${{ inputs.dockerhub_username }}
    - ${{ inputs.dockerhub_token }}

Explaining `action.yaml`

This is the file which stays in root of our project dir and GitHub uses this to understand your custom action.

inputs : we have to define what inputs we want from user when he/she uses our custom action.
runs : we have to define the runtime behaviour of our custom action.
1. using : we define what we want to use for running our code. you can use node versions like node20 , docker or composite
2. image: either you can give direct image from docker hub e.g. docker://k4kratik/container-image-existence-checker:latest or Dockerfile path to build image on the go. For now we are using Dockerfile.
3. args : all the arguments to our code.

Step 5 : Publish To GitHub Marketplace

Create a release - named v1 (You may have to accept agreement.), Check box which says Publish this Action to the GitHub Marketplace :

(FYI - you will only see Publish this Action to the GitHub Marketplace if your repo is public.)

We got it published here : https://github.com/marketplace/actions/container-image-existence-checker

Step 6 : Use this newly created GitHub Action in some other repository

I have created a sample repository to test our new custom action : https://github.com/k4kratik/workflow-testing/

Basically this is sample repository where some code is hosted, every time there is a commit on main branch it will trigger a GitHub Action workflow where code will be built and pushed to DockerHub.

Workflow for ECR :

name: ECR CI/CD Workflow for Service

env:
  SERVICE_NAME: my-backend-service

on:
  push:
    branches: [main]

jobs:
  ci:
    name: CI Job
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ap-south-1

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Check if the image exists
        uses: k4kratik/container-image-check-custom-action@v4
        id: check_image
        with:
          type: ecr
          container_repo_name: ${{env.SERVICE_NAME}}
          image_tag: ${{ github.sha }}

      - name: Docker build and push to ECR (Skip this step if image exists)
        if: steps.check_image.outputs.image_exists != 'true'
        run: |
          docker build -t ${{ secrets.ECR_REGISTRY }}/${{env.SERVICE_NAME}}:${{ github.sha }} .
          docker push ${{ secrets.ECR_REGISTRY }}/${{env.SERVICE_NAME}}:${{ github.sha }}

Notice that I am using some values as GitHub Secrets.

Workflow for DockerHub

name: Docker Hub CI/CD Workflow for Service

on:
  push:
    branches: [main]

jobs:
  ci:
    name: CI Job
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Check if the image exists
        id: check_image
        uses: k4kratik/container-image-check-custom-action@v4
        with:
          type: dockerhub
          container_repo_name: shorty
          image_tag: ${{ github.sha }}
          dockerhub_username: ${{ secrets.DOCKERHUB_USER }}
          dockerhub_token: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Login to Docker Hub
        if: steps.check_image.outputs.image_exists != 'true'
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USER }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Docker build and push (Skip this step if image exists)
        if: steps.check_image.outputs.image_exists != 'true'
        run: |
          docker build -t k4kratik/shorty:${{ github.sha }} .
          docker push k4kratik/shorty:${{ github.sha }}

Step 7 : Demo Time : Verify If the custom Action Work

We pushed some code in your sample repo called workflow-testing to start the workflow.

The first workflow run

we can see it executed all the steps.

Let's re-run it.

Re-Run : ECR

Re-Run: Docker Hub

We can see in these re-runs that how beautifully it skipped the build and push image as images were already existing in relevant registries.

That Makes Me So Happy GIF by America's Got Talent

That was it.

Thanks for reading. 🌻

Links:

GitHub Marketplace Link (to our custom action)
GitHub Repo Link (codebase of our custom action)
Workflow testing Repo (where we used this custom action)

Let me know if you have any suggestions.

-- Kratik

Step by Step Guide to Create a Custom GitHub Action and Publish it to the GitHub Marketplace

Comments

GitHub Actions

More from this blog

ndots : The Hidden DNS Logic Behind resolving Kubernetes Service Names

EKS IAM Access Entries - End of aws-auth ConfigMap!

EKS Pod Identity : A better way to delegate IAM access to pods

How to use Vault with External Secrets for Kubernetes in Production?

Intro to GitHub Actions

How does it work ?

Types of Custom GitHub Action

Building a Custom GitHub Action from scratch

Step 0 : Create and Init an empty git repo

Step 1 : Write your script which solves your problem statement

Flow of the script

Example - ECR

Example - Docker Hub

Step 2 : Test Locally

Step 3.1 : Build Docker Image to Test

Step 3.2: Test/Run Using Docker

Step 4 : Define the Action definition in action.yaml

Explaining `action.yaml`

Step 5 : Publish To GitHub Marketplace

Step 6 : Use this newly created GitHub Action in some other repository

Workflow for ECR :

Workflow for DockerHub

Step 7 : Demo Time : Verify If the custom Action Work

The first workflow run

Re-Run : ECR

Re-Run: Docker Hub

Command Palette

Comments

GitHub Actions

More from this blog

Intro to GitHub Actions

How does it work ?

Types of Custom GitHub Action

Building a Custom GitHub Action from scratch

Step 0 : Create and Init an empty git repo

Step 1 : Write your script which solves your problem statement

Flow of the script

Example - ECR

Example - Docker Hub

Step 2 : Test Locally

Step 3.1 : Build Docker Image to Test

Step 3.2: Test/Run Using Docker

Step 4 : Define the Action definition in action.yaml

Explaining action.yaml

Step 5 : Publish To GitHub Marketplace

Step 6 : Use this newly created GitHub Action in some other repository

Workflow for ECR :

Workflow for DockerHub

Step 7 : Demo Time : Verify If the custom Action Work

The first workflow run

Re-Run : ECR

Re-Run: Docker Hub

Explaining `action.yaml`