Search for a command to run...
Ever noticed your current container image building process? Do you know how you can make it more productive by supporting multiple CPU types?
Hey, Y'all 👋 Kratik Here :)
Hailing from The City of Lakes, India.
I love to learn and share stuff about DevOps, SRE, Security, Cost Optimization, etc.
Join me on this journey where we will learn together with the help bunch of interesting blogs.
I've recently been asked about adding ARM64 support to one of my open source docker projects, something I knew nothing about so your article is very helpful. Do you have any tips for integrating this into a GitHub Actions workflow?
Hi Dave Murray Thanks for your kind words. Integration with GitHub Actions is too easy than you can think :) Check out this pretty neat guide from Docker: https://github.com/docker/build-push-action/blob/master/docs/advanced/multi-platform.md
Let me know if this works out for you or not. 🙏
DNS Search Domains, ndots, and Kubernetes Resolver Behavior
Understand how IAM Access Entries can replace aws-auth for managing EKS IAM authentication.
Effective Techniques for Providing IAM Access to EKS Pods
Ever wondered how can you create your own custom Github Action and publish it to the GitHub Marketplace? Let's learn it step by step in this blog.
Let's setup Vault with External-secrets to manage secrets in Kubernetes like never before!
Today, we are going to learn how we can build docker images for containers that will be compatible with the amd64 and arm64 CPU architecture types.
I chose these two CPU types because amd64 is already widely used with our Linux/Windows/Mac Computers and arm64 is getting popular nowadays, for example - new Mac computers are coming with Apple's in-house M1 processor which is arm64 based and if you talk about Cloud, AWS is offering various VMs which are powered by Graviton processors which are again arm64 based, So if your container images are able to run on these two kinds of CPUs, you will be able to utilize most of it.
Along with that what I want is that we understand each and every word that we will be using.
CPU is short for Central Processing Unit, which is generally called processor(or microprocessor).
Also called the heart/brain of the computer. It performs all arithmetic and logical operations.
Inside the CPU, there are transistors, which control the flow of electricity. You know, a CPU can perform millions of instructions per second but can only do one instruction at any given point in time.
Then we have multi-core CPUs, each core is a sort of separate CPU, which can help the CPU to do multiple tasks simultaneously.
Read these for more details on:
An instruction set is a group of commands for a central processing unit (CPU) in machine language.
Now, ISA (instruction set architecture) is a part of processor architecture. It works as an interface(communication rules) between hardware and software. It's a group of commands which are implemented in the processor's circuitry also called microarchitecture.
Few examples of what ISA defines:
It helps us to separate hardware and software development without worrying about the other.
ISA has two major classifications :
It includes many specialized instructions. It will use fewer instructions but each instruction will take multiple machine cycles.
It uses a smaller, optimized set of generalized, simple instructions. It will use more number of instructions but each instruction will take one clock cycle.
Before that let's see what is 32-bit/64-bit architecture - It simply defines that in a single instruction cycle (fetch-decode-execute) how much data a microprocessor can process.
32-bit means it can process 4 bytes of data and for 64-bit it is 8 bytes.
x86 is a family of instruction set architectures (ISA) for computer processors initially developed by Intel. (86 comes from the microprocessor name which was 8086 )
AMD took this x86 architecture which could only support 32-bit and then added 64-bit computing capabilities, that's why it is known as amd64. Also referred to as, Intel 64, x86-64, x64.
Let's understand what are ARM Processors first.
ARM stands for Advance RISC Machine - It is an ISA developed by a company called ARM Ltd.
So, ARM Licenses the architecture to other companies so they can develop their custom processors. Example - Apple's Silicon processors.
ARM Processors are generally used in smartphones, tablets, IoT devices, etc. due to their low power consumption and high-performance nature.
So ARM64 is the 64-Bit version of ARM processors(aka ARMv8-A).
So now as we have some sort of idea about CPUs, we can guess that when we build docker images on a machine with specific architecture(amd64 or arm64), it is compiled in a way that it can run only on that specific type of platforms.
So what we will try to solve here is to build universal docker images, which can run on these platforms, without us worrying about the rebuilding of images.
I built a docker image on my Macbook with an M1 Pro chip (arm64) and then tried to run it on an EC2 with an amd64 processor, this was the output :
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64) and no specific platform was requested.
exec /docker-entrypoint.sh: exec format error
docker buildx is a CLI plugin that enhances the existing docker build capabilities. (which uses another open-source project called buildkit.)
BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive, and repeatable manner.
If you want to know more about moby/buildkit you can read this Introductory blog post: https://blog.mobyproject.org/introducing-buildkit-17e056cc5317
You can refer to this Installation guide on how to install buildx into your system, It comes pre-installed with Docker Desktop for Windows/macOS.
docker buildx ls
docker buildx create --name mybuilder --driver docker-container --use --bootstrap
We are instructing docker to create a new buildx builder named mybuilder which will use the docker-container driver (which supports multi-architecture builds), also instructing it to use this new builder, also instructing this to boot this builder which means it will pull necessary docker image to run this builder and run the builder container. (as our driver is docker-container)
Consider this simple Dockerfile, which intends to run an NGINX server
Dockerfile
FROM nginx:stable-alpine
LABEL MAINTAINER="KRATIK JAIN"
COPY --chmod=0755 custom-web-page.sh /docker-entrypoint.d/
custom-web-page.sh
#!/bin/sh
uname -a > /usr/share/nginx/html/index.html
what this script does is replaces default HTML with the output of uname -a, which prints out the system info like kernel info, Architecture, etc.
docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t k4kratik/multi-platform:v1 --push .
Here you can notice the flag --platform & --push: here you are specifying the platforms for which you want to build the image, and you also wish to push the image once done. That's amazing as build and push commands are in just one command. Output also seems to be more verbose.
amd64 arch Ubuntu machinedocker run --rm -itd -p 30000:80 k4kratik/multi-platform:v1
Now hit the server endpoint and check the response
curl localhost:30000
Output
Linux e14cc6b4eb86 5.15.0-1011-aws #14-Ubuntu SMP Wed Jun 1 20:54:22 UTC 2022 x86_64 Linux
arm64 arch M1 Chipdocker run --rm -itd -p 30000:80 k4kratik/multi-platform:v1
Now hit the server endpoint and check the response
curl localhost:30000
Output
Linux 595316994de1 5.10.104-linuxkit #1 SMP PREEMPT Thu Mar 17 17:05:54 UTC 2022 aarch64 Linux
Notice the output from both of the machines and observe the second last string, for EC2, it was: x86_64 and for Macbook it was aarch64
which means it pulled the image according to its CPU architecture.
Using the below command you can inspect the image to know more details about it and to know the platforms available:
docker buildx imagetools inspect k4kratik/multi-platform:v1
Output
Name: docker.io/k4kratik/multi-platform:v1
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest: sha256:c534475bb099316622a719407a17e2da21d79d147e9f607b4a1262c2a6f3bb1a
Manifests:
Name: docker.io/k4kratik/multi-platform:v1@sha256:1d321fb1d14afe61e6f5c9ea2fc7e1b20bddd347c98a1b09f9367a6c446898b9
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/amd64
Name: docker.io/k4kratik/multi-platform:v1@sha256:47b6ffc133acac83fc70dd176606de8b033ae735d386160e6d68ee7d472119b3
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm64
Name: docker.io/k4kratik/multi-platform:v1@sha256:89f7b675b538e2aafe060dda535a48311bd090c63d0e3bb0184a440a32079f7d
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm/v7
You can use the following flag to pull or run an image of a specific platform
docker pull --platform linux/arm64 alpine:latest
docker run -itd --platform linux/arm64 nginx-alpine:latest
Well, it was just scratching the surface. I hope it was helpful enough for you to get started on your feet so you can research and dig down further.
See you in the next blog! :)
Update: I can not express my happiness when I got to know that many people are reacting to this blog and this blog got featured on Hashnode
Thank you so much 🫶🏻 everyone! You just made my day 🥂
🛣