Build a multi-architecture docker image with Vagrant

Photo by Maria Teneva on Unsplash

As a developer, you want a consistent environment to do repeated task and have a predictable output of your program. In this case, you may want to consider adding vagrant into your deployment workflow.

Before you jump straight into deployment with Vagrant, you will need a provider. Here I will use virtual box. The reason is simple: free.

As Vagrant official website state: Vagrant comes with support out of the box for VirtualBox, a free, cross-platform consumer virtualisation product.

Installation Guide

There are multiple articles already tell you how to install them.

I am going to use Virtualbox and Vagrant to setup RHEL8 on macOS Catalina 10.15.3.

Objective

My goal is to allow me to run a RHEL server which can build multi architecture docker image and allow you to pass in environment variable for customisation.

Photo by Louis Reed on Unsplash

To build multi architecture, traditionally you are required hardware support. For example, you will need to buy a power server just to build container image that run smoothly in power environment which is not really cost effective.

Or maybe you have raspberry-pi stack, you want to run docker image on that, and realised that you faced exec format error.

Lucky enough, we have docker buildx in 2020! With the recent introduction of Docker’s buildx functionality, you can now easily to build multi-arch container image!

Prerequisite

  1. REDHAT DEVELOPER ACCOUNT

2. DOCKER HUB ACCOUNT

3. Git

Getting Stated with this project

Ensure you have properly set up your environment in Installation Guide section.

You can get the source code here

$ git clone https://github.com/DanielChuDC/multiarch-vagrant-rhel
$ cd multiarch-vagrant-rhel
# Check up the files
$ tree .
.
|-- LICENSE
|-- Readme.md
|-- Vagrantfile
|-- example.env
|-- golang-deployment.yaml
`-- provisioner.sh
0 directories, 6 files

Vagrantfile content

# -*- mode: ruby -*-
# vi: set ft=ruby :
# options are documented and commented below. For a complete reference,
# please see the online documentation at vagrantup.com.
# Every Vagrant development environment requires a box. You can search for
# boxes at https://vagrantcloud.com/search.
VM_BOX = "generic/rhel8"
Vagrant.configure(2) do |config|
config.env.enable # enable the plugin
config.vm.box = VM_BOX
config.ssh.extra_args = ["-t", "cd /home/vagrant; bash --login"] #https://stackoverflow.com/questions/17864047/automatically-chdir-to-vagrant-directory-upon-vagrant-ssh
config.vagrant.plugins = "vagrant-env"
config.vm.provider "virtualbox" do |vb|
vb.memory = 4096
vb.cpus = 4
end
config.vm.provision 'shell' do |s|
s.path = 'provisioner.sh'
s.env = { "RH_ACCOUNT_USERNAME"=>ENV['RH_ACCOUNT_USERNAME'],
"RH_ACCOUNT_PASSWORD"=>ENV['RH_ACCOUNT_PASSWORD'],
"ORG_ID"=>ENV['ORG_ID'],
"Key_NAME"=>ENV['Key_NAME'],
"DOCKER_HUB_USERNAME"=>ENV['DOCKER_HUB_USERNAME'],
"DOCKER_HUB_PASSWORD"=>ENV['DOCKER_HUB_PASSWORD'],
"DOCKER_IMAGE_NAME"=>ENV['DOCKER_IMAGE_NAME'],
"DOCKER_IMAGE_TAG"=>ENV['DOCKER_IMAGE_TAG'],
"GIT_REPO_URL"=>ENV['GIT_REPO_URL'],
"GIT_BRANCH"=>ENV['GIT_BRANCH'],
"TARGET_PLATFORM"=>ENV['TARGET_PLATFORM']}
end
config.trigger.before :destroy do |trigger|
trigger.warn = "Unregister redhat developer account"
trigger.run_remote = {
inline: "
if subscription-manager status; then
sudo subscription-manager unregister
fi
"}
end
end

provisioner.sh content

#! /bin/bash   
set -m
echo "Setting environment variable"echo $RH_ACCOUNT_USERNAME
echo $RH_ACCOUNT_PASSWORD
echo $ORG_ID
echo $Key_NAME
echo $DOCKER_HUB_USERNAME
echo $DOCKER_HUB_PASSWORD
echo $DOCKER_IMAGE_NAME
echo $DOCKER_IMAGE_TAG
echo $GIT_REPO_URL
echo $GIT_BRANCH # Add in git branch feature
echo $TARGET_PLATFORM
echo "Checking this OS image and kernel version"
cat /etc/redhat-release
if ! subscription-manager status; then
sudo subscription-manager register --username=$RH_ACCOUNT_USERNAME --password=$RH_ACCOUNT_PASSWORD --auto-attach
sudo subscription-manager register --org=$ORG_ID --activationkey=$Key_NAME
sudo subscription-manager attach
sudo subscription-manager list
fi
sudo yum update -y
sudo yum install -y yum-utils device-mapper-persistent-data lvm2
echo "Cannot install directly, faced error while install containerd.io"
echo "current work around is to manually install containerd.io"
sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf repolist -v
sudo dnf list docker-ce --showduplicates | sort -r
sudo dnf install docker-ce-3:19.03.8-3.el7
sudo dnf install --nobest docker-ce -y
sudo dnf install https://download.docker.com/linux/centos/7/x86_64/stable/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm -y
sudo systemctl enable docker
sudo systemctl start docker
echo "validate docker running successfully"
sudo docker run hello-world
echo "groupadd docker user, so we don't run permission with sudo"
sudo groupadd docker
sudo usermod -aG docker vagrant
echo "getting root permission"
export DOCKER_CLI_EXPERIMENTAL=enabled
echo "Install qemu for RHEL 8"
sudo yum install qemu-kvm -y
echo "Checking qemu version"
sudo /usr/libexec/qemu-kvm --version
sudo yum info qemu-kvm
echo "Checking Kernel Version"
uname -r
echo "login docker "
sudo docker login -u $DOCKER_HUB_USERNAME --password=$DOCKER_HUB_PASSWORD
echo "Install jq "
sudo yum install jq -y
# Using jq to persist the buildx changes
sudo jq '. + {"experimental": "enabled"}' /root/.docker/config.json > abc.json
cat abc.json
sudo mv abc.json /root/.docker/config.json
# Add dns record for daemon.json
echo '{"dns":["8.8.8.8","8.8.4.4"]}' | sudo tee -a /etc/docker/daemon.json
echo "stop RHEL firewall"
sudo systemctl stop firewalld
echo "make docker bridge able to access internet"
sudo sysctl net.ipv4.conf.all.forwarding=1
sudo iptables-save > your-current-iptables.rules
sudo iptables --flush
sudo iptables -P FORWARD ACCEPT
sudo iptables -I INPUT -j ACCEPT
sudo iptables -I OUTPUT -j ACCEPT
echo "Restart docker to make effect"
sudo systemctl restart docker
echo "Enable DOCKER_CLI_EXPERIMENTAL Flag"
export DOCKER_CLI_EXPERIMENTAL=enabled
echo 'export DOCKER_CLI_EXPERIMENTAL=enabled' > .bash_profile
source .bash_profile
echo "validate buildx command"
echo $DOCKER_CLI_EXPERIMENTAL
docker buildx build
echo "Enable binfmt_misc to run non-native Docker images"
sudo docker run --restart always --privileged multiarch/qemu-user-static --reset -p yes
echo "Checking binfmt_misc"
ls /proc/sys/fs/binfmt_misc/
cat /proc/sys/fs/binfmt_misc/qemu-aarch64
echo "Creating a Buildx Builder"
docker buildx create --use --name mybuilder
docker buildx inspect --bootstrap
docker buildx ls
echo "Installing development tools in RHEL 8"
sudo yum install git -y
echo "Git clone the url"
if [ -d "example" ]; then
echo "folder example exists. Delete it now."
rm -rf ./example
fi
if [ -z "$GIT_BRANCH" ]; then
echo "\$GIT_BRANCH is NULL set to default"
export GIT_BRANCH='master'
else
echo "\$GIT_BRANCH is NOT NULL"
fi
git clone $GIT_REPO_URL ./example
cd /home/vagrant/example && git branch -a # list your repo
cd /home/vagrant/example && git checkout $GIT_BRANCH # Change branch
cd /home/vagrant/example && git branch # Confirm you are now working on that branch
echo "Build Multiarch Image using Dockerfile and Buildx"
cd /home/vagrant/example && docker buildx build -t $DOCKER_HUB_USERNAME/$DOCKER_IMAGE_NAME:$DOCKER_IMAGE_TAG --platform=$TARGET_PLATFORM . --push

Customisation with environment variable

If you are a developer yet not familiar with environment variable, there is a introduction article here to show how important it is.

$ cat example.envRH_ACCOUNT_USERNAME=<your-rh-account-username>
RH_ACCOUNT_PASSWORD=<your-rh-account-password>
ORG_ID=<your-org_id> # optional,can left blank,unless your password has special character
Key_NAME=<your-keyname> # optional,can left ,unless your password has special character
DOCKER_HUB_USERNAME=<your-docker-hub-username>
DOCKER_HUB_PASSWORD=<your-docker-hub-password>
DOCKER_IMAGE_NAME=<your-image-name>
DOCKER_IMAGE_TAG=<your-image-tag>
GIT_REPO_URL=https://github.com/DanielChuDC/loopback-example #change as you wish
GIT_BRANCH=master
TARGET_PLATFORM=linux/arm64,linux/amd64,linux/s390x,linux/ppc64le
# target platform
# possible values: linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

Rename example.env to .env and fill in the correct credential accordingly.

You should not check in .env that contains sensitive information/ credentials in Github.

Here we want to pass in the environment variable to the Vagrantfile using vagrant plugin, called vagrant-env .

$ vagrant plugin install vagrant-env

Optionally, you can use export function without using .envto pass in variable.

export RH_ACCOUNT_USERNAME='<your-rh-account-username>';
export RH_ACCOUNT_PASSWORD='<your-rh-account-password>';
export ORG_ID='<your-org_id>';
export Key_NAME='<your-keyname>';
export DOCKER_HUB_USERNAME='<your-docker-hub-username>';
export DOCKER_HUB_PASSWORD='<your-docker-hub-password>';
export DOCKER_IMAGE_NAME='<your-image-name>';
export DOCKER_IMAGE_TAG='<your-image-tag>';
export GIT_REPO_URL='https://github.com/DanielChuDC/loopback-example';
export TARGET_PLATFORM='linux/arm64,linux/amd64,linux/s390x,linux/ppc64le';

Ready to bake your first multi-arch container image

  • Provision your RHEL server and bake your container image by using
# start server
$ vagrant up

In this project, I have set up an example Github repo by using Loopback.

loopback 4

Waiting for it to download and install all the softwares and push your multi-arch container image into docker hub

If you see the similar message about exporting image and pushing image successful, means it done!

Check the image status on docker hub

You should go to your repo to check if you have pushed your image.

Validation: In zLinux

get your instance of zLinux from IBM LinuxONE Community Cloud, once you have a VM server running, ssh into it and run the following commands. Do refer to Part 1 and Part 2 on how to use LinuxONE.

$ docker run --rm -d -p 3000:3000 moxing9876/multi-arch-vagrant:3.0.3
# expected output
Status: Downloaded newer image for moxing9876/multi-arch-vagrant:3.0.3
26e3475d3a3f697c1c5d655df45a7693b49d50286272efc0c0326538eaa843b5
$ curl http://localhost:3000/ping
# expected output
{"greeting":"Hello from LoopBack","date":"2020-03-17T14:52:27.320Z","url":"/ping","headers":{"user-agent":"curl/7.29.0","host":"localhost:3000","accept":"*/*"}}
$ $ docker image inspect moxing9876/multi-arch-vagrant:3.0.3 | grep Architecture
# expected output
"Architecture": "s390x",

Clean up

When you do not need this RHEL server and want to delete this project, run this command in the terminal

# Run when you want to delete this server
$ vagrant destroy -f #destroy server
# expected output
==> default: Running action triggers before destroy ...
==> default: Running trigger...
==> default: Unregister redhat developer account
default: Running: inline script
default: Unregistering from: subscription.rhsm.redhat.com:443/subscription
default: System has been unregistered.
==> default: Forcing shutdown of VM...
==> default: Destroying VM and associated drives...

Summary

This project show the ability of building multi-arch container image by vagrant, virtual box, buildx, qemu. It can be a proof-of-concept for CI/CD workflow of building multi-arch container image. Not only it save cost, it enable better documentation and consistent development environment.

You may want to host this on IBM Cloud, AWS, Google Cloud by using terraform.

p/s: Thank you Jaric Sng, show me how important environment variable is, and guide me on this project

Do check out his 4 articles on

  • Part 1: Multi-architecture cloud native Loopback (nodejs) in LinuxONE and others
  • Part 2: Multi-architecture cloud native Loopback (nodejs) in LinuxONE and others
  • Part 3: Multi-architecture cloud native Java microservice for LinuxONE and others
  • Part 4: Creating a build server for Cloud native Multi architecture docker build (Part 4)

See you next time!

--

--

--

https://www.youracclaim.com/users/jia-hao-chu/

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Road to Go Pro — Packages & Modules

A practical guide to contributing to Open Source Projects

Embracing TDD in Flutter — Managing large scale project with ease

Technical Case Study for Software Developers

10 Apps Made with Xamarin: Cross-Platform Development in Practice

Ionic Google Map Integration

DB Weekly Issue 16

Scheduling automated deletion of AWS Cloudformation stacks

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Daniel Chu

Daniel Chu

https://www.youracclaim.com/users/jia-hao-chu/

More from Medium

Error-free installation of Owncloud Server on Windows 10 within 30 minutes using WSL!

Image Vulnerability Detection using Trivy

Jenkins Installation On CentOS7

Use a Script to install and launch an Apache Server on CentOS 8