Chapter 7: Integrating with Existing Application Build Processes
After learning how to create custom container images using Podman and Buildah, we can now focus on special use cases that can make our build workflows more efficient and portable. For instance, small images are a very common requirement in an enterprise environment, for performance and security reasons. We will explore how to achieve this goal by breaking down the build process into different stages.
This chapter will also try to uncover scenarios where Buildah is not expected to run directly on a developer machine but is driven instead by a container orchestrator or embedded inside custom applications that are expected to call its libraries or command line interface (CLI).
In this chapter, we're going to cover the following main topics:
- Multistage container builds
- Running Buildah inside a container
- Integrating Buildah with custom builders
Technical requirements
Before proceeding with this chapter, a machine with a working Podman installation is required. As stated in Chapter 3, Running the First Container, all the examples in the book are executed on a Fedora 34 system or later versions but can be reproduced on the reader's OS of choice.
A good understanding of the topics covered in Chapter 6, Meet Buildah – Building Containers from Scratch, will be useful to easily grasp concepts regarding builds, both with native Buildah commands and from Dockerfiles.
Multistage container builds
We have learned so far how to create builds with Podman and Buildah using Dockerfiles or native Buildah commands that unleash potential advanced building techniques.
There is still an important point that we haven't already discussed – the size of the images.
When creating a new image, we should always take care of its final size, which is the result of the total number of layers and the number of changed files inside them.
Minimal images with a small size have the great advantage of being able to be pulled faster from registries. Nevertheless, a large image will eat a lot of precious disk space in the host's local store.
We already showed examples of some best practices to keep images compact in size, such as building from scratch, cleaning up package manager caches, and reducing the amount of RUN, COPY, and ADD instructions to the minimum necessary. However, what happens when we need to build an application from its source and create a final image with the final artifacts?
Let's say we need to build a containerized Go application – we should start from a base image that includes Go runtimes, copy the source code, and compile to produce the final binary with a series of intermediate steps, most notably downloading all the necessary Go packages inside the image cache. At the end of the build, we should clean up all the source code and the downloaded dependencies and put the final binary (which is statically linked in Go) in a working directory. Everything will work, but the final image will still include the Go runtimes included in the base image, which are no longer necessary at the end of the compilation process.
When Docker was introduced and Dockerfiles gained momentum, this problem was circumnavigated in different ways by DevOps teams who struggled to keep images minimal. For example, binary builds were a way to inject the final artifact compiled externally inside the built image. This approach solves the image size problem but removes the advantage of a standardized environment for builds provided by runtime/compiler images.
A better approach is to share volumes between containers and have the final container image grab the compiled artifacts from a first build image.
To provide a standardized approach, Docker, and then the OCI specifications, introduced the concept of multistage builds. Multistage builds, as the name says, allow users to create builds with multiple stages using different FROM instructions and have subsequent images grab contents from the previous ones.
In the next subsections, we will explore how to achieve this result with Dockerfiles/Containerfiles and with Buildah's native commands.
Multistage builds with Dockerfiles
The first approach to multistage builds is by creating multiple stages in a single Dockerfile/Containerfile, with each block beginning with a FROM instruction.
Build stages can copy files and folders from previous ones using the --from option to specify the source stage.
The next examples show how to create a minimal multistage build for the Go application, with the first stage acting as a pure build context and the second stage copying the final artifact inside a minimal image:
Chapter07/http_hello_world/Dockerfile
# Builder image
FROM docker.io/library/golang
# Copy files for build
COPY go.mod /go/src/hello-world/
COPY main.go /go/src/hello-world/
# Set the working directory
WORKDIR /go/src/hello-world
# Download dependencies
RUN go get -d -v ./...
# Install the package
RUN go build -v
# Runtime image
FROM registry.access.redhat.com/ubi8/ubi-micro:latest
COPY --from=0 /go/src/hello-world/hello-world /
EXPOSE 8080
CMD ["/hello-world"]
The first stage copies the source main.go file and the go.mod file to manage the Go module dependencies. After downloading the dependency packages (go get -d -v ./...), the final application is built (go build –v ./...).
The second stage grabs the final artifact (/go/src/hello-world/hello-world) and copies it under the new image root. To specify that the source file should be copied from the first stage, the --from=0 syntax is used.
In the first stage, we used the official docker.io/library/golang image, which includes the latest version of the Go programming language. In the second stage, we used the ubi-micro image, a minimal image from Red Hat with a reduced footprint, optimized for microservices and statically linked binaries. Universal Base Images will be covered in greater detail in Chapter 8, Choosing the Container Base Image.
The Go application listed as follows is a basic web server that listens on port 8080/tcp and prints a crafted HTML page with the "Hello World!" message when it receives a GET / request:
Important Note
For the purpose of this book, it is not necessary to be able to write or understand the Go programming language. However, a basic understanding of the language syntax and logic will prove to be very useful, since the greatest part of container-related software (such as Podman, Docker, Buildah, Skopeo, Kubernetes, and OpenShift) is written in Go.
Chapter07/http_hello_world/main.go
package main
import (
"log"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
log.Printf("%s %s %s ", r.RemoteAddr, r.Method, r.URL)
w.Header().Set("Content-Type", "text/html")
w.Write([]byte("<html> <body> "))
w.Write([]byte("<p>Hello World!</p> "))
w.Write([]byte("</body> </html> "))
}
func main() {
http.HandleFunc("/", handler)
log.Println("Starting http server")
log.Fatal(http.ListenAndServe(":8080", nil))
}
The application can be built using either Podman or Buildah. In this example, we choose to build the application with Buildah:
$ cd http_hello_world
$ buildah build -t hello-world .
Finally, we can check the resulting image size:
$ buildah images --format '{{.Name}} {{.Size}}' localhost/hello-world
localhost/hello-world 45 MB
The final image has a size of only 45 MB!
We can improve our Dockerfile by adding custom names to the base images using the keyword AS. The following example is a rework of the previous Dockerfile following this approach, with the key elements highlighted in bold:
# Builder image
FROM docker.io/library/golang AS builder
# Copy files for build
COPY go.mod /go/src/hello-world/
COPY main.go /go/src/hello-world/
# Set the working directory
WORKDIR /go/src/hello-world
# Download dependencies
RUN go get -d -v ./...
# Install the package
RUN go build -v ./...
# Runtime image
FROM registry.access.redhat.com/ubi8/ubi-micro:latest AS srv
COPY --from=builder /go/src/hello-world/hello-world /
EXPOSE 8080
CMD ["/hello-world"]
In the preceding example, the name of the builder image is set as builder, while the final image is named srv. Interestingly, the COPY instruction can now specify the builder as using the custom name with the --from=builder option.
Dockerfile/Containerfile builds are the most common approach but still lack some flexibility when it comes to implementing a custom build workflow. For those special use cases, Buildah native commands come to our rescue.
Multistage builds with Buildah native commands
As mentioned before, the multistage build feature is a great approach to produce images with a small footprint and a reduced attack surface. To provide greater flexibility during the build process, the Buildah native commands come to our rescue. As we mentioned earlier in Chapter 6, Meet Buildah – Building Containers from Scratch, Buildah offers a series of commands that replicate the behavior of the Dockerfile instructions, thus offering greater control over the build process when those commands are included in scripts or automations.
The same concept applies when working with multistage builds, where we can also apply extra steps between the stages. For instance, we can mount the build container overlay file system and extract the built artifact to release alternate packages, all before building the final runtime image.
The following example builds the same hello-world Go application by translating the previous Dockerfile instructions into native Buildah commands, with everything inside a simple shell script:
#!/bin/bash
# Define builder and runtime images
BUILDER=docker.io/library/golang
RUNTIME=registry.access.redhat.com/ubi8/ubi-micro:latest
# Create builder container
container1=$(buildah from $BUILDER)
# Copy files from host
if [ -f go.mod ]; then
buildah copy $container1 'go.mod' '/go/src/hello-world/'
else
exit 1
fi
if [ -f main.go ]; then
buildah copy $container1 'main.go' '/go/src/hello-world/'
else
exit 1
fi
# Configure and start build
buildah config --workingdir /go/src/hello-world $container1
buildah run $container1 go get -d -v ./...
buildah run $container1 go build -v ./...
# Create runtime container
container2=$(buildah from $RUNTIME)
# Copy files from the builder container
buildah copy --chown=1001:1001
--from=$container1 $container2
'/go/src/hello-world/hello-world' '/'
# Configure exposed ports
buildah config --port 8080 $container2
# Configure default CMD
buildah config --cmd /hello-world $container2
# Configure default user
buildah config --user=1001 $container2
# Commit final image
buildah commit $container2 hello-world
# Remove build containers
buildah rm $container1 $container2
In the preceding example, we highlighted the two working containers' creation commands and the related container1 and container2 variables that store the container ID.
Also, note the buildah copy command, where we have defined the source container with the --from option, and used the --chown option to define user and group owners of the copied resource. This approach proves to be more flexible than the Dockerfile-based workflow, since we can enrich our script with variables, conditionals, and loops.
For instance, we have tested with the if condition in the Bash script to check the existence of the go.mod and main.go files before copying them inside the working container dedicated to the build.
Let's now add an extra feature to the script. In the following example, we evolved the previous one by adding a semantic versioning for the build and creating a version archive before starting the build of the final runtime image:
Important Note
The concept of semantic versioning is aimed to provide a clear and standardized way to manage software versioning and dependency management. It is a set of standard rules whose purpose is to define how software release versions are applied and follows the X.Y.Z versioning pattern, where X is the major version, Y is the minor version, and Z is the patch version. For more information, check out the official specifications: https://semver.org/.
#!/bin/bash
# Define builder and runtime images
BUILDER=docker.io/library/golang
RUNTIME=registry.access.redhat.com/ubi8/ubi-micro:latest
RELEASE=1.0.0
# Create builder container
container1=$(buildah from $BUILDER)
# Copy files from host
if [ -f go.mod ]; then
buildah copy $container1 'go.mod' '/go/src/hello-world/'
else
exit 1
fi
if [ -f main.go ]; then
buildah copy $container1 'main.go' '/go/src/hello-world/'
else
exit 1
fi
# Configure and start build
buildah config --workingdir /go/src/hello-world $container1
buildah run $container1 go get -d -v ./...
buildah run $container1 go build -v ./...
# Extract build artifact and create a version archive
buildah unshare --mount mnt=$container1
sh -c 'cp $mnt/go/src/hello-world/hello-world .'
cat > README << EOF
Version $RELEASE release notes:
- Implement basic features
EOF
tar zcf hello-world-${RELEASE}.tar.gz hello-world README
rm -f hello-world README
# Create runtime container
container2=$(buildah from $RUNTIME)
# Copy files from the builder container
buildah copy --chown=1001:1001
--from=$container1 $container2
'/go/src/hello-world/hello-world' '/'
# Configure exposed ports
buildah config --port 8080 $container2
# Configure default CMD
buildah config --cmd /hello-world $container2
# Configure default user
buildah config--user=1001 $container2
# Commit final image
buildah commit $container2 hello-world:$RELEASE
# Remove build containers
buildah rm $container1 $container2
The key changes in the script are again highlighted in bold. First, we added a RELEASE variable that tracks the release version of the application. Then, we extracted the build artifact using the buildah unshare command, followed by the --mount option to pass the container mount point. The user namespace unshare was necessary to make the script capable of running rootless.
After extracting the artifact, we created a gzipped archive using the $RELEASE variable inside the archive name and removed the temporary files.
Finally, we started the build of the runtime image and committed using the $RELEASE variable again as the image tag.
In this section, we have learned how to run multistage builds with Buildah using both Dockerfiles/Containerfiles and native commands. In the next section, we will learn how to isolate Buildah builds inside a container.
Running Buildah inside a container
Podman and Buildah follow a fork/exec approach that makes them very easy to run inside a container, including rootless containers scenarios.
There are many use cases that imply the need for containerized builds. Nowadays, one of the most common adoption scenarios is the application build workflow running on top of a Kubernetes cluster.
Kubernetes is basically a container orchestrator that manages the scheduling of containers from a control plane over a set of worker nodes that run a container engine compatible with the Container Runtime Interface (CRI). Its design allows great flexibility in customizing networking, storage, and runtimes, and leads to the great flourishing of side projects that are now incubating or matured inside the Cloud Native Computing Foundation (CNCF).
Vanilla Kubernetes (which is the basic community release without any customization or add-ons) doesn't have any native build feature but offers the proper framework to implement one. Over time, many solutions appeared trying to address this need.
For example, Red Hat OpenShift introduced, way back when Kubernetes 1.0 was released, its own build APIs and the Source-to-Image toolkit to create container images from source code directly on top of the OpenShift cluster.
Another interesting solution is Google's kaniko, which is a build tool to create container images inside a Kubernetes cluster that runs every build step inside user space.
Besides using already implemented solutions, we can design our own running Buildah inside containers that are orchestrated by Kubernetes. We can also leverage the rootless-ready design to implement secure build workflows.
It is possible to run CI/CD pipelines on top of a Kubernetes cluster and embed containerized builds within a pipeline. One of the most interesting CNCF projects, Tekton Pipelines, offers a cloud-native approach to accomplish this goal. Tekton allows running pipelines that are driven by Kubernetes' custom resources – special APIs that extend the basic API set.
Tekton Pipelines are made up of many different tasks, and users can either create their own or grab them from Tekton Hub (https://hub.tekton.dev/), a free repository where many pre-baked tasks are available to be consumed immediately, including examples from Buildah (https://hub.tekton.dev/tekton/task/buildah).
The preceding examples are useful to understand why containerized builds are important. In this book, we want to focus on the details of running builds within containers, with special attention paid to security-related constraints.
Running rootless Buildah containers with volume stores
For the examples in this subsection, the stable upstream quay.io/buildah/stable Buildah image will be used. This image already embeds the latest stable Buildah binary.
Let's run our first example with a rootless container that builds the contents of the ~/build directory in the host and stores the output in a local volume named storevol:
$ podman run --device /dev/fuse
-v ~/build:/build:z
-v storevol:/var/lib/containers quay.io/buildah/stable
buildah build -t build_test1 /build
This example brings some peculiar options that deserve attention, as follows:
- The --device /dev/fuse option, which loads the fuse kernel module in the container, which is necessary to run fuse-overlay commands
- The -v ~/build:/build:z option, which bind-mounts the /root/build directory inside the container, assigning proper SELinux labeling with the :z suffix
- The -v storevol:/var/lib/containers option, which creates a fresh volume mounted on the default container store where all the layers are created
When the build is complete, we can run a new container using the same volume and inspect or manipulate the built image:
$ podman run --rm -v storevol:/var/lib/containers quay.io/buildah/stable buildah images
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/build_test1 latest cd36bf58daff 12 minutes ago 283
docker.io/library/fedora latest b080de8a4da3 4 days ago 159 MB
We have successfully built an image whose layers have been stored inside the storevol volume. To recursively list the content of the store, we can extract the volume mount point with the podman volume inspect command:
$ ls -alR
$(podman volume inspect storevol --format '{{.Mountpoint}}')
From now on, it is possible to launch a new Buildah container to authenticate to the remote registry, and tag and push the image. In the next example, Buildah tags the resulting image, authenticates to the remote registry, and finally pushes the image:
$ podman run --rm -v storevol:/var/lib/containers
quay.io/buildah/stable
sh -c 'buildah tag build_test1
registry.example.com/build_test1
&& buildah login -u=<USERNAME> -p=<PASSWORD>
registry.example.com &&
buildah push registry.example.com/build_test1'
When the image is successfully pushed, it is finally safe to remove the volume:
# podman volume rm storevol
Despite working perfectly, this approach has some limits that are worth discussing.
The first limit we can notice is that the store volume is not isolated, and thus any other container can access its contents. To overcome this issue, we can use SELinux's Multi-Category Security (MCS) with the :Z suffix in order to apply categories to the volume and make it accessible exclusively to the running container.
However, since a second container would run by default with different category labels, we should grab the volume categories and run the second tag/push container with the --security-opt label=level:s0:<CAT1>,<CAT2> option.
Alternatively, we can just run build, tag, and push commands in one single container, as shown in the following example:
$ podman run --device /dev/fuse
-v ~/build:/build
-v secure_storevol:/var/lib/containers:Z
quay.io/buildah/stable
sh -c 'buildah build -t test2 /build &&
buildah tag test2 registry.example.com/build_test2 &&
buildah login -u=<USERNAME>
-p=<PASSWORD>
registry.example.com &&
buildah push registry.example.com/build_test2'
Important Note
In the preceding examples, we used the Buildah login by directly passing the username and password in the command. Needless to say, this is far from being an acceptable security practice.
Instead of passing sensitive data in the command line, we can mount the authentication file that contains a valid session token as a volume inside the container.
The next example mounts a valid auth.json file, stored under the /run/user/<UID> tmpfs, inside the build container, and the --authfile /auth.json option is then passed to the buildah push command:
$ podman run --device /dev/fuse
-v ~/build:/build
-v /run/user/<UID>/containers/auth.json:/auth.json:z
-v secure_storevol:/var/lib/containers:Z
quay.io/buildah/stable
sh -c 'buildah build -t test3 /build &&
buildah tag test3 registry.example.com/build_test3 &&
buildah push --authfile /auth.json
registry.example.com/build_test3'
Finally, we have a working example that avoids exposing clear credentials in the commands passed to the container.
To provide a working authentication file, we need to authenticate from the host that will run the containerized build or copy a valid authentication file. To authenticate with Podman, we'll use the following command:
$ podman login –u <USERNAME> -p <PASSWORD> <REGISTRY>
If the authentication process succeeds, the obtained token is stored in the /run/user/<UID>/containers/auth.json file, which stores a JSON-encoded object with a structure similar to the following example:
{
"auths": {
"registry.example.com": {
"auth": "<base64_encoded_token>"
}
}
}
Security Alert!
If the authentication file mounted inside the container has multiple authentication records for different registries, they will be exposed inside the build container. This can lead to potential security issues, since the container will be able to authenticate on those registries using the tokens specified in the file.
The volume-based approach we just described has some small impact on the performance when compared to a native host build but provides better isolation of the build process, a reduced attack surface, thanks to the rootless execution and standardization of the build environment across different hosts.
Let's now inspect how to run containerized builds using bind-mounted stores.
Running Buildah containers with bind-mounted stores
In the highest isolation scenario, where DevOps teams follow a zero-trust approach, every build container should have its own isolated store populated at the beginning of the build and destroyed upon completion. Isolation can be easily achieved with SELinux MCS security.
To test this approach, let's start by creating a temporary directory that will host the build layers. We also want to generate a random suffix for a name in order to host multiple builds without conflicts:
# BUILD_STORE=/var/lib/containers-$(echo $RANDOM | md5sum | head -c 8)
# mkdir $BUILD_STORE
Important Note
The preceding example and the next builds are executed as root.
We can now run the build and bind-mount the new directory to the /var/lib/containers folder inside the container and add the :Z suffix to ensure multi-category security isolation:
# podman run --device /dev/fuse
-v ./build:/build:z
-v $BUILD_STORE:/var/lib/containers:Z
-v /run/containers/0/auth.json:/auth.json
quay.io/buildah/stable
bash -c 'set -euo pipefail;
buildah build -t registry.example.com/test4 /build;
buildah push --authfile /auth.json
registry.example.com/test4'
The MCS isolation guarantees isolation from other containers. Every build container will have its own custom store, and this implies the need to re-pull the base image layers on every execution, since they are never cached.
Despite being the most secure in terms of isolation, this approach also offers the slowest performance because of the continuous pulls on the build run.
On the other hand, the less secure approach does not expect any store isolation, and all the build containers mount the default host store under /var/lib/containers. This approach provides better performance, since it allows the reuse of cached layers from the host store.
SELinux will not allow a containerized process to access the host store; therefore, we need to relax SELinux security restrictions to run the following example using the --security-opt label=disable option.
The following example runs another build using the default host store:
# podman run --device /dev/fuse
-v ./build:/build:z
-v /var/lib/containers:/var/lib/containers
--security-opt label=disable
-v /run/containers/0/auth.json:/auth.json
quay.io/buildah/stable
bash -c 'set -euo pipefail;
buildah build -t registry.example.com/test5 /build;
buildah push --authfile /auth.json
registry.example.com/test5'
The approach described in this example is the opposite of the previous one – better performances but worse security isolation.
A good compromise between the two implies the usage of a secondary, read-only image store to provide access to the cached layers. Buildah supports the usage of multiple image stores, and the /etc/containers/storage.conf file inside the Buildah stable image already configures the /var/lib/shared folder for this purpose.
To prove this, we can inspect the content of the /etc/containers/storage.conf file, where the following section is defined:
# AdditionalImageStores is used to pass paths to additional Read/Only image stores
# Must be comma separated list.
additionalimagestores = [
"/var/lib/shared",
]
This way, we can get good isolation and better performance, since cached images from the host will be already available in the read-only store. The read-only store can be prepopulated with the most used images to speed up builds or can be mounted from a network share.
The following example shows this approach, by bind-mounting the read-only store to the container and executing the build with the advantage of reusing pre-pulled images:
# podman run --device /dev/fuse
-v ./build:/build:z
-v $BUILD_STORE:/var/lib/containers:Z
-v /var/lib/containers/storage:/var/lib/shared:ro
-v /run/containers/0/auth.json:/auth.json:z
quay.io/buildah/stable
bash -c 'set -euo pipefail;
buildah build -t registry.example.com/test6 /build;
buildah push --authfile /auth.json
registry.example.com/test6'
The examples showed in this subsection are also inspired by a great technical article written by Dan Walsh (one of the leads of the Buildah and Podman projects) on the Red Hat Developer blog; refer to the Further reading section for the original article link. Let's close this section with an example of native Buildah commands.
Running native Buildah commands inside containers
We have so far illustrated examples using Dockerfiles/Containerfiles, but nothing prevents us from running containerized native Buildah commands. The following example creates a custom Python image built from a Fedora base image:
# BUILD_STORE=/var/lib/containers-$(echo $RANDOM | md5sum | head -c 8)# mkdir $BUILD_STORE
# podman run --device /dev/fuse
-e REGISTRY=<USER_DEFINED_REGISTRY:PORT>
--security-opt label=disable
-v $BUILD_STORE:/var/lib/containers:Z
-v /var/lib/containers/storage:/var/lib/shared:ro
-v /run/containers/0:/run/containers/0
quay.io/buildah/stable
bash -c 'set -euo pipefail;
container=$(buildah from fedora);
buildah run $container dnf install -y python3 python3;
buildah commit $container $REGISTRY/python_demo;
buildah push –authfile
/run/containers/0/auth.json $REGISTRY/python_demo'
From a performance standpoint as well as the build process, nothing changes from the previous examples. As already stated, this approach provides more flexibility in the build operations.
If the commands to be passed are too many, a good workaround can be to create a shell script and inject it into the Buildah image using a dedicated volume:
# BUILD_STORE=/var/lib/containers-$(echo $RANDOM | md5sum | head -c 8)
# PATH_TO_SCRIPT=/path/to/script
# REGISTRY=<USER_DEFINED_REGISTRY:PORT>
# mkdir $BUILD_STORE
# podman run --device /dev/fuse
-v $BUILD_STORE:/var/lib/containers:Z
-v /var/lib/containers/storage:/var/lib/shared:ro
-v /run/containers/0:/run/containers/0
-v $PATH_TO_SCRIPT:/root:z
quay.io/buildah/stable /root/build.sh
build.sh is the name of the shell script file containing all the build custom commands.
In this section, we have learned how to run Buildah in containers covering both volume mounts and bind mounts. We have learned how to run rootless build containers that can be easily integrated into pipelines or Kubernetes clusters to provide an end-to-end application life cycle workflow. This is due to the flexible nature of Buildah, and for the same reason, it is very easy to embed Buildah inside custom builders, as we will see in the next section.
Integrating Buildah in custom builders
As we saw in the previous section of this chapter, Buildah is a key component of Podman's container ecosystem. Buildah is a dynamic and flexible tool that can be adapted to different scenarios to build brand-new containers. It has several options and configurations available, but our exploration is not yet finished.
Podman and all the projects developed around it have been built with extensibility in mind, making every programmable interface available to be reused from the outside world.
Podman, for example, inherits Buildah capabilities for building brand-new containers through the podman build command; with the same principle, we can embed Buildah interfaces and its engine in our custom builder.
Let's see how to build a custom builder in the Go language; we will see that the process is pretty straightforward, because Podman, Buildah, and many other projects in this ecosystem are actually written in the Go language.
Including Buildah in our Go build tool
As a first step, we need to prepare our development environment, downloading and installing all the required tools and libraries for creating our custom build tool.
In Chapter 3, Running the First Container, we saw various Podman installation methods. In the following section, we will use a similar procedure while going through the preliminary steps for building a Buildah project from scratch, downloading its source file to include in our custom builder.
First of all, let's ensure we have all the needed packages installed on our development host system:
# dnf install -y golang git go-md2man btrfs-progs-devel gpgme-devel device-mapper-devel
Last metadata expiration check: 0:43:05 ago on mar 9 nov 2021, 17:21:23.
Package git-2.33.1-1.fc35.x86_64 is already installed.
Dependencies resolved.
=============================================================================================================================================================================
Package Architecture Version Repository Size
=============================================================================================================================================================================
Installing:
btrfs-progs-devel x86_64 5.14.2-1.fc35 updates 50 k
device-mapper-devel x86_64 1.02.175-6.fc35 fedora 45 k
golang x86_64 1.16.8-2.fc35 fedora 608 k
golang-github-cpuguy83-md2man x86_64 2.0.1-1.fc35 fedora 818 k
gpgme-devel x86_64 1.15.1-6.fc35 updates 163 k
Installing dependencies:
[... omitted output]
After installing the Go language core libraries and some other development tools, we are ready to create the directory structure for our project and initialize it:
$ mkdir ~/custombuilder
$ cd ~/custombuilder
[custombuilder]$ export GOPATH=`pwd`
As shown in the previous example, we followed these steps:
- Created the project root directory
- Defined the Go language root path that we are going to use
We are now ready to create our Go module that will create our customized container image with a few easy steps.
To speed up the example and avoid any writing errors, we can download the Go language code that we are going to use for this test from the official GitHub repository of this book:
- Go to https://github.com/PacktPublishing/Podman-for-DevOps or run the following command:
$ git clone https://github.com/PacktPublishing/Podman-for-DevOps
- After that, copy the files provided in the Chapter07/* directory into the newly created ~/custombuilder/ directory.
You should have the following files in your directory at this point:
$ cd ~/custombuilder/src/builder
$ ls -latotal 148
drwxrwxr-x. 1 alex alex 74 9 nov 15.22 .
drwxrwxr-x. 1 alex alex 14 9 nov 14.10 ..
-rw-rw-r--. 1 alex alex 1466 9 nov 14.10 custombuilder.go
-rw-rw-r--. 1 alex alex 161 9 nov 15.22 go.mod
-rw-rw-r--. 1 alex alex 135471 9 nov 15.22 go.sum
-rw-rw-r--. 1 alex alex 337 9 nov 14.17 script.js
At this point, we can run the following command to let the Go tools acquire all the needed dependencies to ready the module for execution:
$ go mod tidy
go: finding module for package github.com/containers/storage/pkg/unshare
go: finding module for package github.com/containers/image/v5/storage
go: finding module for package github.com/containers/storage
go: finding module for package github.com/containers/image/v5/types
go: finding module for package github.com/containers/buildah/define
go: finding module for package github.com/containers/buildah
go: found github.com/containers/buildah in github.com/containers/buildah v1.23.1
go: found github.com/containers/buildah/define in github.com/containers/buildah v1.23.1
go: found github.com/containers/image/v5/storage in github.com/containers/image/v5 v5.16.1
go: found github.com/containers/image/v5/types in github.com/containers/image/v5 v5.16.1
go: found github.com/containers/storage in github.com/containers/storage v1.37.0
go: found github.com/containers/storage/pkg/unshare in github.com/containers/storage v1.37.0
The tool analyzed the provided custombuilder.go file, and it found all the required libraries, populating the go.mod file.
Important Note
Please be aware that the previous command will verify whether a module is available, and if it is not, the tool will start downloading it from the internet. So, be patient during this step!
We can check that the previous commands downloaded all the required packages by inspecting the directory structure we created earlier:
$ cd ~/custombuilder
[custombuilder]$ ls
pkg src
[custombuilder]$ ls -la pkg/
total 0
drwxrwxr-x. 1 alex alex 28 9 nov 18.27 .
drwxrwxr-x. 1 alex alex 12 9 nov 18.18 ..
drwxrwxr-x. 1 alex alex 20 9 nov 18.27 linux_amd64
drwxrwxr-x. 1 alex alex 196 9 nov 18.27 mod
[custombuilder]$ ls -la pkg/mod/
total 0
drwxrwxr-x. 1 alex alex 196 9 nov 18.27 .
drwxrwxr-x. 1 alex alex 28 9 nov 18.27 ..
drwxrwxr-x. 1 alex alex 22 9 nov 18.18 cache
drwxrwxr-x. 1 alex alex 918 9 nov 18.27 github.com
drwxrwxr-x. 1 alex alex 24 9 nov 18.27 go.etcd.io
drwxrwxr-x. 1 alex alex 2 9 nov 18.27 golang.org
[... omitted output]
[custombuilder]$ ls -la pkg/mod/github.com/
[... omitted output]
drwxrwxr-x. 1 alex alex 98 9 nov 18.27 containerd
drwxrwxr-x. 1 alex alex 20 9 nov 18.27 containernetworking
drwxrwxr-x. 1 alex alex 184 9 nov 18.27 containers
drwxrwxr-x. 1 alex alex 110 9 nov 18.27 coreos
[... omitted output]
We are now ready to run our custom builder module, but before going forward, let's take a look at the key elements contained in the Go source file.
If we start looking at the custombuilder.go file, just after defining the package and the libraries to use, we defined the main function of our module.
In the main function, at the beginning of the definition, we inserted a fundamental code block:
if buildah.InitReexec() {
return
}
unshare.MaybeReexecUsingUserNamespace(false)
This piece of code enables the usage of rootless mode by leveraging the Go unshare package, available through github.com/containers/storage/pkg/unshare.
To leverage the build features of Buildah, we have to instantiate buildah.Builder. This object has all the methods to define the build steps, configure the build, and finally run it.
To create Builder, we need an object called storage.Store from the github.com/containers/storage package. This element is responsible for storing the intermediate and resultant container images. Let's see the code block we are discussing:
buildStoreOptions, err := storage.DefaultStoreOptions(unshare.IsRootless(), unshare.GetRootlessUID())
buildStore, err := storage.GetStore(buildStoreOptions)
As you can see from the previous example, we are getting the default options and passing them to the storage module to request a Store object.
Another element we need for creating Builder is the BuilderOptions object. This element contains all the default and custom options we might assign to Buildah's Builder. Let's see how to define it:
builderOpts := buildah.BuilderOptions{
FromImage: "node:12-alpine", // Starting image
Isolation: define.IsolationChroot, // Isolation environment
CommonBuildOpts: &define.CommonBuildOptions{},
ConfigureNetwork: define.NetworkDefault,
SystemContext: &types.SystemContext {},
}
In the previous code block, we defined a BuilderOptions object that contains the following:
- An initial image that we are going to use to build our target container image:
- In this case, we chose the Node.js image based on Alpine Linux distribution. This is because, in our example, we are simulating the build process of a Node.js application.
- Isolation mode to adopt once the build starts. In this case, we are going to use chroot isolation that fits a lot of build scenarios well – less isolation but fewer requirements.
- Some default options for the build, network, and system contexts:
- SystemContext objects define the information contained in configuration files as parameters.
Now that we have all the necessary data for instantiating Builder, let's do it:
builder, err := buildah.NewBuilder(context.TODO(), buildStore, builderOpts)
As you can see, we are calling the NewBuilder function, with all the required options that we created in code earlier in this section, to get Builder ready to create our custom container image.
Now that we are ready to instruct Builder with the required options to create the custom image, let's first add into the container image the JavaScript file containing our application, for which we are creating this container image:
err = builder.Add("/home/node/", false, buildah.AddAndCopyOptions{}, "script.js")
We are assuming that the JavaScript main file is stored next to the Go module that we are writing and using in this example, and we are copying this file into the /home/node directory, which is the default path where the base container image expects to find this kind of data.
The JavaScript program that we are going to copy into the container image and use for this test is really simple – let's inspect it:
var http = require("http");
http.createServer(function(request, response) {
response.writeHead(200, {"Content-Type": "text/plain"});
response.write("Hello Podman and Buildah friends. This page is provided to you through a container running Node.js version: ");
response.write(process.version);
response.end();
}).listen(8080);
Without going deep into the JavaScript language syntax and its concepts, we can note looking at the JavaScript file that we are using the HTTP library for listening on port 8080 for incoming requests, responding to these requests with a default welcome message: Hello Podman and Buildah friends. This page is provided to you through a container running Node.js. We also append the Node.js version to the response string.
Important Note
Please consider that JavaScript, also known as JS, is a high-level programming language that is compiled just in time. As we stated earlier, we are neither going deep into the definition of the JavaScript language nor its most famous runtime environment, Node.js.
After that, we configure the default command to run for our custom container image:
builder.SetCmd([]string{"node", "/home/node/script.js"})
We just set the command to execute the Node.js execution runtime, referring to the JavaScript program that we just added to the container image.
For committing the changes we made, we need to get the image reference that we are working on. At the same time, we will also define the container image name that Builder will create:
imageRef, err := is.Transport.ParseStoreReference(buildStore, "podmanbook/nodejs-welcome")
Now, we are ready to commit the changes and call the commit function of Builder:
imageId, _, _, err := builder.Commit(context.TODO(), imageRef, define.CommitOptions{})
fmt.Printf("Image built! %s ", imageId)
As we can see, we just requested Builder to commit the changes, passing the image reference we obtained earlier, and then we finally print it as a reference.
We are now ready to run our program! Let's execute it:
[builder]$ go run custombuilder.go
Image built! e60fa98051522a51f4585e46829ad6a18df704dde774634dbc010baae440 4849
We can now test the custom container image we just built:
[builder]$ podman run -dt -p 8080:8080/tcp podmanbook/nodejs-welcome:latest
747805c1b59558a70c4a2f1a1d258913cae5ffc08cc026c74ad3ac21aab1 8974
[builder]$ curl localhost:8080
Hello Podman and Buildah friends. This page is provided to you through a container running Node.js version: v12.22.7
As we can see in the previous code block, we are running the container image we just created with the following options:
- -d: Detached mode, which runs the container in the background
- -t: Allocates a new pseudo-TTY
- -p: Publishes the container port to the host system
- podmanbook/nodejs-welcome:latest: The name of our custom container image
Finally, we use the curl command-line tool for requesting and printing the HTTP response provided by our JavaScript program, which is containerized in the custom container image that we created!
Important Note
The example described in this section is just a simple overview of all the great features that the Buildah Go module can enable for our custom image builders. To learn more about the various functions, variables, and code documentation, you can refer to the docs at https://pkg.go.dev/github.com/containers/buildah.
As we saw in this section, Buildah is a really flexible tool, and with its libraries, it can support custom builders in many different scenarios.
If we try to search on the internet, we can find many examples of Buildah supporting the creation of custom container images. Let's see some of them.
Quarkus-native executables in containers
Quarkus is defined as the Kubernetes-native Java stack leveraging OpenJDK (the open Java development kit) project and the GraalVM project. GraalVM is a Java virtual machine that has many special features, such as the compilation of Java applications for fast startup and low memory footprint.
Important note
We will not go into the details of Quarkus, GraalVM, and any other companion projects. The example that we will deep-dive into is only for your reference. We encourage you to learn more about these projects by going through their web pages and reading the related documentation.
If we take a look at the Quarkus documentation web page, we can easily find that, after a long tutorial in which we can learn how to build a Quarkus-native executable, we can then pack and execute this executable in a container image.
The steps provided in the Quarkus documentation leverage a Maven wrapper with a special option. Maven was born as a Java build automation tool, but then it was also extended to other programming languages. If we take a quick look at this command, we will note the name of Podman inside:
$ ./mvnw package -Pnative -Dquarkus.native.container-build=true -Dquarkus.native.container-runtime=podman
This means that the Maven wrapper program will invoke a Podman build to create a container image with the preconfigured environment shipped by the Quarkus project and the binary application that we are developing.
We saw the name of Podman inside the option. This is because, as we saw in Chapter 6, Meet Buildah – Building Containers from Scratch, Podman borrows Buildah's build logic by vendoring its libraries.
To explore this example further, we can take a look at https://quarkus.io/guides/building-native-image.
A Buildah wrapper for the Rust language
Another cool example of build tools made through the Buildah library or CLI is the Buildah wrapper for the Rust programming language. Rust is a programming language similar to C++, designed for performance and safe concurrency. The main project page is available at this URL: https://github.com/Dennis-Krasnov/Buildah-Rust.
This Buildah wrapper leverages the Rust package manager names Cargo for downloading the needed dependencies, compiles it in a package, and makes it distributable.
Important Note
We will not go into the details of Rust, Cargo, and any other companion projects. The example that we will deep-dive into is only for your reference. We encourage you to learn more about these projects by going through their web pages and reading the related documentation.
The example in the project homepage is really simple, as you can see in the following code block:
$ cd examples/
$ cargo run --example nginx
$ podman run --rm -it -p 8080:80 nginx_rust
The first command, after selecting the directory named examples, executes a simple block of code that is needed to create a container, while the second tests the container image that the Buildah wrapper has just made through Buildah itself.
We can take a look at the Rust code used in the first command of the previous code block. The first command executes the small piece of code in the nginx.rs file:
use buildah_rs::container::Container;
fn main() {
let mut container = Container::from("nginx:1.21");
container.copy("html", "/usr/share/nginx/html").unwrap();
container.commit("nginx_rust").unwrap();
}
As stated before, we will not dive deep into the code syntax or into the library itself; anyway, the code is pretty simple, and it just imports the Buildah wrapper library, creates a container image starting from nginx:1.21, and finally, copies the local html directory to the container image's destination path.
To explore this example further, take a look at https://github.com/Dennis-Krasnov/Buildah-Rust.
This concludes this section. We have learned, through a lot of useful examples, about how to integrate Buildah in different scenarios to support custom builders of the container images of our projects.
Summary
In this chapter, we have learned how to leverage Podman's companion, Buildah, in some advanced scenarios to support our development projects.
We saw how to use Buildah for multistage container image creation, which allows us to create builds with multiple stages using different FROM instructions and, subsequently, to have images that grab contents from the previous ones.
Then, we discovered that there are many use cases that imply the need for containerized builds. Nowadays, one of the most common adoption scenarios is the application build workflow running on top of a Kubernetes cluster. For this reason, we went into the details of containerizing Buildah.
Finally, we learned through a lot of interesting examples how to integrate Buildah to create custom builders for container images. As we saw in this chapter, there are several options and methods to actually build a container image with the Podman ecosystem tools, and most of the time, we usually start from a base image for customizing and extending a previous OS layer to fit our use cases.
In the next chapter, we will learn more about container base images, how to choose them, and what to look out for when we are making our choice.
Further readings
- A list of CNCF projects: https://landscape.cncf.io/
- Best practices for running Buildah in a container: https://developers.redhat.com/blog/2019/08/14/best-practices-for-running-buildah-in-a-container
- The Buildah Go module documentation: https://pkg.go.dev/github.com/containers/buildah
- Quarkus-native executables: https://quarkus.io/guides/building-native-image
- The Buildah wrapper for the Rust language: https://github.com/Dennis-Krasnov/Buildah-Rust