faas-cli using buildkit and / or --cache-from Docker options for CI build process optimization

[antoine-em]

My actions before raising this issue

• Followed the troubleshooting guide
• Read/searched the docs
• Searched past issues

How to use buildkit cache optimization in combination with --cache-from argument option to accelerate the build with faas-cli in a CI pipeline (I am trying it with Github actions)

Expected Behaviour

Use remote registry cached layers to build only the necessary changes and not the whole image each time the build process is launched from faas-cli in a CI pipeline

Current Behaviour

The image is built from scratch which takes quite some time.

Possible Solution

Allow --cache-from argument and buildkit

Steps to mimic Reproduce (for bugs)

1. docker build -t myname/myapp --build-arg BUILDKIT_INLINE_CACHE=1 .
2. docker push myname/myapp
3. docker build --cache-from myname/myapp .

Source : Specifying external cache sources

Context

In a CI process like Github action the Docker image build process is not using (by default for what I could check) any cache so the image is built from the ground.
As my openfaas functions are using pandas python library some steps in the docker build process are taking a very long time.
But those layers are not changing frequently, so locally on the workstation as the cache is used the build process is fast but not on the CI pipeline as all the layers are rebuilt.

Your Environment

• FaaS-CLI version ( Full output from: faas-cli version ): 0.12.14

• Docker version docker version (e.g. Docker 17.0.05 ): Github action Docker version so I am not sure exactly

• Are you using Docker Swarm or Kubernetes (FaaS-netes)? Target is Kubernetes but it doesn't matter here I guess

• Operating System and version (e.g. Linux, Windows, MacOS): Github CI

• Code example or link to GitHub repo or gist to reproduce problem:

• Other diagnostic information / logs from troubleshooting guide

If using Swarm, run the following and include the output:

curl -sLS https://raw.githubusercontent.com/moby/moby/master/contrib/check-config.sh | bash

Next steps

You may join Slack for community support.

[LucasRoesler]

The build arg BUILDKIT_INLINE_CACHE can be supported today via the existing build arg support:
https://docs.openfaas.com/reference/yaml/#function-build-args-build-args

This will require that you have enabled Buildkit by default in your Docker, for example setting the env variable DOCKER_BUILDKIT=1

The --cache-from flag is not currently supported, but you could use the --shrinkwrap support via option 3 here https://docs.openfaas.com/cli/build/#build-functions. See also https://docs.openfaas.com/reference/cicd/intro/

Because you mention Github Actions, checkout our blog post https://www.openfaas.com/blog/openfaas-functions-with-github-actions/. You can see it also using shrinkwrap to prepare the function build context and then enabling and using buildx with the docker/setup-buildx-action@v1 action.

[antoine-em]

@LucasRoesler

Thank you very much for your help.
I could use successfully the cache with my remote registry on AWS ECR and that's a huge gain of time (~1min30s instead of my previous ~30min... to build this openfaas function)

As for the env variables for buildkit I didn't had to change anything in my Github action workflow but I have added those 2 instructions for using the cache properly with this action.

${{ steps.login-ecr.outputs.registry }} is the output of another action to connect to ECR aws-actions/amazon-ecr-login@v1

- name: Build and Push the OpenFaaS function
  uses: docker/build-push-action@v2
  ...
  with:
    ...
    cache-from: type=registry,ref=${{ steps.login-ecr.outputs.registry }}/my/path/openfaas_fn:latest
    cache-to: type=inline

[alexellis]

Why did your build take 30 minutes?

[antoine-em]

@alexellis

My docker image contains some instructions to pip install pandas and as it required some rebuild it took a long time.
So using the cache, as this layer is not changing, I can prevent having a full rebuild of the whole image for this function.

Maybe a better approach would be to build a base image including my requirements but I need to study more about how to use a private function store I can pull.

To be more precise:

• ~30 min. was for a built from scratch
• ~1 min. 30s. with the cache used for all layers

[alexellis]

The idiomatic way to solve this problem would be to create a base image with pandas already installed

[antoine-em]

I will do at some point but as the base images in openfaas come from the template store I guess I will need to create at first my own template including my requirements and push it in a store.
The difficulty here was for me to create the necessary environment to store and pull (if not public) rather than using the docker registry straight away.
But I will try to have a look at it when getting more experienced with openfaas templating.
Thank you.