Nickez/slim down docker image

[NickeZ]

1. Use python3 interpreter instead of python which is ambiguous.

2. All temporary files need to be deleted in every RUN command to avoid adding it in a layer. Removing them in a later RUN command will still make them bloat the final image.

   Environment variables that are intended only for when the container is built should be prepended on the lines where they are needed to avoid polluting the environment in the running container.

[benma]

utACK, but prefer to keep cargo install for the two rust tools.

If your goal is to reduce layers, you could also move all of it to a docker_install.sh script and call that in a single RUN. @cstenglein has done that in this open PR #992, but somehow that PR was never merged.

[NickeZ]

utACK, but prefer to keep cargo install for the two rust tools.

If your goal is to reduce layers, you could also move all of it to a docker_install.sh script and call that in a single RUN. @cstenglein has done that in this open PR #992, but somehow that PR was never merged.

I have a few goals I guess:

• Reducing number of layers (this PR cuts layers in half, which I think is a decent improvement)
• Reducing size of individual layers (Remove all temporary files in the same layer they are created so that they don't end up in the final container, mount files that are only needed temporarily)
• Working towards a functionally reproducible container.
  • What is left here is to pin the ubuntu base image and remove apt upgrade. And then probably move all apt installs to a new "base image" that we almost never rebuild.
• Increasing speed of building image. This is good when you are building many images, like if you are testing all different GCC toolchains. 2 minutes saved per build, might make you save 20 minutes if you test 10 different versions.

I do think you can take the released versions of bindgen and cbindgen where available. We aren't building any other tools from source. Perhaps we could build our own linux/arm64 versions and publish somewhere in case you don't want to have two approaches in the dockerfile.

The reason a reproducible container is important is so that we can build older versions of the firmware at all. Today, since the image isn't reproducible we depend on docker hub having a copy of all the versions. You can imagine some future version of a debian or pip package after some "security update" isn't compatible anymore.

I think calling out to a single install.sh script that installs everything is a bit extreme. If you then for example want to try out different toolchains or something you have to rebuild the whole image and cannot use any caching at all. Defeats one of the purposes of docker. I guess it depends a bit on what you do in the install script.

[benma]

The reason a reproducible container is important is so that we can build older versions of the firmware at all.

That would be nice, but I would rank this as low priority for two reasons:

1. we address this problem directly here and gather signatures for a historic record

What is the purpose of this?

The purpose is twofold:

• [...]
  The firmware binaries are only practically reproducible for a limited amount of time after the release. In the future, the dependencies of our Dockerfile might have changed, or Docker itself might even become incompatible or obsolete. This historic record allows you to be reasonably sure that the released binary was created correctly, even if it becomes infeasible to re-build the binary.

2. Failing to reproduce old firmwares has never come up as an issue so far. Users I guess are interested the reproducibility of the firmware they want to run, which is usually the latest version.

[benma]

utACK