Hello LEAF Community,
With the release of LEAF 3.8, our team made some changes to our dev containers: Kind is now the default local Kubernetes (k8s) solution for LEAFctl, instead of minikube. This change has occurred because minikube does not interact well with dev containers. Previously, we offered a workaround that allowed minikube to run from within our dev containers; however, on later versions of Docker, yet another workaround is needed. Instead of trying to force minikube to work, we have opted to use Kind which interacts much better with dev containers. Additionally, the version of Docker Desktop used by Leidos developers is often determined by what is requested from the Leidos Service Catalog. This means we can’t easily lock our users to a particular version(s) of Docker, nor do we want to do that; we recognize how important Docker is for software development and that not every user of LEAF only uses LEAF. Every other tool we recommend can be self-contained within our dev containers, but Docker is an exception.
Running minikube within a dev container has continued to be a challenge and kind generally works better for this use case. At this time, LEAFctl supports both cluster types; however, Kind will now be the default as it seems to be lighter, faster, and easier to use in a typical LEAF development workflow.
Currently, much of our documentation and training material are built around using Minikube as the local k8s option; with this change, we will continue to update our documentation and training material.
What does this mean for you?
If you are using LEAF 3.8+ (leafctl, generators, documentation, training), we’ve made this a seamless change. If the leafctl k8s start command started a Minikube cluster in LEAF 3.7, the same command will start a Kind cluster in LEAF 3.8+.
THIS DOES NOT MEAN WE NO LONGER SUPPORT MINIKUBE, only that Kind is now the default. If you still wish to use Minikube outside of a dev container, you simply need to apply the --cluster flag. This same flag can also be applied for the few deploy options which differ between Minikube and Kind (metrics-server and nginx-ingress-controller are “add-ons” in Minikube).
leafctl k8s start --cluster minikube
leafctl deploy metrics-server nginx --cluster minikube
Users can also see what commands equate to with the --dry-run flag.
Multi-architecture images
If you generate a LEAF service in 3.8+, it ships with the ability to build/deploy a container image that is specific to your machine’s architecture. One difference to note when using Kind instead of Minikube is that you can’t load ARM images into an AMD Kind cluster and vice versa.
This can be beneficial because, if you load an image of a different architecture to your cluster, you can experience unwanted or confusing behavior and, very often, will see performance issues. Because Kind stops you from doing this, you can avoid excess troubleshooting.
We recognize this is different behavior than our users have previously known. Minikube operates using its own Docker daemon and images are built directly to its daemon instead of the host daemon. Kind/Kubernetes, on the other hand, uses containerd under the hood. Images are built against the host docker daemon and loaded into the cluster. More importantly, older versions of LEAF-generated services had their image building configuration hard-coded to a specific image/platform (AMD). If you are on an ARM machine, trying to load an AMD image into your ARM Kind cluster, an error will be throw similar to the following:
Command Output: ctr: image might be filtered out
If you are using a LEAF generator that has been updated to produce a build file/Dockerfile that supports dynamic image building based on architecture, you should remain unaffected. If you are on an AMD machine, you also shouldn’t notice this, regardless of the generator you used.
It’s important to note that, while Kind prevents you from loading images of a different architecture into your cluster (kind image load, skaffold build, skaffold run), this DOES NOT mean that you can’t operate AMD images in an ARM cluster or vice versa. If your deployment specifies an image, whether ARM or AMD, and it’s either referencing a public container registry, or you have an image pull secret(s) to authenticate against a private container registry, this will work with no issue. However, there is no guarantee that you won’t see performance issues or unexpected behavior, but k8s will not stop you from pulling the image; Kind disallows the loading of images of a different architecture, not Kubernetes.
Use ironbank images
Another important note, if you’ve chosen to use ironbank images in your generated service and you are on an ARM machine, LEAF will default you to use arm64v8/openjdk:11-jre-slim for backend services while doing local development as it doesn’t seem as though ironbank has ARM support for many images. Additionally, the react-app generator has been updated to use the nginxinc/nginx-unprivileged:1.19.6-alpine image by default for local development. The production profile has been updated to target the docker-group.leidos-leaf.com/ironbank/opensource/nginx/nginx:1.19.6 image, if you choose to use ironbank images.
If you have any questions about these changes, let us know! Our commitment to responding to your questions and helping you solve problems will never change. 