This is a continuation of my previous post where I talked about the challenges of using serverless/Function as a Service (FaaS) compute systems for ETL (Extract, Transform, Load) jobs. It sat in my drafts folder for a long time, so I just decided to publish it as is.

I used to work at AWS, and predictably we used a lot of AWS cloud services. In many cases, when an engineer looks for a service platform, they’ll often go directly to AWS Lambda because “it’s Serverless” with the justification that it’s simple and alternatives are too complex and not worth considering.

What’s in a Docker image? Nobody knows! Unless, you use an SBOM. What is an SBOM you say? Source Bill of Materials is a file containing all of the resources and tools that were used to build or are included in a software release artifact. SBOMs can be used to identify vulnerable software or even ensure compliance with software licenses. Imagine being able to block vulnerable software using a Kyverno policy.

I wanted to generate a Docker image that contained .net AOT compiled application and also generate an SBOM. Unfortunately, it wasn’t as straight forward as generating it from the output Docker image because it didn’t contain any build packages or any dotnet packages.

Additionally, the build tools have just as much of a role to play in software security as the libraries that end up in the artifact. For example, there have been several supply chain attacks that injected code into a binary or modified release artifacts using stole tokens. An SBOM should contain the build tools too, which simply running Syft won’t give you.

Kyverno is a handy policy engine for Kubernetes. I frequently use to validate or mutate resources that are created.

However, the YAML format can be a bit tricky for some of my more complicated policies. LLMs do alright creating many policies, but can struggle with my more complicated policies because they have to one-shot the policy and create it in one pass with no feedback or validation.

In this post, I’m going to show how I configure OpenWebUI to help me draft Kyverno policies.

As I switch from GitHub to Forgejo for my various project repositories, I find myself looking for a way to securely deploy new services into my Kubernetes cluster without having to keep updating Kubernetes auth tokens and storing them as a workflow secret.

GitHub Actions and, as of Forgejo v15, Forgejo Actions also supports providing temporary JWT tokens to workflow steps that securely state what repository and other commits details a given build is for. When a workflow is running, it can pass it to Kubernetes and other supported services and authenticate as a secure client with all the identity information of that repository and workflow. This is called OpenID Connect or OIDC.

This is super powerful because can federate identity and minimize the number of secrets required–great the next time a widely used workflow step gets compromised and steps all your secrets. Having fewer secrets helps.

In this post, I’m going to show I configured Nix, Kubernetes, and Forgejo to securely update a deployment.

In my Kubernetes, I sometimes try to run a pod on a specific worker node. Maybe one of them has a folder that I need or a specific hardware characteristic. Historically, I’ve used Pod spec.nodeName: srv5 However, when that node becomes unavailable, say because it’s run out of disk space and has DiskPressure on it, then Kubernetes will continually try to spin up thousands of pods on it.

A screenshot of CPU usage growing, then Prometheus falls over and can’t scrap anymore.