Test your configuration files

4 min readSep 26, 2020

So you have been following Test Driven Development and have a steady and reliable suite of test cases for your application. All your tests in your CICD pipeline work well and you have good sensible test coverages to show for it in your tool of choice. Now you start looking into the git repo you have your application code in and you realize that you have a lot of configuration files in your repo.

A few HCL config files for terraform-ing your infrastructure
A Dockerfile
A few YAML files for your Kubernetes deployments.

Below is a sample deployment YAML from the Kubernetes documentation. Anyone familiar with the K8s deployment YAMLs would notice a few things missing from the spec including the securityContext , memory & cpu limits which are vital to improve the security posture.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image:  nginx:1.14.2
        ports:
        - containerport: 80

As I started writing these configuration files for deployment, I wanted to find some good practices around it. Talking to the security folks around the team helped me understand the things I should include at a minimum like avoiding privileged escalation or ensuring there is a resource request/limit set for some of these configuration files. How do I ensure that my learnings are documented for everyone else who follows ? I could very well write these down in a wiki and assume they will be reviewed while producing new manifests.

How would you add some validations as the number of services you are creating grows and you want to add some standard validations before they go through the stages of deployment ? Kubesec was the first thing I noticed which does some of these security best practice validations.

I preferred to have a validation mechanism in place in the code or pipeline itself which could help me do this in practice and not get ahead with deployments with these vulnerabilities opened up.One of the easy options is using Conftest.

What is Conftest ?

Conftest is a utility to help you write tests against structured configuration data. Conftest relies on the Rego language from Open Policy Agent for writing policies. If you’re unsure what exactly a policy is, or unfamiliar with the Rego policy language, the Policy Language documentation provided by the Open Policy Agent documentation site is a great resource to read.

Installation

Conftest is available for Windows, macOS and Linux on the releases page. On Linux and macOS you can download as follows:

$ wget https://github.com/open-policy-agent/conftest/releases/download/v0.21.0/conftest_0.21.0_Linux_x86_64.tar.gz
$ tar xzf conftest_0.21.0_Linux_x86_64.tar.gz
$ sudo mv conftest /usr/local/bin

How can it be used

The policy language is easy to understand once you stitch up the path to the files or configuration elements to validate. Let us take an example for a Kubernetes deployment YAML.

Let us start with a file with an extension “.rego” and add some of the examples Conftest already have in their examples list.

package maindeny[msg] {
  input.kind == "Deployment"
  not input.spec.template.spec.securityContext.runAsNonRootmsg := "Containers must not run as root"
}

Conftest looks for deny, violation, and warn rules. The above “policy” is looking for a specific element called “runAsNonRoot” in the element tree below in a kind of a resource called” Deployment”.

spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image:  nginx:1.14.2
    securityContext:
      runAsUser: 2000

You can keep all of these policy rules in a folder called “policy” where Conftest looks by default or you can specify the path using a runtime argument if needed.

How do you run it ?

Once the installation is complete you can run the below command in your terminal.

conftest test ./deployment.yaml

I have a sample K8s YAML and a few examples of the policy rules you can write as part of your validations here. Some of these rules were readily available in the Conftest examples and creating others wasn’t such a big task.

If you are getting used to the Rego language or specification, you can use the Rego Playground to test your rego policies before integrating it with your suite.

Validation results

Below are the results of running tests on my original YAML. It gives me an idea what Iam missing from the deployment manifest I have created.

Advantages:

Security policies are not in someone’s head
Easy integration with any CICD tool ( GitHub Actions in this case)
Ability to extend this as a pod admission controller in K8s if you need to prevent pods being scheduled with any of the security constraints missed
Easy to read and understand
Involve compliance and security early
Supporting a huge number of configuration types including HCL, Dockerfile, K8S yamls, SAM templates and so on