The Center for Internet Security (CIS) is a non-profit community-driven organization focused on improving the security posture of technology with industry best practices. These best practices are then documented and shared freely in CIS Benchmarks such this one for Kubernetes container orchestration. As part of our continuing effort to help vendors ship software securely to their customers, we added a new feature that enables you to create a Kubernetes node that meets CIS benchmark best practices. Let’s take a look at the results of this configuration when we created a new Kubernetes cluster for an application using kURL as our installer.
What CIS benchmark results can show about security
Here’s an example of `kube-bench` test results, using version 0.6.2 from Aqua Security’s GitHub repo, on a fresh install of Kubernetes via kURL with CIS compliance configuration turned on:
[.pre][INFO] 1 Master Node Security Configuration
[INFO] 1.1 Master Node Configuration Files
[PASS] 1.1.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Automated)
[PASS] 1.1.2 Ensure that the API server pod specification file ownership is set to root:root (Automated)
[PASS] 1.1.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Automated)
[PASS] 1.1.4 Ensure that the controller manager pod specification file ownership is set to root:root (Automated)
[PASS] 1.1.5 Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Automated)
[PASS] 1.1.6 Ensure that the scheduler pod specification file ownership is set to root:root (Automated)
[PASS] 1.1.7 Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Automated)
[PASS] 1.1.8 Ensure that the etcd pod specification file ownership is set to root:root (Automated)
[WARN] 1.1.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Manual)
[WARN] 1.1.10 Ensure that the Container Network Interface file ownership is set to root:root (Manual)
[PASS] 1.1.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Automated)
[FAIL] 1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated)
[PASS] 1.1.13 Ensure that the admin.conf file permissions are set to 644 or more restrictive (Automated)
[PASS] 1.1.14 Ensure that the admin.conf file ownership is set to root:root (Automated)
[PASS] 1.1.15 Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Automated) [.pre]
How to remediate warnings and failed tests from kube-bench
A keen reader might notice that not all the automated checks in kube-bench passed on this first try. That’s concerning, but the good news is that kube-bench immediately provided advice for remediation of the failures and warnings, as shown here:
[.pre]== Remediations master ==1.1.9 Run the below command (based on the file location on your system) on the master node.For example,chmod 644 <path/to/cni/files>
1.1.10 Run the below command (based on the file location on your system) on the master node.For example,chown root:root <path/to/cni/files>
1.1.12 On the etcd server node, get the etcd data directory, passed as an argument --data-dir,from the below command:ps -ef | grep etcdRun the below command (based on the etcd data directory found above).
For example, chown etcd:etcd /var/lib/etcd[.pre]
Which gives us these results:
[.pre]== Summary master ==54 checks PASS1 checks FAIL9 checks WARN0 checks INFO[.pre]
This is super helpful as it clearly recommends what should be changed in the configuration to make the installed Kubernetes environment more secure for customers, along with a summary as a quick indication.
Now the results shown above were only a small, truncated portion of the tests run, and only looked at the Master Node security configuration. Our overall test returned:
[.pre]== Summary total ==82 checks PASS1 checks FAIL40 checks WARN0 checks INFO [.pre]
We see the complete test reported 82 checks passed, 1 failed, and 40 warnings. Of course, you should always try to be as comprehensive as possible in applying security controls, though sometimes there may be necessary exceptions. The point is that we can quickly scan and analyze to identify whether the software being installed is secure, adjust as needed, and make informed decisions on how to fix issues.
So acting on this information, we updated our configuration and then re-ran the tests using the latest version of kubebench, which is version 0.6.8 (as of this writing.) We also updated our testing environment to always use the latest version of kube-bench; this will help us and our vendors remain confident in knowing that we’re up to date on the CIS benchmarks since new releases with the kubernetes installer will be tested against the latest version of kube-bench. Bonus: we can make more quick fixes should we ever fail another test. Yay!
[.pre]== Summary total ==81 checks PASS0 checks FAIL44 checks WARN0 checks INFO [.pre]
How to turn on CIS benchmarks for Kubernetes installed with kURL
If you’re curious, here’s the kURL spec used in the example above, as shown in the Replicated vendor portal:
If you look closely, you’ll spot that it was easy to trigger the CIS benchmark configuration for Kubernetes by simply adding ‘cisCompliance: true’ to our kubernetes specification.
We hope you’ll find this new capability a valuable tool in improving the secure delivery of Kubernetes software to your customers. Note: This feature has been tested with kURL upgrades, however we strongly recommend testing this with your development environments prior to upgrading production.
Check out our docs to learn more about using CIS compliance testing with Replicated.