While working with GKE cluster, how do you define rules to keep your cluster in compliance. For example – you may want that all pods must specify resource limits or say, all deployments should have a specific label. It can be achieved by something called as a policy. So why do you need a policy? As a company, you could be dealing with sensitive domain where applications or services may require secured and regulated environment – for example – banking and financial domain. For services in such a domain, you would want to setup policies within you cluster that meets governance and compliance requirements. Even if you are not dealing with sensitive applications, you may still want to apply policies to control your cluster environment and setup best practice guidelines around your organisation security framework.
Kubernetes do have something called as network and pod security policy. The network policy allows you to control traffic coming in and out of your service or application. The pod security policy allows you to control and secure the pod runtime environment. However these policies are implemented at runtime when the workloads in the cluster are in action. What we also need is the policy that can control what you write as part of the resource manifest file. You should be able to define a policy that lets you control what fields or values can go as part of the resource manifest (YAML) file. If the resource manifest file is created as per the policy, then only it should be allowed to commit that resource in the cluster. Such a policy specifically meets the governance requirement of the cluster.
Anthos Config Management’s (ACM) Policy Controller allows us to write such governance based policies for your clusters. Policy Controller is based on Open Policy Agent (OPA) Gatekeeper project and contains library of pre-defined policies that can be used to guard your cluster against any compliance or security violation. The policy you write acts a guardrail that enforces a rule that determines whether the target resource should be admitted into the cluster or not. It acts as admission controller webhook that integrates with Kubernetes API server to validate the objects as they are admitted to the cluster. The guardrail can also be used to audit objects for any security loophole.
Constraint based Framework
The Policy Controller uses OPA Gatekeeper project which models a constraint based framework. The said framework has three main components:
- Constraint
- Rego
- Constraint template
Constraint
A constraint represents a policy. Constraints are objects that lets you specify what field or values, in manifest file, are not allowed or denied as part of the policy. If there is no constraint defined, it means the manifest or resource specification is implicitly allowed to be processed by the Kubernetes API server thereby admitting it into the cluster. A constraint enforces validation. The ACM Policy Controller comes with set of predefined constraints as part of the constraint library. The library has lot of useful policies that can be used with your cluster to create a standard governance model.
Rego
Policy is written in a OPA based query language called Rego. Rego can easily reference nested documents and therefore more suitable with JSON and YAML format files. The queries you write in Rego are more like assertions on data stored in OPA. The language itself is declarative allowing you to write simple statement like queries that simply returns values. A rego language is used in the constraint template which we will see in the next section.
Constraint template
It is a policy template that can be made portable and reusable. It is in effect a Custom Resource Definition (CRD) that allows you to templatize the behavior of the policy using typed parameters and custom logic. The custom logic is written in Rego query language, which along with parameters and error messages defines your policy. Once the template is created, it can be invoked by using a constraint. It is in the constraint object that you pass the actual values, to be applied to a policy, to the template parameters.
Enforcing a Sample Policy
Let us see each of the above in action. You will create a policy that will allow only Google container registry as the source for your container images. If the Kubernetes deployment contains an image from a non-Google container registry then it should not allow to create the deployments.
Creating Constraint template
The following constraint template will allow only those container image repos whose name matches a value provided as part of the constraint (we will see later how to create a constraint).
## constraint-template.yaml
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
name: k8svalidimagerepos
spec:
crd:
spec:
names:
kind: K8sValidImageRepos
validation:
# Schema for the `parameters` field
openAPIV3Schema:
properties:
repos:
type: array
items:
type: string
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package k8svalidimagerepos
violation[{"msg": msg}] {
container := input.review.object.spec.containers[_]
satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
not any(satisfied)
msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
}There are three significant parts to observe from the above code. The first is the name of the resource k8svalidimagerepos that we will reference later (from the constraint). The second is the schema for the input parameter as part of the validation field. In our case, the parameter is the list of repos specified as array of strings. You will pass the name of the container image repo from the constraint resource. The third part is the rego code as part of the target field. It checks if the deployment container image name provided as an input is valid or not. If not then it will indicate a violation of policy and return with an error message.
You will now create the constraint that will refer the above template and pass the container image repo name as an input.
Creating Constraint
The constraint is a way of enforcing a policy. The below constraint will pass the container image repo name as an input that will be used as an image source.
## contraint.yaml
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sValidImageRepos
metadata:
name: allowed-repo-gcr
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Deployment"]
namespaces:
- "staging"
parameters:
repos:
- "gcr"Let’s observe the above code carefully. The constraint kind K8sValidImageRepos is the same as the template name. This indicates that the above policy uses the k8svalidimagerepos constraint template. The spec field contains the matching resource on which this policy will be applied. The above code applies the policy to Deployments as part of staging namespace. The parameters field is the input you provide that matches the constraint template schema that you defined earlier. In this case, it is the name(s) of the container image repo you want to allow. For this example, you will allow only the Google container registries as the image source and it starts with the prefix gcr.
To sum up, the above constraint is a policy in effect that only allows image registries with prefix as gcr defined in the deployment as part of staging namespace. The constraint makes use of k8svalidimagerepos constraint template to enforce this policy.
Testing the Policy
Apply the constraint template and the constraint manifest files to the cluster
kubectl apply -f constraint-template.yaml kubectl apply -f constraint.yaml
Once the template and constraint resources are created, you will test the policy by creating a sample Deployment in the staging namespace.
Create the sample Deployment manifest file that will fetch the container image from the docker hub. As our policy only allows image from Google container repo prefixed with gcr, the Deployment creation should fail.
## nodeapp-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nodeapp
namespace: staging
labels:
app: nodeapp
spec:
replicas: 1
selector:
matchLabels:
app: nodeapp
template:
metadata:
labels:
app: nodeapp
spec:
containers:
- name: nodeapp
image: registry.docker.hub.com/nodeapp
ports:
- containerPort: 9000The above deployment nodeapp tries to fetch the image from the docker hub. Let’s try to create the deployment.
kubectl -n staging apply -f nodeapp-deploy.yaml
Output:
Error from server ([allowed-repo-gcr] container <nodeapp> has an invalid image repo <registry.docker.hub.com/nodeapp>, allowed repos are ["gcr"]): error when creating "nodeapp-deploy.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [allowed-repo-gcr] container <nodeapp> has an invalid image repo <registry.docker.hub.com/nodeapp>, allowed repos are ["gcr"]
The above fails to create the deployment as the container image source is the docker hub. It will display the error message, we programmed as part of our Rego code earlier.
As you can see, the policy and governance are important security aspects that are addressed by ACM’s Policy Controller through OPA Gatekeeper framework. It allows you to write user defined policies that keeps the Kubernetes clusters compliant as per the policy.
Published on Java Code Geeks with permission by Rajeev Hathi, partner at our JCG program. See the original article here: Enforcing a Policy using Anthos Config Management’s Policy Controller Opinions expressed by Java Code Geeks contributors are their own. |
