# AWS - Nitro Enum

{{#include ../../../../banners/hacktricks-training.md}}

## Basic Information

AWS Nitro is a suite of **innovative technologies** that form the underlying platform for AWS EC2 instances. Introduced by Amazon to **enhance security, performance, and reliability**, Nitro leverages custom **hardware components and a lightweight hypervisor**. It abstracts much of the traditional virtualization functionality to dedicated hardware and software, **minimizing the attack surface** and improving resource efficiency. By offloading virtualization functions, Nitro allows EC2 instances to deliver **near bare-metal performance**, making it particularly beneficial for resource-intensive applications. Additionally, the Nitro Security Chip specifically ensures the **security of the hardware and firmware**, further solidifying its robust architecture.

### Nitro Enclaves

**AWS Nitro Enclaves** provides a secure, **isolated compute environment within Amazon EC2 instances**, specifically designed for processing highly sensitive data. Leveraging the AWS Nitro System, these enclaves ensure robust **isolation and security**, ideal for **handling confidential information** such as PII or financial records. They feature a minimalist environment, significantly reducing the risk of data exposure. Additionally, Nitro Enclaves support cryptographic attestation, allowing users to verify that only authorized code is running, crucial for maintaining strict compliance and data protection standards.

> [!CAUTION]
> Nitro Enclave images are **run from inside EC2 instances** and you cannot see from the AWS web console if an EC2 instances is running images in Nitro Enclave or not.

## Nitro Enclave CLI installation

Follow the all instructions [**from the documentation**](https://catalog.us-east-1.prod.workshops.aws/event/dashboard/en-US/workshop/1-my-first-enclave/1-1-nitro-enclaves-cli#run-connect-and-terminate-the-enclave). However, these are the most important ones:

```bash
# Install tools
sudo amazon-linux-extras install aws-nitro-enclaves-cli -y
sudo yum install aws-nitro-enclaves-cli-devel -y

# Config perms
sudo usermod -aG ne $USER
sudo usermod -aG docker $USER

# Check installation
nitro-cli --version

# Start and enable the Nitro Enclaves allocator service.
sudo systemctl start nitro-enclaves-allocator.service && sudo systemctl enable nitro-enclaves-allocator.service
```

## Nitro Enclave Images

The images that you can run in Nitro Enclave are based on docker images, so you can create your Nitro Enclave images from docker images like:

```bash
# You need to have the docker image accesible in your running local registry
# Or indicate the full docker image URL to access the image
nitro-cli build-enclave --docker-uri <docker-img>:<tag> --output-file nitro-img.eif
```

As you can see the Nitro Enclave images use the extension **`eif`** (Enclave Image File).

The output will look similar to:

```
Using the locally available Docker image...
Enclave Image successfully created.
{
  "Measurements": {
    "HashAlgorithm": "Sha384 { ... }",
    "PCR0": "e199261541a944a93129a52a8909d29435dd89e31299b59c371158fc9ab3017d9c450b0a580a487e330b4ac691943284",
    "PCR1": "bcdf05fefccaa8e55bf2c8d6dee9e79bbff31e34bf28a99aa19e6b29c37ee80b214a414b7607236edf26fcb78654e63f",
    "PCR2": "2e1fca1dbb84622ec141557dfa971b4f8ea2127031b264136a20278c43d1bba6c75fea286cd4de9f00450b6a8db0e6d3"
  }
}
```

### Run an Image

As per [**the documentation**](https://catalog.us-east-1.prod.workshops.aws/event/dashboard/en-US/workshop/1-my-first-enclave/1-1-nitro-enclaves-cli#run-connect-and-terminate-the-enclave), in order to run an enclave image you need to assign it memory of **at least 4 times the size of the `eif` file**. It's possible to configure the default resources to give to it in the file

```shell
/etc/nitro_enclaves/allocator.yaml
```

> [!CAUTION]
> Always remember that you need to **reserve some resources for the parent EC2** instance also!

After knowing the resources to give to an image and even having modified the configuration file it's possible to run an enclave image with:

```shell
# Restart the service so the new default values apply
sudo systemctl start nitro-enclaves-allocator.service && sudo systemctl enable nitro-enclaves-allocator.service

# Indicate the CPUs and memory to give
nitro-cli run-enclave --cpu-count 2 --memory 3072 --eif-path hello.eif --debug-mode --enclave-cid 16
```

### Enumerate Enclaves

If you compromise and EC2 host it's possible to get a list of running enclave images with:

```bash
nitro-cli describe-enclaves
```

It's **not possible to get a shell** inside a running enclave image because thats the main purpose of enclave, however, if you used the parameter **`--debug-mode`**, it's possible to get the **stdout** of it with:

```shell
ENCLAVE_ID=$(nitro-cli describe-enclaves | jq -r ".[0].EnclaveID")
nitro-cli console --enclave-id ${ENCLAVE_ID}
```

### Terminate Enclaves

If an attacker compromise an EC2 instance by default he won't be able to get a shell inside of them, but he will be able to **terminate them** with:

```shell
nitro-cli terminate-enclave --enclave-id ${ENCLAVE_ID}
```

## Vsocks

The only way to communicate with an **enclave** running image is using **vsocks**.

**Virtual Socket (vsock)** is a socket family in Linux specifically designed to facilitate **communication** between virtual machines (**VMs**) and their **hypervisors**, or between VMs **themselves**. Vsock enables efficient, **bi-directional communication** without relying on the host's networking stack. This makes it possible for VMs to communicate even without network configurations, **using a 32-bit Context ID (CID) and port numbers** to identify and manage connections. The vsock API supports both stream and datagram socket types, similar to TCP and UDP, providing a versatile tool for user-level applications in virtual environments.

> [!TIP]
> Therefore, an vsock address looks like this: `<CID>:<Port>`

To find **CIDs** of the enclave running images you could just execute the following cmd and thet the **`EnclaveCID`**:

<pre class="language-bash"><code class="lang-bash">nitro-cli describe-enclaves

[
  {
    "EnclaveName": "secure-channel-example",
    "EnclaveID": "i-0bc274f83ade02a62-enc18ef3d09c886748",
    "ProcessID": 10131,
<strong>    "EnclaveCID": 16,
</strong>    "NumberOfCPUs": 2,
    "CPUIDs": [
      1,
      3
    ],
    "MemoryMiB": 1024,
    "State": "RUNNING",
    "Flags": "DEBUG_MODE",
    "Measurements": {
      "HashAlgorithm": "Sha384 { ... }",
      "PCR0": "e199261541a944a93129a52a8909d29435dd89e31299b59c371158fc9ab3017d9c450b0a580a487e330b4ac691943284",
      "PCR1": "bcdf05fefccaa8e55bf2c8d6dee9e79bbff31e34bf28a99aa19e6b29c37ee80b214a414b7607236edf26fcb78654e63f",
      "PCR2": "2e1fca1dbb84622ec141557dfa971b4f8ea2127031b264136a20278c43d1bba6c75fea286cd4de9f00450b6a8db0e6d3"
    }
  }
]
</code></pre>

> [!WARNING]
> Note that from the host there isn't any way to know if a CID is exposing any port! Unless using some **vsock port scanner like** [**https://github.com/carlospolop/Vsock-scanner**](https://github.com/carlospolop/Vsock-scanner).

### Vsock Server/Listener

Find here a couple of examples:

- [https://github.com/aws-samples/aws-nitro-enclaves-workshop/blob/main/resources/code/my-first-enclave/secure-local-channel/server.py](https://github.com/aws-samples/aws-nitro-enclaves-workshop/blob/main/resources/code/my-first-enclave/secure-local-channel/server.py)

<details>

<summary>Simple Python Listener</summary>

```python
#!/usr/bin/env python3

# From
https://medium.com/@F.DL/understanding-vsock-684016cf0eb0

import socket

CID = socket.VMADDR_CID_HOST
PORT = 9999

s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
s.bind((CID, PORT))
s.listen()
(conn, (remote_cid, remote_port)) = s.accept()

print(f"Connection opened by cid={remote_cid} port={remote_port}")

while True:
    buf = conn.recv(64)
    if not buf:
        break

    print(f"Received bytes: {buf}")
```

</details>

```bash
# Using socat
socat VSOCK-LISTEN:<port>,fork EXEC:"echo Hello from server!"
```

### Vsock Client

Examples:

- [https://github.com/aws-samples/aws-nitro-enclaves-workshop/blob/main/resources/code/my-first-enclave/secure-local-channel/client.py](https://github.com/aws-samples/aws-nitro-enclaves-workshop/blob/main/resources/code/my-first-enclave/secure-local-channel/client.py)

<details>

<summary>Simple Python Client</summary>

```python
#!/usr/bin/env python3

#From https://medium.com/@F.DL/understanding-vsock-684016cf0eb0

import socket

CID = socket.VMADDR_CID_HOST
PORT = 9999

s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
s.connect((CID, PORT))
s.sendall(b"Hello, world!")
s.close()
```

</details>

```bash
# Using socat
echo "Hello, vsock!" | socat - VSOCK-CONNECT:3:5000
```

### Vsock Proxy

The tool vsock-proxy allows to proxy a vsock proxy with another address, for example:

```bash
vsock-proxy 8001 ip-ranges.amazonaws.com 443 --config your-vsock-proxy.yaml
```

This will forward the **local port 8001 in vsock** to `ip-ranges.amazonaws.com:443` and the file **`your-vsock-proxy.yaml`** might have this content allowing to access `ip-ranges.amazonaws.com:443`:

```yaml
allowlist:
  - { address: ip-ranges.amazonaws.com, port: 443 }
```

It's possible to see the vsock addresses (**`<CID>:<Port>`**) used by the EC2 host with (note the `3:8001`, 3 is the CID and 8001 the port):

```bash
sudo ss -l -p -n | grep v_str
v_str LISTEN 0      0                                                                              3:8001                   *:*     users:(("vsock-proxy",pid=9458,fd=3))
```

## Nitro Enclave Atestation & KMS

The Nitro Enclaves SDK allows an enclave to request a **cryptographically signed attestation document** from the Nitro **Hypervisor**, which includes **unique measurements** specific to that enclave. These measurements, which include **hashes and platform configuration registers (PCRs)**, are used during the attestation process to **prove the enclave's identity** and **build trust with external services**. The attestation document typically contains values like PCR0, PCR1, and PCR2, which you have encountered before when building and saving an enclave EIF.

From the [**docs**](https://catalog.us-east-1.prod.workshops.aws/event/dashboard/en-US/workshop/1-my-first-enclave/1-3-cryptographic-attestation#a-unique-feature-on-nitro-enclaves), these are the PCR values:

<table><thead><tr><th width="97">PCR</th><th width="221">Hash of ...</th><th>Description</th></tr></thead><tbody><tr><td>PCR0</td><td>Enclave image file</td><td>A contiguous measure of the contents of the image file, without the section data.</td></tr><tr><td>PCR1</td><td>Linux kernel and bootstrap</td><td>A contiguous measurement of the kernel and boot ramfs data.</td></tr><tr><td>PCR2</td><td>Application</td><td>A contiguous, in-order measurement of the user applications, without the boot ramfs.</td></tr><tr><td>PCR3</td><td>IAM role assigned to the parent instance</td><td>A contiguous measurement of the IAM role assigned to the parent instance. Ensures that the attestation process succeeds only when the parent instance has the correct IAM role.</td></tr><tr><td>PCR4</td><td>Instance ID of the parent instance</td><td>A contiguous measurement of the ID of the parent instance. Ensures that the attestation process succeeds only when the parent instance has a specific instance ID.</td></tr><tr><td>PCR8</td><td>Enclave image file signing certificate</td><td>A measure of the signing certificate specified for the enclave image file. Ensures that the attestation process succeeds only when the enclave was booted from an enclave image file signed by a specific certificate.</td></tr></tbody></table>

You can integrate **cryptographic attestation** into your applications and leverage pre-built integrations with services like **AWS KMS**. AWS KMS can **validate enclave attestations** and offers attestation-based condition keys (`kms:RecipientAttestation:ImageSha384` and `kms:RecipientAttestation:PCR`) in its key policies. These policies ensure that AWS KMS permits operations using the KMS key **only if the enclave's attestation document is valid** and meets the **specified conditions**.

> [!TIP]
> Note that Enclaves in debug (--debug) mode generate attestation documents with PCRs that are made of zeros (`000000000000000000000000000000000000000000000000`). Therefore, KMS policies checking these values will fail.

### PCR Bypass

From an attackers perspective, notice that some PCRs would allow to modify some parts or all the enclave image and would still be valid (for example PCR4 just checks the ID of the parent instance so running any enclave image in that EC2 will allow to fulfil this potential PCR requirement).

Therefore, an attacker that compromise the EC2 instance might be able to run other enclave images in order to bypass these protections.

The research on how to modify/create new images to bypass each protection (spcially the not taht obvious ones) is still TODO.

## References

- [https://medium.com/@F.DL/understanding-vsock-684016cf0eb0](https://medium.com/@F.DL/understanding-vsock-684016cf0eb0)
- All the parts of the Nitro tutorial from AWS: [https://catalog.us-east-1.prod.workshops.aws/event/dashboard/en-US/workshop/1-my-first-enclave/1-1-nitro-enclaves-cli](https://catalog.us-east-1.prod.workshops.aws/event/dashboard/en-US/workshop/1-my-first-enclave/1-1-nitro-enclaves-cli)

{{#include ../../../../banners/hacktricks-training.md}}




