Encrypt OS disks with a customer-managed key on Azure Red Hat OpenShift

By default, Microsoft Azure uses autogenerated keys to encrypt the OS disks of the virtual machines (VMs) in an Azure Red Hat OpenShift cluster. For extra security, you can encrypt the OS disks by using self-managed keys when you deploy a cluster. This feature gives you more control by encrypting confidential data with customer-managed keys.

Clusters with customer-managed keys have a default storage class enabled with their keys. Therefore, these keys encrypt both OS disks and data disks. Azure Key Vault stores the customer-managed keys. For more information about using Key Vault to create and maintain keys, see Azure Key Vault basic concepts.

With host-based encryption, the data stored on the virtual machine host of your Azure Red Hat OpenShift node VMs is encrypted at rest. This data flows encrypted to Azure Storage. Host-based encryption means the temp disks are encrypted at rest by using platform-managed keys.

The cache of OS and data disks is encrypted at rest with either platform-managed keys or customer-managed keys. The type of keys depends on the encryption type that you set on those disks. By default, when you use Azure Red Hat OpenShift, OS and data disks are encrypted at rest with platform-managed keys. This default setting means that the caches for these disks are also by default encrypted at rest with platform-managed keys.

You can specify your own managed keys by following the encryption steps in the next section. The keys that you specify in this step also encrypts the cache for these disks.

Limitation

Customers must maintain the key vault and disk encryption set in Azure. Failure to maintain the keys results in broken Azure Red Hat OpenShift clusters. The VMs stop working and, as a result, the entire Azure Red Hat OpenShift cluster stops functioning.

The Azure Red Hat OpenShift engineering team can't access the keys. Therefore, they can't back up, replicate, or retrieve the keys.

For details about using disk encryption sets to manage your encryption keys, see Server-side encryption of Azure Disk Storage.

Prerequisites

  • Verify your permissions. You must have permissions to create an Azure Red Hat OpenShift cluster, a key vault, and a disk encryption set.

  • Confirm that you have the Key Vault Crypto Officer role. This role has permissions to create a key.

  • If you have multiple Azure subscriptions, register the resource providers. For registration details, see Register the resource providers.

  • Confirm that you have the EncryptionAtHost feature enabled on your subscription. You can enable it by running:

    az feature register --namespace Microsoft.Compute --name EncryptionAtHost

    You can check the current status of the feature by running:

    az feature show --namespace Microsoft.Compute --name EncryptionAtHost

Create an instance of Key Vault

You must use an instance of Key Vault to store your keys. Create a new key vault with purge protection enabled. Then, create a new key within the key vault to store your own custom key.

  1. Set environment variables:

    export KEYVAULT_NAME=aro-enckv
export KEYVAULT_KEY_NAME=aro-key
export DISK_ENCRYPTION_SET_NAME=aro-des
export LOCATION=<location-name>
export RESOURCEGROUP=<resource-group-name>

  2. Create a key vault:

    az keyvault create \
    --name $KEYVAULT_NAME \
    --resource-group $RESOURCEGROUP \
    --location $LOCATION \
    --enable-purge-protection true

    Note

    You must enable purge protection. See Set up your Azure Key Vault.

    Optional: Because purge protection is enabled, you can't immediately reuse the key vault name after deletion. By adding the --retention-days <NUMBER_OF_DAYS> specification, you can control how long the vault is held in a "soft-delete" state. The default is 90 days, but setting it to a lower value (minimum 7 days) means that you can reclaim the name more quickly.

  3. Create a key in the key vault:

    az keyvault key create \
    --vault-name $KEYVAULT_NAME \
    --name $KEYVAULT_KEY_NAME \
    --protection software

  4. Set the key vault resource ID and key URL variables:

    KEYVAULT_ID=$(az keyvault show --name $KEYVAULT_NAME --query "[id]" -o tsv)

KEYVAULT_KEY_URL=$(az keyvault key show --vault-name $KEYVAULT_NAME --name $KEYVAULT_KEY_NAME --query "[key.kid]" -o tsv)

Create a disk encryption set

Use the Azure disk encryption set as the reference point for disks in Azure Red Hat OpenShift clusters. It's connected to the key vault that you created in the previous step, and pulls the customer-managed keys from that location.

  1. Create the disk encryption set:

    az disk-encryption-set create \
    --name $DISK_ENCRYPTION_SET_NAME \
    --location $LOCATION \
    --resource-group $RESOURCEGROUP \
    --source-vault $KEYVAULT_ID \
    --key-url $KEYVAULT_KEY_URL

  2. Specify the disk encryption set resource ID and identity variables:

    DES_ID=$(az disk-encryption-set show --name $DISK_ENCRYPTION_SET_NAME --resource-group $RESOURCEGROUP --query 'id' -o tsv)

DES_IDENTITY=$(az disk-encryption-set show --name $DISK_ENCRYPTION_SET_NAME --resource-group $RESOURCEGROUP --query "[identity.principalId]" -o tsv)

Grant permissions for the disk encryption set to access the key vault

You want the managed identity of the disk encryption set to use the encryption key. Assign a built-in role that provides the required key permissions, such as Key Vault Crypto Service Encryption User or Key Vault Crypto User. Alternatively, use a custom role, as long as it includes the necessary permissions (wrapkey, unwrapkey, and get) to perform key operations required by disk encryption sets.

az role assignment create \
    --assignee $DES_IDENTITY \
    --role "Key Vault Crypto Service Encryption User" \
    --scope $KEYVAULT_ID

Create an Azure Red Hat OpenShift cluster

When you create a cluster, you can enable customer-managed keys by specifying the disk encryption set. Use the --disk-encryption-set parameter. For example, --disk-encryption-set $DES_ID.

To create a cluster with a service principal, follow the instructions in Create an Azure Red Hat OpenShift cluster.

Here's an example of creating a cluster with the --disk-encryption-set parameter:

az aro create \
    --resource-group $RESOURCEGROUP \
    --name $CLUSTER \
    --vnet aro-vnet \
    --master-subnet master-subnet \
    --worker-subnet worker-subnet \
    --disk-encryption-set $DES_ID

Note

You can only enable customer-managed keys on existing clusters for worker nodes, not master nodes. It's possible to achieve this configuration. For more information, see the Red Hat documentation, including Enabling customer-managed encryption keys for a machine set and Modifying a compute machine set.

To Create an Azure Red Hat OpenShift cluster with managed identities, prior to creating the cluster, you must grant the machine-api identity and the disk-csi-driver identity. You must also grant the Azure Red Hat OpenShift first party service principal Microsoft.Compute/diskEncryptionSets/read permissions for the disk encryption set.

Example role assignments for the machine-api identity, the disk-csi-driver identity, and the ARO first party service principal:

az role assignment create \
  --assignee-object-id "$(az identity show --resource-group $RESOURCEGROUP --name disk-csi-driver --query principalId -o tsv)" \
  --assignee-principal-type ServicePrincipal \
  --role <ROLE> \
  --scope $DES_ID"

az role assignment create \
  --assignee-object-id "$(az identity show --resource-group $RESOURCEGROUP --name machine-api --query principalId -o tsv)" \
  --assignee-principal-type ServicePrincipal \
  --role <ROLE> \
  --scope $DES_ID

az role assignment create \
  --assignee-object-id "$(az ad sp list --display-name "Azure Red Hat OpenShift RP" --query '[0].id' -o tsv)" \
  --assignee-principal-type ServicePrincipal \
  --role <ROLE> \
  --scope $DES_ID

After you create the Azure Red Hat OpenShift cluster, all VMs are encrypted with the customer-managed encryption keys.

To verify that you configured the keys correctly, run the following commands:

  1. Get the name of the cluster resource group where the cluster's VMs, disks, and other infrastructure are located:

    CLUSTERRESOURCEGROUP=$(az aro show --resource-group $RESOURCEGROUP --name $CLUSTER --query 'clusterProfile.resourceGroupId' -o tsv | cut -d '/' -f 5)

  2. Check that the disks have the correct disk encryption set attached:

    az disk list --resource-group $CLUSTERRESOURCEGROUP --query '[].encryption'

The field diskEncryptionSetId in the output must point to the disk encryption set that you specified while creating the Azure Red Hat OpenShift cluster.

Add managed HSM (mHSM) support for customer-managed keys and Azure Policy enforcement

You can also add managed HSM (mHSM) support for customer-managed keys, and require and deny certain encryption attributes. This support requires API and Azure Policy changes.

Follow these steps to create an Azure Red Hat OpenShift cluster with mHSM and customer-managed keys:

  1. Provision and activate a managed HSM by using the Azure CLI.

  2. Export environment variables from the previous step:

    export HSM_NAME=<hsm-name>
export RESOURCE_GROUP=<resource-group>

  3. Enable purge protection and key management through Azure Resource Manager:

    az keyvault update-hsm \
  --hsm-name $HSM_NAME \
  --resource-group $RESOURCE_GROUP \
  --enable-purge-protection true

az keyvault setting update \
  --hsm-name $HSM_NAME \
  --name AllowKeyManagementOperationsThroughARM \
  --value true

  4. Complete the remaining steps to activate the mHSM: Activate your managed HSM.

  5. Grant yourself the Managed HSM Crypto User role so you can create keys in the mHSM.

    oid=$(az ad signed-in-user show --query id -o tsv)
az keyvault role assignment create \        
  --hsm-name $HSM_NAME \ 
  --role "Managed HSM Crypto User" \ 
  --assignee $oid \ 
  --scope /keys

  6. Create an mHSM key.

  7. Create a managed identity and grant it the Managed HSM Crypto Service Encryption User role over the mHSM.

    export UA_IDENTITY=<new-ua-identity>

    HSM_ID="https://${HSM_NAME}.managedhsm.azure.net"

az identity create \
  --resource-group $RESOURCE_GROUP \
  --name $UA_IDENTITY

IDENTITY_OBJECT_ID=$(az identity show \ 
  --resource-group $RESOURCE_GROUP \ 
  --name $UA_IDENTITY \ 
  --query principalId -o tsv) 

az keyvault role assignment create \ 
  --id "$HSM_ID" \ 
  --role "Managed HSM Crypto Service Encryption User" \ 
  --assignee-object-id "$IDENTITY_OBJECT_ID" \ 
  --scope "/keys/byok-key"

  8. Create a disk encryption set, taking into consideration:

    1. Set the encryption type to Encryption at-rest with a customer-managed key.

    2. Use the user-assigned identity you created in the previous step.

    3. Complete the role assignment before you create the disk encryption set.

  9. Export environment variables from the previous step:

    export LOCATION=<yourLocation>
export CLUSTER=<yourNewClusterName> 
export VIRTUAL_NETWORK=<yourVirtualNetwork>
export KEYVAULT_NAME=<yourKeyvaultName>
export KEYVAULT_KEY_NAME=byok-key 
export DISK_ENCRYPTION_SET_NAME=<yourDiskEncryptionSetName>

  10. Create a cluster with your new disk encryption set:

    DES_ID=$(az disk-encryption-set show -n $DISK_ENCRYPTION_SET_NAME -g $RESOURCE_GROUP --query 'id' -o tsv) 

az aro create --resource-group $RESOURCE_GROUP \ 
  --name $CLUSTER  \ 
  --vnet $VIRTUAL_NETWORK  \ 
  --master-subnet master-subnet \ 
  --worker-subnet worker-subnet \ 
  --disk-encryption-set $DES_ID

  11. After the cluster comes up, confirm the encryption:

    az disk list -g $VIRTUAL_NETWORK --query "[].{DiskName:name, VM:managedBy, EncryptionType:encryption.type, DiskEncryptionSet:encryption.diskEncryptionSetId}" -o table

    To verify the encryption, look for EncryptionAtRestWithCustomerKey under the EncryptionType column.

  • Last updated on