Access Managed Lustre instances on GKE with the Managed Lustre CSI driver

This guide describes how you can create a new Kubernetes volume backed by the Managed Lustre CSI driver in GKE with dynamic provisioning. The Managed Lustre CSI driver lets you create storage backed by Managed Lustre instances on-demand, and access them as volumes for your stateful workloads.

Multi-NIC support for high-performance networking

For GKE clusters running version 1.35.2-gke.1842000 or later, the Managed Lustre CSI driver is enabled by default to use all available Network Interface Cards (NICs) for increased throughput. This support aggregates bandwidth by spreading TCP storage traffic across your network interfaces.

To use multi-NIC support, your nodes must meet the following requirements:

Standard NICs for TCP: your nodes must use standard NICs, such as Google Virtual NIC (gVNIC) or VirtIO-Net, to handle TCP storage traffic.
Same VPC: all standard NICs must reside in the same VPC network.
RDMA considerations: your nodes can also have RDMA NICs attached; however, the Managed Lustre CSI driver only uses the standard NICs for TCP storage traffic.

If you want to disable the multi-NIC support, see Disable multi-NIC for Lustre.

Lustre communication ports

The GKE Managed Lustre CSI driver uses different ports for communication with Managed Lustre instances, depending on your GKE cluster version and existing Managed Lustre configurations.

Default port (Recommended): for new GKE clusters that run version 1.33.2-gke.4780000 or later, the driver uses port 988 for Lustre communication by default.
Legacy Port (Deprecated): use port 6988 by appending the --enable-legacy-lustre-port flag to your gcloud commands in the following scenarios:
- Earlier GKE versions: if your GKE cluster runs a version earlier than 1.33.2-gke.4780000, the --enable-legacy-lustre-port flag works around a port conflict with the gke-metadata-server on GKE nodes.
- Existing Lustre instances: if you are connecting to an existing Managed Lustre instance that was created with the gke-support-enabled flag, you must still include --enable-legacy-lustre-port in your gcloud commands, irrespective of your cluster version. Without this flag, your GKE cluster will fail to mount the existing Lustre instance.

You can configure the new and existing clusters to use either the default port 988, or the legacy port 6988.

Before you begin

Before you start, make sure that you have performed the following tasks:

Enable the Google Cloud Managed Lustre API and the Google Kubernetes Engine API.

Enable APIs

To use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running the gcloud components update command. Earlier gcloud CLI versions might not support running the commands in this document.
Note: For existing gcloud CLI installations, make sure to set the compute/region property. If you use primarily zonal clusters, set the compute/zone instead. By setting a default location, you can avoid errors in the gcloud CLI like the following: One of [--zone, --region] must be supplied: Please specify location. You might need to specify the location in certain commands if the location of your cluster differs from the default that you set.

For limitations and requirements, see the CSI driver overview.
Make sure to enable the Managed Lustre CSI driver. It is disabled by default in Standard and Autopilot clusters.

Set up environment variables

Set up the following environment variables:

export CLUSTER_NAME=CLUSTER_NAME
export PROJECT_ID=PROJECT_ID
export NETWORK_NAME=LUSTRE_NETWORK
export IP_RANGE_NAME=LUSTRE_IP_RANGE
export FIREWALL_RULE_NAME=LUSTRE_FIREWALL_RULE
export LOCATION=ZONE
export CLUSTER_VERSION=CLUSTER_VERSION

Replace the following:

CLUSTER_NAME: the name of the cluster.
PROJECT_ID: your Google Cloud project ID.
LUSTRE_NETWORK: the shared Virtual Private Cloud (VPC) network where both the GKE cluster and Managed Lustre instance reside.
LUSTRE_IP_RANGE: the name for the IP address range created for VPC Network Peering with Managed Lustre.
LUSTRE_FIREWALL_RULE: the name for the firewall rule to allow TCP traffic from the IP address range.
ZONE: the geographical zone of your GKE cluster; for example, us-central1-a.
CLUSTER_VERSION: the GKE cluster version.

Set up a VPC network

You must specify the same VPC network when creating the Managed Lustre instance and your GKE clusters, or connected through Network Connectivity Center if using a peered VPC network.

To enable service networking, run the following command:

gcloud services enable servicenetworking.googleapis.com \
    --project=${PROJECT_ID}

Create a VPC network. Setting the --mtu flag to 8896 results in a 10% performance gain.

gcloud compute networks create ${NETWORK_NAME} \
    --subnet-mode=auto --project=${PROJECT_ID} \
    --mtu=8896

Create an IP address range.

gcloud compute addresses create ${IP_RANGE_NAME} \
    --global \
    --purpose=VPC_PEERING \
    --prefix-length=20 \
    --description="Managed Lustre VPC Peering" \
    --network=${NETWORK_NAME} \
    --project=${PROJECT_ID}

Get the CIDR range associated with the range you created in the preceding step.

CIDR_RANGE=$(
  gcloud compute addresses describe ${IP_RANGE_NAME} \
      --global  \
      --format="value[separator=/](address, prefixLength)" \
      --project=${PROJECT_ID}
)

Create a firewall rule to allow TCP traffic from the IP address range you created.

gcloud compute firewall-rules create ${FIREWALL_RULE_NAME} \
    --allow=tcp:988,tcp:6988 \