How to Deploy Google Cloud Anthos Service Mesh (ASM) for Multi-Cluster Traffic Routing

As organizations scale their Kubernetes footprint, they inevitably move from a single massive cluster to a multi-cluster architecture. Operating multiple Google Kubernetes Engine (GKE) clusters across different regions (e.g., us-central1 and europe-west1) provides profound high availability and disaster recovery capabilities.

However, routing traffic dynamically between microservices that live in entirely different clusters—without exposing that internal traffic to the public internet—is a massive networking challenge.

Google Cloud Anthos Service Mesh (ASM), which is Google’s fully managed enterprise distribution of Istio, solves this by extending a single logical service mesh across multiple distinct Kubernetes clusters. With Multi-Cluster ASM, a frontend microservice in New York can communicate securely (via mutual TLS) with a backend database service in London, routed purely via internal Google Cloud backbone IPs.

This guide explains how to architect and deploy a multi-cluster Anthos Service Mesh using the Multi-Primary on different networks topology.

Understanding Multi-Cluster ASM Architecture

In a multi-cluster mesh, there are several topological choices. The most robust enterprise configuration is Multi-Primary.

1. Separate Control Planes: Each GKE cluster runs its own dedicated Istio control plane (istiod). If the network link between the regions drops, both clusters continue to function autonomously.
2. Cross-Cluster Trust: You establish a shared cryptographic root of trust (a common Root CA) across both clusters so that the Envoy sidecars in Cluster A trust the Envoy sidecars in Cluster B, allowing mTLS to function seamlessly across the ocean.
3. East-West Gateways: Because the clusters exist in different VPC subnets (or different VPCs entirely), direct Pod-to-Pod IP routing is often impossible or insecure. Traffic leaving Cluster A destined for Cluster B is routed through a dedicated Istio East-West Gateway (an internal Load Balancer) in Cluster B, which then forwards the traffic to the destination Pod.
4. Endpoint Discovery: The control plane in Cluster A is granted read-only access to the Kubernetes API server in Cluster B. This allows Cluster A to learn about the Pod IPs and Services existing in Cluster B, unifying the service registry.

Step 1: Establishing the Shared Root of Trust

Before installing ASM, you must ensure both clusters share a Root Certificate Authority. If Cluster A uses its self-signed certs and Cluster B uses its own, mTLS will fail.

Google provides the Mesh CA (a fully managed CA integrated into Google Cloud), which is the recommended approach for ASM. When you register both GKE clusters to the same Google Cloud Fleet, they automatically share the Mesh CA root of trust.

gcloud container fleet memberships register cluster-east \
    --gke-cluster=us-east1-b/cluster-east \
    --enable-workload-identity

gcloud container fleet memberships register cluster-west \
    --gke-cluster=us-west1-b/cluster-west \
    --enable-workload-identity

Step 2: Installing ASM on Both Clusters

You install ASM on both clusters independently. Google recommends using the managed control plane, where Google handles the istiod lifecycle.

Enable the Mesh API and apply the ASM configuration to both clusters using the asmcli tool:

./asmcli install \
  --project_id $PROJECT_ID \
  --cluster_name cluster-east \
  --cluster_location us-east1-b \
  --fleet_id $PROJECT_ID \
  --managed \
  --enable_all

Repeat this exact command for cluster-west.

Step 3: Deploying the East-West Gateways

To route traffic across the VPC network, you must deploy the East-West Gateway in both clusters.

Google provides a specialized script within the ASM package (gen-eastwest-gateway.sh). This generates a Kubernetes manifest for an Internal TCP Load Balancer exposed to your VPC.

# Generate and apply the gateway in Cluster East
./gen-eastwest-gateway.sh \
    --mesh mesh_id --cluster cluster-east --network network-east | \
    kubectl apply --context=ctx-cluster-east -f -

Repeat for Cluster West.

Once deployed, you must apply the expose-services.yaml Gateway configuration, which instructs the East-West Gateway to accept traffic on port 15443 (the SNI mTLS routing port used by Istio) and forward it into the mesh.

Step 4: Configuring Cross-Cluster Endpoint Discovery

The final architectural step is cross-linking the API servers. istiod in Cluster East needs to know when a new Pod spins up in Cluster West.

You use the istioctl create-remote-secret command to generate a Kubernetes Secret containing the Kubeconfig of Cluster West, and apply that secret into Cluster East (and vice versa).

# Give Cluster East access to read endpoints in Cluster West
istioctl x create-remote-secret \
  --context=ctx-cluster-west \
  --name=cluster-west | \
  kubectl apply -f - --context=ctx-cluster-east

Once this secret is applied, the istiod control plane in Cluster East connects to the API server in Cluster West and synchronizes the service registry.

Step 5: Verifying Multi-Cluster Routing

With the architecture complete, the developer experience is entirely seamless.

If a developer deploys a standard Kubernetes Service named backend-db in both Cluster East and Cluster West, ASM merges them into a single logical entity. When a frontend Pod in Cluster East sends an HTTP request to http://backend-db, the Envoy sidecar intercepts it.

Envoy knows that endpoints for backend-db exist locally in Cluster East, and remotely in Cluster West. Using standard Istio DestinationRules, you can configure the traffic to route 100% locally to minimize latency, but instantly fail over across the East-West Gateway to Cluster West if the local pods crash—achieving multi-region high availability with zero code changes in the application layer.

Conclusion

Anthos Service Mesh transforms fragmented multi-cluster Kubernetes deployments into a single, cohesive, highly available application fabric. By establishing cross-cluster trust, deploying East-West gateways, and synchronizing service registries, platform engineers can orchestrate resilient, geo-distributed architectures where secure mTLS traffic flows seamlessly across the global Google Cloud network.