Initial setup stage
GKE supports Kubernetes Gateway API load balancing and routing of network traffic to 2 or more clusters. Kubernetes gateway API here refers to HTTPRoute, Gateway native objects. This means traffic from a client will be redirected to a public IP and these traffic gets redirect to one or more GKE clusters.
export PROJECT_ID=your-project-id
export VERSION=1.32
export PROJECT_NUMBER=your-project-number
Create 2 gke clusters by running the following commands
gcloud container clusters create gke-west-1 \
--gateway-api=standard \
--location=us-west1-a \
--workload-pool=$PROJECT_ID.svc.id.goog \
--cluster-version=$VERSION \
--enable-fleet \
--project=$PROJECT_ID
Create a east cluster
gcloud container clusters create gke-east-1 \
--gateway-api=standard \
--location=us-east1-b \
--workload-pool=$PROJECT_ID.svc.id.goog \
--cluster-version=$VERSION \
--enable-fleet \
--project=$PROJECT_ID
Check your gke membership by running the following command
gcloud container fleet memberships list --project=$PROJECT_ID
Next, you can see the cluster membership.
Setup your kubectl contexts
gcloud container clusters get-credentials gke-west-1 --location=us-west1-a --project=$PROJECT_ID
gcloud container clusters get-credentials gke-east-1 --location=us-east1-b --project=$PROJECT_ID
Enable gateway multicluster
To enable gateway for a multi-cluster
gcloud container fleet multi-cluster-services enable \
--project $PROJECT_ID
Then setup service account permission
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member "serviceAccount:$PROJECT_ID.svc.id.goog[gke-mcs/gke-mcs-importer]" \
--role "roles/compute.networkViewer" \
--project=$PROJECT_ID
You can retrieve the status of your cluster by running this command
gcloud container fleet multi-cluster-services describe --project=$PROJECT_ID
Next we will enable the ingress - and notice here that we are setting the west cluster to be the config cluster.
gcloud container fleet ingress enable \
--config-membership=projects/PROJECT_ID/locations/us-west1/memberships/gke-west-1 \
--project=$PROJECT_ID
Ensure that we have the right permission in place
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member "serviceAccount:service-$PROJECT_NUMBER@gcp-sa-multiclusteringress.iam.gserviceaccount.com" \
--role "roles/container.admin" \
--project=$PROJECT_ID
And let's verify the ingres setup
gcloud container fleet ingress describe --project=$PROJECT_ID
And then we will verify that the kubernetes class resources are being created:-
kubectl get gatewayclasses --context=gke_secondary-471605_us-west1-a_gke-west-1
Application deployment stage
Now that we have our cluster setup, it is time to deploy our application and configure our gateway.
We need to deploy the application to 2 different cluster and the store app looks like this
store.yaml
kind: Namespace
apiVersion: v1
metadata:
name: store
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: store
namespace: store
spec:
replicas: 2
selector:
matchLabels:
app: store
version: v1
template:
metadata:
labels:
app: store
version: v1
spec:
containers:
- name: whereami
image: us-docker.pkg.dev/google-samples/containers/gke/whereami:v1.2.20
ports:
- containerPort: 8080
Run the following command
kubectl apply --context your-west-context -f store.yaml
kubectl apply --context your-east-context -f store.yaml
Next run deploy the service for both cluster separately here
service-west.yaml
apiVersion: v1
kind: Service
metadata:
name: store
namespace: store
spec:
selector:
app: store
ports:
- port: 8080
targetPort: 8080
---
kind: ServiceExport
apiVersion: net.gke.io/v1
metadata:
name: store
namespace: store
---
apiVersion: v1
kind: Service
metadata:
name: store-west-1
namespace: store
spec:
selector:
app: store
ports:
- port: 8080
targetPort: 8080
---
kind: ServiceExport
apiVersion: net.gke.io/v1
metadata:
name: store-west-1
namespace: store
service-east.yaml
apiVersion: v1
kind: Service
metadata:
name: store
namespace: store
spec:
selector:
app: store
ports:
- port: 8080
targetPort: 8080
---
kind: ServiceExport
apiVersion: net.gke.io/v1
metadata:
name: store
namespace: store
---
apiVersion: v1
kind: Service
metadata:
name: store-east-1
namespace: store
spec:
selector:
app: store
ports:
- port: 8080
targetPort: 8080
---
kind: ServiceExport
apiVersion: net.gke.io/v1
metadata:
name: store-east-1
namespace: store
Next we will setup gateway by running the following command Notice we only apply gateway and route on the WEST cluster. The gateway we are using is a global application load balancer.
The service export above will automatically create servieimport for both cluster and you will notice each has a different IP address.
gateway.yaml
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
name: external-http
namespace: store
spec:
gatewayClassName: gke-l7-global-external-managed-mc
listeners:
- name: http
protocol: HTTP
port: 80
allowedRoutes:
kinds:
- kind: HTTPRoute
And we will run the following command to deploy it
kubectl --context your-west-context apply -f gateway.yaml
Next apply the httproute.yaml here
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
metadata:
name: public-store-route
namespace: store
labels:
gateway: external-http
spec:
hostnames:
- "store.example.com"
parentRefs:
- name: external-http
rules:
- matches:
- path:
type: PathPrefix
value: /west
backendRefs:
- group: net.gke.io
kind: ServiceImport
name: store-west-1
port: 8080
- matches:
- path:
type: PathPrefix
value: /east
backendRefs:
- group: net.gke.io
kind: ServiceImport
name: store-east-1
port: 8080
- backendRefs:
- group: net.gke.io
kind: ServiceImport
name: store
port: 8080
Run the following command to deploy it.
kubectl apply --context your-west-context -f httproute.yaml
To verify our setup
kubectl get gateway internal-cross-region-gateway -n store -o yaml --context your-west-context
kubectl get httproute store-route -n store -o yaml --context your-west-context
Lets' get the IP address of our gateway
kubectl get gateway cross-region-gateway -n store --context your-west-context
-o=jsonpath="{.status.addresses[*].value}".
And the issue test request out to the public ip
curl -H "host: store.example.internal" http://VIP_WEST/west
curl -H "host: store.example.internal" http://VIP_WEST/east
Comments