GKE - Create and Import Cluster

Create GKE cluster using Kosmos

Prerequisites

Required tools:

  • Kosmos CLI

  • Google Cloud CLI (gcloud)

  • gke-gcloud-auth-plugin :

    gcloud components install gke-gcloud-auth-plugin
    

    Add to your shell profile (~/.bashrc, ~/.zshrc, or equivalent):

    export USE_GKE_GCLOUD_AUTH_PLUGIN=True
    # Homebrew (macOS): source "$(brew --prefix)/share/google-cloud-sdk/path.zsh.inc"
    # Linux/manual: source "$HOME/google-cloud-sdk/path.bash.inc"
    
  • kubectl

Required access:

  • Kosmos Access Key – Authenticate with Kosmos console
  • GCP IAM Account with permissions to create Workload Identity pools, service accounts, and GKE clusters

Environment variables:

Set these before starting:

export PROJECT_ID="your-gcp-project"
export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")
export FLEET_ID="your-fleet-id"
export CLUSTER_NAME="your-cluster-name"
export REGION="us-west1"
export WI_POOL_ID="kosmos-wi-pool"
export WI_PROVIDER_ID="kosmos-oidc-provider"
export SA_NAME="kosmos-gke-sa"
export SA_EMAIL="${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"
export VPC_NAME="your-vpc"
export SUBNET_NAME="your-subnet"
export USER="your-kosmos-username"  # Your Kosmos console username for adminUsers

Network configuration reference

GKE clusters require several network components. Understanding these prerequisites helps avoid common configuration issues.

Required GCP resources

Before creating a cluster, these resources must exist (or be created in step 6):

ResourceDescriptionCreate vs Import
VPC NetworkVirtual network for cluster nodesCreate: can auto-create or use existing. Import: must pre-exist
SubnetIP range for node primary IPsCreate: can auto-create or use existing. Import: must pre-exist

IP address ranges

GKE uses three distinct IP ranges. Plan these carefully as Service CIDR cannot be changed after cluster creation.

FieldPurposeDefault if emptyNotes
clusterIpv4CidrPod IP addressesAuto: /14 from 10.0.0.0/8 (~262k IPs)Can conflict with existing routes if auto-allocated
Service CIDRClusterIP servicesGKE-managed: 34.118.224.0/20Immutable after creation
masterIpv4CidrBlockControl plane IPsRequired only if enablePrivateNodes: trueMust be /28, cannot overlap node/pod ranges

Node subnet sizing

Cluster sizeRecommended subnetExample CIDR
≤250 nodes/24 (256 IPs)10.0.0.0/24
≤1000 nodes/22 (1024 IPs)10.0.0.0/22

Common CIDR conflicts

Avoid these overlaps when planning IP ranges:

  • Pod CIDR must not overlap VPC subnets, VPN routes, or peered VPCs
  • Service CIDR must not overlap Pod CIDR or node subnets
  • Master CIDR (private clusters) must not overlap any other range

Tip: Use gcloud compute networks subnets list and gcloud compute routes list to check existing allocations before creating a cluster.

Kubernetes version selection

Check available versions in your region:

# List supported control plane versions
gcloud container get-server-config --region=${REGION} \
  --format="yaml(validMasterVersions)"

# Get the latest supported version
gcloud container get-server-config --region=${REGION} \
  --format="value(validMasterVersions[0])"

Warning: Hardcoded versions become unsupported. The admission webhook will reject with: k8s version 'X.X.X-gke.XXX' is not supported for master


0. Enable required GCP APIs

gcloud config set project $PROJECT_ID

gcloud services enable \
  compute.googleapis.com \
  container.googleapis.com \
  iam.googleapis.com \
  iamcredentials.googleapis.com \
  cloudresourcemanager.googleapis.com

1. Create workload identity pools

1-A. Create workload-identity-pools

Sample pool name: [user-test-pool]

gcloud iam workload-identity-pools create [user-test-pool] --display-name="user-test-pool" --location="global"

Output:

Created workload identity pool [user-test-pool].

1-B. Describe workload-identity-pools

gcloud iam workload-identity-pools describe [user-test-pool] --location="global"

Output:

displayName: user-test-pool
name: projects/[AccountID]/locations/global/workloadIdentityPools/user-test-pool
state: ACTIVE

2. Create OIDC provider

2-A. Create OIDC provider

provider : [user-test-pool-provider]<– sample name for the provider Use 1-A [user-test-pool] for “workload-identity-pool"

gcloud iam workload-identity-pools providers create-oidc [user-test-pool-provider] \
  --location="global" \
  --workload-identity-pool="user-test-pool" \
  --attribute-mapping="google.subject=assertion.sub" \
  --issuer-uri="https://console.kosmos.spcplatform.com/kosmos-oidc" \
  --allowed-audiences="kosmos-operator" \
  --display-name="user-test-provider"

Output:

Created workload identity pool provider [user-test-pool-provider].

2-B. List workload-identity-pools providers

gcloud iam workload-identity-pools providers list \
  --location=global \
  --workload-identity-pool=user-test-pool

Output:

attributeMapping:
  google.subject: assertion.sub
displayName: user-test-provider
name: projects/[AccountID]/locations/global/workloadIdentityPools/[user-test-pool]/providers/[user-test-pool-provider]
oidc:
  allowedAudiences:
  - kosmos-operator
  issuerUri: https://console.kosmos.spcplatform.com/kosmos-oidc
state: ACTIVE

3. Create service account for external workload

3-A. Create service account

Sample name for service account

service-account : [user-test-service-name]

gcloud iam service-accounts create [user-test-service-name] --display-name="[user-test-service-name]"

3-B. Describe service account

gcloud iam service-accounts list --filter=[user-test-service-name]

Output:

DISPLAY NAME            EMAIL                                                      DISABLED
user-test-service-name  user-test-service-name@[PROJECT_ID].iam.gserviceaccount.com  False

4. Allow external identities to impersonate a service account

4-A. Get current project number

gcloud projects describe $(gcloud config get-value core/project) --format="value(projectNumber)"

4-B. Grant workload identity user role

Grant roles to external identities that meet a certain criteria.

Use 3-B Email for ${SERVICE_ACCOUNT_EMAIL}.

Use 4-A project number for ${PROJECT_NUMBER}.

Use 1-A name for ${POOL_ID}.

$ gcloud iam service-accounts add-iam-policy-binding ${SERVICE_ACCOUNT_EMAIL} \
    --role=roles/iam.workloadIdentityUser \
    --member="principal://iam.googleapis.com/projects/${PROJECT_NUMBER}/locations/global/workloadIdentityPools/${POOL_ID}/subject/fleet-${FLEET_ID}"

Output:

Updated IAM policy for serviceAccount [SERVICE_ACCOUNT_EMAIL].
bindings:
- members:
  - principal://iam.googleapis.com/projects/[PROJECT_NUMBER]/locations/global/workloadIdentityPools/[user-test-pool]/subject/fleet-${FLEET_ID}
  role: roles/iam.workloadIdentityUser
etag: BwYVoHWX4BQ=
version: 1

5. Add IAM policy with service accounts

5-A. Add IAM policy binding with roles

Use following roles for binding.

Roles:

  • Compute Viewer: roles/compute.viewer
  • Project Viewer: roles/viewer
  • Kubernetes Engine Admin: roles/container.admin
  • Service Account User: roles/iam.serviceAccountUser

Use 3-B email for [SERVICE_ACCOUNT_EMAIL].

$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
  --member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
  --role='roles/compute.viewer'

$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
  --member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
  --role='roles/viewer'

$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
  --member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
  --role='roles/container.admin'

$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
  --member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
  --role='roles/iam.serviceAccountUser'

$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
  --member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
  --role='roles/iam.workloadIdentityUser'

5-B. Describe project IAM policy

gcloud projects get-iam-policy [PROJECT_ID]

Output:

  ...
  - members:
    - serviceAccount:[SERVICE_ACCOUNT_EMAIL]
    role: roles/compute.viewer
  - members:
    - serviceAccount:[SERVICE_ACCOUNT_EMAIL]
    role: roles/viewer
  - members:
    - serviceAccount:[SERVICE_ACCOUNT_EMAIL]
    role: roles/container.admin
  - members:
    - serviceAccount:[SERVICE_ACCOUNT_EMAIL]
    role: roles/iam.serviceAccountUser
  - members:
    - serviceAccount:[SERVICE_ACCOUNT_EMAIL]
    role: roles/iam.workloadIdentityUser

6. Create VPC Network

6-A. Create Simple VPC Network

gcloud compute networks create ${VPC_NAME} --subnet-mode=custom --bgp-routing-mode=regional --mtu=1460

6-B. Create Subnet

gcloud compute networks subnets create ${SUBNET_NAME} --network=${VPC_NAME} --range=${CIDR_RANGE} --region=${REGION}

GKE - Create cluster

Create a sample file

  • Use 4-B ${FLEET_ID} as metadata > name.
  • Use target cluster’s name as ${CLUSTER_NAME}.
  • Use target cluster’s region as ${REGION}.
  • Use 3-B Email for ${SERVICE_ACCOUNT_EMAIL}.
  • Use 4-A project number for ${PROJECT_NUMBER}.
  • Use 1-B name for ${POOL_ID}.
  • Use 2-A [user-test-pool-provider] for ${USER_TEST_POOL_PROVIDER}.
  • Use 6-A name for ${VPC_NAME}.
  • Use 6-B name for ${SUBNET_NAME}.

FileName: gkecluster_sample.yaml

apiVersion: storage.kosmos.spcplatform.com/v1
kind: GKECluster
metadata:
  labels:
    app.kubernetes.io/name: ${CLUSTER_NAME}
    app.kubernetes.io/instance: ${FLEET_ID}
    app.kubernetes.io/part-of: kosmos
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: kosmos
  name: ${CLUSTER_NAME}
  namespace: ${FLEET_ID}
spec:
  description: "GKE cluster created via Kosmos CLI"
  name: "${CLUSTER_NAME}"
  authorization:
    adminUsers:
      - ${USER}
  gkeConfig:
    clusterName: "${CLUSTER_NAME}"
    description: "Kosmos managed GKE cluster"
    labels: {}
    region: "${REGION}"
    projectID: "${PROJECT_ID}"
    projectNumber: "${PROJECT_NUMBER}"
    kubernetesVersion: ""  # Empty = latest stable version
    loggingService: ""
    monitoringService: ""
    enableKubernetesAlpha: false
    clusterIpv4Cidr: ""  # Auto-allocated if empty
    ipAllocationPolicy:
      useIpAliases: true
    nodePools:
      - name: ${CLUSTER_NAME}-nodes
        autoscaling:
          enabled: false
        config:
          machineType: "e2-medium"
          diskSizeGb: 50   
          diskType: "pd-standard"  # https://docs.cloud.google.com/compute/docs/disks/persistent-disks#disk-types
        initialNodeCount: 1
        maxPodsConstraint: 110
        version: ""
        management:
          autoRepair: true
          autoUpgrade: true
    clusterAddons:
      httpLoadBalancing: true
      networkPolicyConfig: false
      horizontalPodAutoscaling: true
    networkPolicyEnabled: false
    network: ${VPC_NAME}
    subnetwork: ${SUBNET_NAME}
    privateClusterConfig:
      enablePrivateEndpoint: false
      enablePrivateNodes: false
      # Control plane CIDR - only used if enablePrivateNodes is true
      masterIpv4CidrBlock: "172.16.0.0/28"
    masterAuthorizedNetworks:
      cidrBlocks:
        # displayName is optional - human-readable label shown in GCP Console
        - cidrBlock: "YOUR_IP/32"  # Replace with your public IP
          displayName: "my-ip"
        # Kosmos platform IPs (required for management)
        - cidrBlock: "3.208.49.242/32"
          displayName: "kosmos-1"
        - cidrBlock: "35.172.72.171/32"
          displayName: "kosmos-2"
        - cidrBlock: "34.214.188.139/32"
          displayName: "kosmos-3"
        - cidrBlock: "35.165.241.214/32"
          displayName: "kosmos-4"
        - cidrBlock: "13.209.31.187/32"
          displayName: "kosmos-5"
        - cidrBlock: "15.165.61.114/32"
          displayName: "kosmos-6"
      enabled: true
    locations: []  # Required field - empty for regional clusters
    maintenanceWindow: ""
    imported: false
    zone: ""
    serviceAccount: ${SA_EMAIL}
    workloadIdentityPoolId: ${WI_POOL_ID}
    workloadIdentityProviderId: ${WI_PROVIDER_ID}

Login to Kosmos

kosmos login console.kosmos.spcplatform.com --access-key <access-key-of-cluster-owner>

Create GkeCluster

Important: Always specify the --fleet flag explicitly to avoid using defaults from ~/.kosmos/defaults.json.

kosmos create gke --file <YAML-file> --fleet ${FLEET_ID}

The GKE cluster and node are created and have status “Ready” on the GCP side. But on the Kosmos side, the cluster has status “connecting”. Execute the following command to see your cluster in Kosmos:

kosmos list gke --fleet <FLEET_ID>

Verify that you can see the new GKE cluster in GCP

gcloud container clusters list

Update the kube config file

gcloud container clusters get-credentials <cluster-name> --zone <zone>

Verify that kubectl works

kubectl get ns

Connect GkeCluster

Update the config and describe the Gkecluster and get the cluster-name in Annotations

  • The ${CLUSTER-NAME} is [CLUSTER_NAME]-[HASH_NUMBER]

  • [target-cluster-context] format is: gke_${PROJECT_NAME}${REGION}${CLUSTER_NAME}

  1. Describe the GkeCluster:

    kubectl describe gkeclusters.storage.kosmos.spcplatform.com ${CLUSTER_NAME} -n ${FLEET_ID}
    

    Extract <CLUSTER_NAME> from annotations.

  2. Connect to the cluster:

    gcloud container clusters get-credentials ${CLUSTER_NAME} --region=${REGION}
    kubectl config use-context [target-cluster-context]
    
  3. Connect the cluster with the ${CLUSTER_NAME} by the Kosmos.

    kosmos join cluster <cluster-name> --fleet <FLEET_ID>
    

    After the kosmos join cluster command installs the platform agent successfully, the status for the cluster is changed from “connecting” to “ready” on the Kosmos side.

Open a new Terminal Window and verify cluster usage

kosmos login console.kosmos.spcplatform.com --access-key <access-key-of-cluster-owner>
kosmos use cluster <cluster-name> --fleet ${FLEET_ID}
kubectl get ns

GKE - Import cluster

Import an existing GKE cluster into Kosmos for management.

Note: When importing, the GKE cluster must already exist. Kosmos will not create the cluster, only register it for management.

Prerequisites for import

Complete steps 0-5 from the Create workflow above to set up:

  • Workload Identity Pool and OIDC Provider
  • Service Account with required IAM roles

Get existing cluster details

Retrieve the cluster configuration to populate the import YAML:

gcloud container clusters describe ${CLUSTER_NAME} --region=${REGION} \
  --format="yaml(name,currentMasterVersion,network,subnetwork,clusterIpv4Cidr)"

Important: Copy the currentMasterVersion value for the kubernetesVersion field in your import YAML.

Update the kube config file

gcloud container clusters get-credentials ${CLUSTER_NAME} --region=${REGION}

Verify that kubectl works

kubectl get ns

Create import YAML file

Important: For imported clusters, you must set imported: true and match the existing cluster’s configuration (version, network, CIDR).

apiVersion: storage.kosmos.spcplatform.com/v1
kind: GKECluster
metadata:
  labels:
    app.kubernetes.io/name: ${CLUSTER_NAME}
    app.kubernetes.io/instance: ${FLEET_ID}
    app.kubernetes.io/part-of: kosmos
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: kosmos
  name: ${CLUSTER_NAME}
  namespace: ${FLEET_ID}
spec:
  description: "Existing GKE cluster imported via Kosmos CLI"
  name: "${CLUSTER_NAME}"
  authorization:
    adminUsers:
      - ${USER}
  gkeConfig:
    # Cluster identity - must match existing cluster
    clusterName: "${CLUSTER_NAME}"
    projectID: "${PROJECT_ID}"
    projectNumber: "${PROJECT_NUMBER}"
    region: "${REGION}"

    # IMPORTANT: imported=true tells Kosmos this cluster already exists
    imported: true

    # REQUIRED: Must match existing cluster version
    # Get from: gcloud container clusters describe ${CLUSTER_NAME} --region=${REGION} --format="value(currentMasterVersion)"
    kubernetesVersion: ""

    # Network settings - must match existing cluster
    network: ${VPC_NAME}
    subnetwork: ${SUBNET_NAME}
    clusterIpv4Cidr: ""  # Get from gcloud describe output

    # Workload Identity for Kosmos authentication
    serviceAccount: ${SA_EMAIL}
    workloadIdentityPoolId: ${WI_POOL_ID}
    workloadIdentityProviderId: ${WI_PROVIDER_ID}

    # Required fields (even if empty for regional clusters)
    locations: []
    labels: {}
    zone: ""

Login to Kosmos

kosmos login console.kosmos.spcplatform.com --access-key <access-key-of-cluster-owner>

Execute CLI to import cluster

Important: Always specify the --fleet flag explicitly.

kosmos create gke --file <YAML-file> --fleet ${FLEET_ID}

The GKE cluster is imported to Kosmos but has a status of “connecting”. Execute the following command to see your cluster in Kosmos:

kosmos list gke --fleet ${FLEET_ID}

Connect to the cluster

kosmos join cluster ${CLUSTER_NAME} --fleet ${FLEET_ID}

After the kosmos join cluster command installs the platform agent successfully, the status for the cluster is changed to “ready” on the Kosmos side.

Verify cluster usage

kosmos use cluster ${CLUSTER_NAME} --fleet ${FLEET_ID}
kubectl get ns

Key differences: Import vs Create

AspectCreate (imported: false)Import (imported: true)
GKE ClusterKosmos creates itMust pre-exist
Kubernetes VersionCan be empty (latest)Must match existing
Network settingsConfigured by KosmosMust match existing
Deletion behaviorKosmos deletes GKE clusterKosmos only removes from management

Troubleshooting

gke-gcloud-auth-plugin not found

Error:

executable gke-gcloud-auth-plugin not found

Solution: Install the plugin and add to PATH:

gcloud components install gke-gcloud-auth-plugin

# Add to ~/.bashrc or ~/.zshrc:
export USE_GKE_GCLOUD_AUTH_PLUGIN=True
export PATH="$(gcloud info --format='value(installation.sdk_root)')/bin:$PATH"

Fleet namespace mismatch

Error:

User "username" cannot create resource "gkeclusters" in namespace "wrong-fleet"

Solution: The Kosmos CLI may use a cached fleet from ~/.kosmos/defaults.json. Always specify the --fleet flag explicitly:

kosmos create gke --file cluster.yaml --fleet ${FLEET_ID}

Import validation error: locations field

Error:

spec.gkeConfig.locations: Invalid value: "null": spec.gkeConfig.locations in body must be of type array

Solution: Add locations: [] to your import YAML, even for regional clusters:

gkeConfig:
  locations: []

Cleanup

Delete cluster from Kosmos

kosmos delete gke --name ${CLUSTER_NAME} --fleet ${FLEET_ID}

Note: For clusters created via Kosmos (imported: false), this also deletes the GKE cluster in GCP. For imported clusters (imported: true), only the Kosmos registration is removed.

Wait for GKE cluster deletion (created clusters only)

# Monitor deletion (~3-5 minutes)
gcloud container clusters list --region ${REGION}

Delete VPC resources

# Delete subnet first
gcloud compute networks subnets delete ${SUBNET_NAME} --region=${REGION} --quiet

# Delete VPC
gcloud compute networks delete ${VPC_NAME} --quiet

Remove IAM bindings and service account

# Remove project IAM bindings
for role in roles/compute.viewer roles/viewer roles/container.admin roles/iam.serviceAccountUser; do
  gcloud projects remove-iam-policy-binding ${PROJECT_ID} \
    --member="serviceAccount:${SA_EMAIL}" \
    --role="${role}" --quiet
done

# Delete service account
gcloud iam service-accounts delete ${SA_EMAIL} --quiet

Delete Workload Identity resources

# Delete OIDC Provider first
gcloud iam workload-identity-pools providers delete ${WI_PROVIDER_ID} \
  --location="global" \
  --workload-identity-pool=${WI_POOL_ID} \
  --quiet

# Delete Workload Identity Pool
gcloud iam workload-identity-pools delete ${WI_POOL_ID} \
  --location="global" \
  --quiet

Verify cleanup

gcloud container clusters list --region ${REGION}
gcloud compute networks list --filter="name=${VPC_NAME}"
gcloud iam service-accounts list --filter="email:${SA_NAME}"
gcloud iam workload-identity-pools list --location=global
kosmos list gke --fleet ${FLEET_ID}

Edit this page on GitHub