AKS Cluster Module - Reference Guide

Overview #

The Kosmos AKS Cluster Module is a Terraform module that creates a secure Azure Kubernetes Service (AKS) cluster integrated with the Kosmos platform. This module follows Azure Security checklist v3.0.1 requirements and provides a production-ready Kubernetes cluster with comprehensive security features.

Requirements #

Artifacts #

Download the Terraform module from the Terraform Artifacts page:

Quick Start #

module "aks" {
  source = "./kosmos/aks-cluster-module"

  resource_group_name         = "my-resource-group"
  kosmos_access_key           = "your-kosmos-access-key"
  kosmos_user                 = "kosmosuser"
  fleet_name                  = "fleet1"
  k8s_api_allowed_cidr_ranges = ["210.94.41.89/32"]
}

Inputs #

Examples #

Basic Usage #

module "aks" {
  source = "./kosmos/aks-cluster-module"

  resource_group_name         = "my-resource-group"
  kosmos_access_key           = "your-kosmos-access-key"
  kosmos_user                 = "kosmosuser"
  fleet_id                    = "fleet1"
  k8s_api_allowed_cidr_ranges = ["0.0.0.0/0"]
  jumphost_allowed_cidr_ranges = ["210.94.41.89/32"]
}

Custom VPC Usage #

module "aks" {
  source = "./kosmos/aks-cluster-module"

  resource_group_name         = "my-resource-group"
  fleet_name                  = "fleet1"
  kosmos_access_key           = "your-kosmos-access-key"
  kosmos_user                 = "kosmosuser"
  k8s_api_allowed_cidr_ranges = ["0.0.0.0/0"]
  jumphost_allowed_cidr_ranges = ["210.94.41.89/32"]

  # VPC configuration
  create_vnet     = false
  vnet_name       = "existing-vnet"
  aks_subnet_name = "aks-subnet"
}

Advanced Configuration #

module "aks" {
  source = "./kosmos/aks-cluster-module"

  resource_group_name         = "my-resource-group"
  kosmos_access_key           = "your-kosmos-access-key"
  kosmos_user                 = "kosmosuser"
  fleet_name                  = "fleet1"
  k8s_api_allowed_cidr_ranges = ["0.0.0.0/0"]
  jumphost_allowed_cidr_ranges = ["210.94.41.89/32"]

  # VPC configuration
  create_vnet     = false
  vnet_name       = "existing-vnet"
  aks_subnet_name = "aks-subnet"

  # Advanced configuration
  location     = "Koreacentral"
  name_prefix  = "kosmos"
  name_suffix  = "prod"
  environment  = "production"
  cluster_name = "kosmos-aks-prod"

  # Custom node pools
  node_pools = [
    {
      name            = "systemnode"
      count           = 3
      max_pods        = 50
      os_disk_size_gb = 50
      vm_size         = "Standard_DS3_v2"
      mode            = "System"
      os_type         = "Linux"
      os_disk_type    = "Managed"
    },
    {
      name            = "usernode"
      count           = 5
      max_pods        = 100
      os_disk_size_gb = 100
      vm_size         = "Standard_DS4_v2"
      mode            = "User"
      os_type         = "Linux"
      os_disk_type    = "Managed"
    }
  ]

  # Extra node rules
  additional_nsg_rules = {
    "all-inbound" = {
      name                       = "Allow-Every-Port-between-Services-Inbound"
      priority                   = 1000
      direction                  = "Inbound"
      access                     = "Allow"
      protocol                   = "*"
      source_port_range          = "*"
      destination_port_range     = "*"
      source_address_prefixes    = ["10.0.4.0/24", "10.0.10.0/24"]
      destination_address_prefix = "*"
      description                = "Allow All Inbound between services"
    },
    "all-outbound" = {
      name                         = "Allow-Every-Port-between-Services-Outbound"
      priority                     = 1000
      direction                    = "Outbound"
      access                       = "Allow"
      protocol                     = "*"
      source_port_range            = "*"
      destination_port_range       = "*"
      source_address_prefix        = "*"
      destination_address_prefixes = ["10.0.4.0/24", "10.0.10.0/24"]
      description                  = "Allow All Outbound between services"
    }
  }
  # Network configuration
  address_space  = ["10.0.0.0/16"]
  service_cidr   = "10.0.10.0/24"
  dns_service_ip = "10.0.10.10"
}

Resources Created #

• Storage Account: Private storage account with diagnostic settings
• Jump Host: Virtual Machine used to access the cluster
• Virtual Network: With aks and jumphost subnets
• Network Security Groups: For aks and jumphost subnets
• Virtual Network Flow Logs: For network monitoring
• Kosmos Cluster: Integration with Kosmos platform
• Azure Kubernetes Service: Managed Kubernetes cluster
• Kubernetes Cluster Flow Logs: For cluster monitoring

Samsung Security Checklist #

List of checklist that conform the Samsung Security Checklist #

3. Managed Identities

• No RBAC permission is granted to Managed Identity
• Managed Identity does not have Subscription type roles
• Managed Identity is created in the same subscription as resources
• External managed identities have mandatory tags

5. Containers

• Kubernetes service API access is configured with authorized IP ranges
• Authorized IP ranges is set to control which IP can access the cluster
• CIS Driver is enabled in the cluster
• Logs are stored in storage account with diagnostic settings

14. Storage Accounts

• Storage accounts are set to private access
• Copy operations are limited to PrivateLink
• Secure encryption policies are applied
• HTTPS communication is enforced
• Access keys are disabled for private storage
• Diagnostic settings are enabled with proper retention

15. Network Security Group

• All deny rules are registered in inbound/outbound rules
• All rules are properly managed according to rules
• NSG Flow logs are enabled and maintained
• Flow logs are stored with 365-day retention

18. Virtual Network

• Virtual network is properly separated into subnets
• Private subnets do not use service endpoints
• Virtual network flow logs are enabled

List of checklist that does not conform to the Samsung Security Checklist #

This script has not implemented the following security checklist:

• Managed Identity should not be granted Privileged administrator roles

  Managed Identity is used for AKS cluster, which automatically is granted Contributor role to Cluster Node’s resource group.

• Register All Deny Rule in In/Outbound Rule of Network Security Group

  We can’t setup all-deny-inbound or all-deny-outbound rule in auto-generated NSG for AKS nodes.

Required Permissions #

The following Azure permissions are required:

Storage Permissions #

• Microsoft.Storage/storageAccounts/* - Create and manage storage accounts

Network Permissions #

• Microsoft.Network/virtualNetworks/* - Create and manage virtual networks
• Microsoft.Network/networkSecurityGroups/* - Create and manage network security groups
• Microsoft.Network/networkWatchers/* - Configure network monitoring and flow logs
• Microsoft.Network/privateEndpoints/* - Create and manage private endpoints
• Microsoft.Network/privateDnsZones/* - Manage private DNS zones
• Microsoft.Network/publicIPAddresses/* - Manage public IP addresses
• Microsoft.Network/loadBalancers/* - Create and manage load balancers
• Microsoft.Network/natGateways/* - Create and manage NAT gateways
• Microsoft.Network/networkInterfaces/* - Manage network interfaces
• Microsoft.Network/routeTables/* - Manage route tables

Container Service Permissions #

• Microsoft.ContainerService/managedClusters/* - Create and manage AKS clusters

Monitoring Permissions #

• Microsoft.Insights/diagnosticSettings/* - Configure diagnostic settings

Identity Permissions #

• Microsoft.ManagedIdentity/userAssignedIdentities/* - Create and manage managed identities

Authorization Permissions #

• Microsoft.Authorization/roleDefinitions/* - Manage role definitions
• Microsoft.Authorization/roleAssignments/* - Assign roles to resources

Compute Permissions #

• Microsoft.Compute/virtualMachines/* - Create and manage virtual machines
• Microsoft.Compute/disks/* - Manage compute disks

Container Registry Permissions #

• Microsoft.ContainerRegistry/registries/* - Create and manage container registries

Operational Insights Permissions #

• Microsoft.OperationalInsights/workspaces/* - Manage Log Analytics workspaces

Resource Management Permissions #

• Microsoft.Resources/subscriptions/resourcegroups/* - Manage resource groups

Related Documentation #

• Kosmos Provider Documentation
• Setting Up the Minimum Permission Role
• Azure Kubernetes Service (AKS) Documentation
• Terraform Documentation

Download the full assets

Edit this page on GitHub