DevSpace in Kosmos

Prerequisite #

• You must have a Corp-AD account.
• You must have access to Kosmos . For more details on how to request access to Kosmos, please refer to Kosmos Account Request .

Request a DevSpace for your project #

You can request a Fleet and DevSpace for your project by submitting a Knox request using Tenant Management .

Once assigned, the Devspaces will appear in your Kosmos account. Visit: https://console.kosmos.spcplatform.com/devspaces

DevSpace in Kosmos #

Kosmos DevSpaces are high-level logical entities designed to organize and manage virtual clusters effectively. They act as collections or workspaces containing virtual clusters, simplifying user and team access management as well as resource allocation.

Key elements of a DevSpace #

1. Users A DevSpace allows you to assign roles to users or teams as either Owners or User. Each member can be assigned one of the following roles, determining their level of access within the DevSpace:

   • Admin: Full access and administrative control.
   • Owner: Ownership and management rights over the DevSpace.
   • User: Limited access, primarily for interacting with resources.

2. Allowed templates DevSpace owners can restrict the types of virtual clusters that can be created within a DevSpace by specifying “Allowed Templates.” These templates are predefined cluster configurations built into Kosmos, ensuring consistency and governance.

3. Allowed clusters Allowed Clusters are the underlying Loft host clusters that support the virtual clusters created within a DevSpace. This ensures that virtual clusters are only created on approved infrastructure.

4. Quotas Quotas define resource usage limits across the DevSpace or on a per-user basis. These limits apply to virtual cluster instances, which share resources within the DevSpace. If a resource limit is exceeded, attempts to create new virtual cluster instances will fail, triggering a “resource limit exceeded” error.

By defining these elements, Kosmos DevSpaces provide a structured and efficient way to manage virtual clusters and resources at scale.

DevSpace management #

A DevSpace is a workspace where users or teams can create virtual clusters and manage access and resource usage.

Technically, a DevSpace is defined using a declarative YAML approach, similar to Kubernetes manifests. It is implemented as a Kubernetes Custom Resource (CR), and requests are handled by the Kosmos Operator, a custom Kubernetes operator developed by the Kosmos team.

Note: DevSpace parameters cannot be changed by Kosmos users

This document describes the key sections of the DevSpace.yaml file.

DevSpace manifest structure #

apiVersion: storage.kosmos.spcplatform.com/v2alpha1
kind: DevSpace
metadata:
  # more details below
spec:
  # more details below

Field descriptions #

• apiVersion Defines the Kosmos object version in use. The above example uses v2alpha1.

• kind Defines the resource type. For a DevSpace, this is always DevSpace, and the Kosmos operator acts on this request.

• metadata Defines metadata such as the resource name and labels. Used to identify and manage DevSpaces.

• spec Defines the desired state of the DevSpace. The Kosmos operator continuously works to bring the cluster to this state.

Metadata section #

The metadata section in a DevSpace manifest defines two fields:

metadata:
  name: devspace-v2alpha1-test
  labels:
    venv: dev

Field descriptions:

• name Name of the DevSpace.

• labels A key-value map used for identifying and filtering DevSpaces.

Spec section #

The spec section defines the desired state of the DevSpace.

Example spec #

apiVersion: storage.kosmos.spcplatform.com/v2alpha1
kind: DevSpace
metadata:
  name: my-dev-space
spec:
  displayName: my-dev-space
  owner:
    user: example
  allowedClusters:
    - name: dx-aws-sandbox-dev-apne2-eks
    - name: dx-spc-devpool-dev-apne1-mks
  allowedTemplates:
    - kind: VirtualClusterTemplate
      name: vcluster-pro
      group: storage.kosmos.spcplatform.com
  users:
    - example
  description: Testing v2alpha1 devspace
  defaultDevSpaceRole: devspace-admin
  permissions:
    - fleetRole: devspace-admin
      teams: []
      users:
        - example
  quotas:
    devspace:
      limits.cpu: '16'
      limits.memory: 64Gi
      limits.ephemeral-storage: 200Gi
      requests.ephemeral-storage: 10Gi
      gp3.storageclass.storage.k8s.io/requests.storage: 500Gi
      gp3.storageclass.storage.k8s.io/persistentvolumeclaims: '10'
      services.loadbalancers: '3'
      services.nodeports: '0'

DevSpace Configuration Fields #

displayName #

• The name of the DevSpace.
• Can be user-provided or auto-generated.
• Currently defaults to the same value as metadata.name.

description #

• Optional field describing the purpose of the DevSpace.

allowedClusters #

• Defines the list of host clusters backing the DevSpace.
• Users can create virtual clusters only on these clusters.
• Effectively controls where DevSpace members can deploy workloads.

allowedTemplates #

• Specifies the pre-defined virtual cluster templates available to users.
• DevSpace members can only create virtual clusters using these templates.

Why this matters:

• Ensures standardization and governance

• Gives administrators control over:

  • Cluster configurations
  • Resource usage
  • Deployment patterns

Template structure:

Each template is defined as an object in the YAML file:

• kind → Always the same
• group → Always the same
• name → Unique template identifier

owner #

• Defines the owner of the DevSpace.
• Typically has the highest level of control.

members #

• Defines who can access the DevSpace.
• A member can be:
  • A user
  • A team

Access levels (clusterRole)

• loft-management-project-owner → Full control
• loft-management-project-user → Standard access
• loft-management-project-viewer → Read-only access

permissions #

• Maps roles to users or teams.
• Defines fine-grained access control within the DevSpace.

defaultDevSpaceRole #

• Default role assigned when:
  • No explicit role is defined for a user or team
• Ensures no user is left without permissions

quotas #

• Defines resource limits using key-value pairs.
• Can be applied to:
  • Users
  • Teams

Understanding Kosmos Quotas #

Kosmos follows a layered architecture built on top of Kubernetes.

Where quotas can be applied #

• DevSpaces
• Clusters
• Kubernetes-native resources

How quotas are enforced #

Most quotas are implemented using Kubernetes mechanisms:

• Resource Quotas
• Limit Ranges

These help control:

• Resource consumption
• Fair usage
• Cluster stability

Example Quota #

[gp3.storageclass.storage.k8s.io/requests.storage] #

• Applies to a specific storage class (e.g., gp3)
• Defines the maximum total storage that can be requested

Note:

• Across all Persistent Volume Claims (PVCs) using the given storage class,
• The sum of storage requests must not exceed this value.

Spec field descriptions #

displayName #

Generated or user-provided DevSpace name. Currently, this is the same as metadata.name but name can be defined as per user.

description #

Optional description of the DevSpace.

owner #

Defines the user or team that owns of the DevSpace.

allowedClusters #

A list of host clusters on which DevSpace users can create virtual clusters.

allowedTemplates #

Pre-defined virtual cluster templates that users can use to create virtual clusters.

• Regular users can only create virtual clusters using these templates.
• This gives DevSpace administrators control over what types of virtual clusters are allowed.

Each template entry includes:

• kind (always the same)
• group (always the same)
• name (template name)

users #

A user can be either a user or a team.

Each user is assigned a role using a cluster role.

permissions #

A list of DevSpace roles and the users or teams assigned to each role.

defaultDevSpaceRole #

If no role is explicitly assigned to a user or team, this role is applied by default.

quotas #

Defines Kosmos quotas as key-value pairs. Quotas can be applied at multiple levels, including:

• DevSpaces
• Clusters
• Kubernetes built-in resources

Kosmos uses Kubernetes constructs such as:

• ResourceQuotas
• LimitRanges

For more details, refer to:

• Kubernetes Resource Quotas
• Kubernetes Limit Ranges

Supported quotas #

• limits.cpu: ‘16’
• limits.memory: 64Gi
• limits.ephemeral-storage: 200Gi
• requests.ephemeral-storage: 10Gi
• gp3.storageclass.storage.k8s.io/requests.storage: 500Gi
• gp3.storageclass.storage.k8s.io/persistentvolumeclaims: ‘10’
• services.loadbalancers: ‘3’
• services.nodeports: ‘0’

Available quotas #

Note: Ingresses are not synced to the host cluster by default .

If the vCluster template enables ingress syncing, you can use:

• count/ingresses.networking.k8s.io
• count/ingresses

Supported quota keys #

limits.cpu,
limits.memory,
limits.ephemeral-storage,
requests.cpu,
requests.memory,
requests.ephemeral-storage,
cpu,memory,
ephemeral-storage,
requests.storage,
services,
services.loadbalancers,
services.nodeports,
persistentvolumeclaims,
spaceinstances,
spaceinstances-active,
virtualclusterinstances,
virtualclusterinstances.active,
count/*,
count/agentauditevents,
count/announcements,
count/apps,
count/backups,
count/clusters,
count/clusteraccesses,
count/clusters,
count/clusterconnect,
count/clusters,
count/clusterroletemplates,
count/clusters,
count/configs,
count/devpodworkspaceinstances,
count/devpodworkspacetemplates,
count/directclusterendpointtokens,
count/events,
count/features,
count/ingressauthtokens,
count/kiosk,
count/licenses,
count/licensetokens,
count/loftupgrades,
count/ownedaccesskeys,
count/policyviolations,
count/projects,
count/projectsecrets,
count/projects,
count/redirecttokens,
count/resetaccesskeys,
count/runners,
count/selves,
count/selfsubjectaccessreviews,
count/sharedsecrets,
count/spaceconstraints,
count/spaceinstances,
count/spacetemplates,
count/subjectaccessreviews,
count/tasks,
count/teams,
count/users,
count/virtualclusterinstances,
count/virtualclustertemplates,
count/sleepmodeconfigs,
count/spaces,
count/clusterquotas,
count/helmreleases,
count/virtualclusters,
count/localclusteraccesses,
count/chartinfos,
count/helmreleases,
count/customresourcedefinitions,
count/jspolicies,
count/jspolicyviolations,
count/virtualclusters,
count/clusterquotas,
count/localusers,
count/statefulsets,
count/deployments,
count/events,
count/pods,
count/nodes,
count/namespaces,
count/selfsubjectaccessreviews,
count/clusterroles,
count/clusterrolebindings,
count/roles,
count/rolebindings,
count/secrets,
count/configmaps,
count/services,
count/users,
count/storageclasses,
count/pods

In this section

• Getting started with vClusters
  • Create a virtual cluster via the Kosmos UI
  • Creating a vCluster using CLI
  • Setup Ingress with vcluster
• DevSpace Permissions and Access Management
• Using DevSpace Secrets
• Pod Identity in vCluster

Edit this page on GitHub