# ServingStack Source: /reference/servingstacks/ A ServingStack installs the serving substrate on a Kubernetes cluster: LeaderWorkerSet (Standard) or Grove and the KAI Scheduler (Dynamo), plus Gateway API, cert-manager, and Prometheus. Apply instances as `apiVersion: infrastructure.modelplane.ai/v1alpha1`, `kind: ServingStack`. ## Example ```yaml apiVersion: infrastructure.modelplane.ai/v1alpha1 kind: ServingStack metadata: name: west-gke-stack namespace: platform spec: secrets: - type: Kubeconfig name: west-gke-kubeconfig key: kubeconfig - type: GoogleApplicationCredentials name: west-gke-sa-key key: private_key versions: certManager: "v1.21.1" envoyGateway: "v1.8.4" gateway: listeners: - name: http port: 80 protocol: HTTP ``` ## Definition The CompositeResourceDefinition this reference is generated from, with the complete OpenAPI schema, validation rules, and defaults: ```yaml apiVersion: apiextensions.crossplane.io/v2 kind: CompositeResourceDefinition metadata: name: servingstacks.infrastructure.modelplane.ai spec: group: infrastructure.modelplane.ai names: categories: - crossplane - modelplane kind: ServingStack plural: servingstacks shortNames: - ss scope: Namespaced versions: - name: v1alpha1 referenceable: true additionalPrinterColumns: - name: GATEWAY type: string jsonPath: .status.gateway.address schema: openAPIV3Schema: description: >- A ServingStack installs the serving substrate on a Kubernetes cluster: LeaderWorkerSet (Standard) or Grove and the KAI Scheduler (Dynamo), plus Gateway API, cert-manager, and Prometheus. properties: spec: description: ServingStackSpec defines the desired state of ServingStack. required: - secrets # A ServingStack is machine-generated, but the API server still # serves and accepts it, so this lets the compose function trust # that a Kubeconfig secret is present rather than checking in code. # The kubeconfig provides the cluster endpoint and CA cert every # ProviderConfig needs; a cloud-specific credential secret is # optional and only layers identity-based auth on top. x-kubernetes-validations: - rule: "self.secrets.exists(s, s.type == 'Kubeconfig')" message: spec.secrets must include a Kubeconfig entry. properties: secrets: type: array description: >- Secrets used to authenticate to the target cluster. Typically sourced from a GKECluster's status.secrets. Secrets are in the same namespace as this ServingStack unless an entry says otherwise. A Kubeconfig secret is required. If a cloud identity secret is present, the serving stack authenticates as that identity instead of relying on the kubeconfig's embedded credentials. minItems: 1 maxItems: 8 items: type: object required: - type - name - key properties: type: type: string description: >- The type of credential this secret contains. Kubeconfig is required. Any other value is a cloud identity type; when present, the serving stack authenticates to the cluster as that identity instead of using the kubeconfig's embedded credentials. enum: - Kubeconfig - GoogleApplicationCredentials - AWSWebIdentityCredentials - NebiusServiceAccountCredentials name: type: string description: Name of the Secret. maxLength: 253 key: type: string description: Key within the Secret that holds the credential data. maxLength: 253 namespace: type: string description: >- Namespace of the Secret, when it isn't this ServingStack's namespace. Set on cloud identity entries whose credential is the Secret the cloud provider's ProviderConfig references. maxLength: 253 stack: type: string default: Standard description: >- Which serving stack this installs. Standard (the default) is the Modelplane-composed serving layer: a Deployment or LeaderWorkerSet, Gateway API, and the endpoint picker. Dynamo swaps in NVIDIA's components: Grove with the KAI Scheduler for multi-node gang scheduling, and a shared ModelExpress server for weight distribution. Propagated from the InferenceCluster. enum: - Standard - Dynamo nvidiaDriverRoot: type: string default: "/" description: >- Host path where the NVIDIA driver is installed, passed to the DRA driver as nvidiaDriverRoot. Defaults to / (the upstream default), which suits EKS and self-managed clusters. Set it for platforms that install the driver elsewhere — GKE uses /home/kubernetes/bin/nvidia. A non-default value also makes the serving stack compose a ResourceQuota permitting the DRA driver's system-critical pods, which GKE requires. The cluster composition sets this; the serving stack never inspects its own cloud. maxLength: 512 versions: type: object description: >- Version pins for each component. Defaults are the latest tested combination. Override individual versions to upgrade components independently. properties: certManager: type: string default: "v1.21.1" description: cert-manager chart version. minLength: 1 maxLength: 32 trustManager: type: string default: "v0.24.0" description: >- trust-manager chart version. trust-manager distributes the cluster gateway's CA certificate without its private key, which is what lets the control plane read the certificate to hand to a fleet gateway. minLength: 1 maxLength: 32 envoyGateway: type: string default: "v1.8.4" description: >- Envoy Gateway chart version. Must support InferencePool backend resources (the disaggregated-serving routing path), which requires v1.8.x or newer; older releases lack the Gateway API CRDs (ListenerSet) the AI Gateway needs. Envoy AI Gateway v1.1.x is tested against Envoy Gateway v1.8.x with Gateway API v1.5.x, so v1.9.x is out of range until the AI Gateway release that pairs with it. minLength: 1 maxLength: 32 prometheus: type: string default: "72.6.2" description: kube-prometheus-stack chart version. minLength: 1 maxLength: 32 nodeFeatureDiscovery: type: string default: "0.18.3" description: >- Node Feature Discovery chart version. NFD labels GPU nodes so the NVIDIA DRA driver targets its kubelet plugin to them. minLength: 1 maxLength: 32 nvidiaDraDriver: type: string default: "0.4.0" description: >- NVIDIA DRA driver chart version. Publishes GPUs as DRA ResourceSlices and the gpu.nvidia.com DeviceClass that ModelReplica ResourceClaims bind through. minLength: 1 maxLength: 32 standard: type: object description: >- Version pins for Standard-stack components. Ignored when stack is Dynamo. properties: leaderWorkerSet: type: string default: "v0.8.0" description: LeaderWorkerSet chart version. minLength: 1 maxLength: 32 dynamo: type: object description: >- Version pins for Dynamo-stack components. Ignored when stack is Standard. properties: grove: type: string default: "v0.1.0-alpha.12-rc2" description: >- Grove chart version. Grove gang-schedules a multi-node engine as a PodCliqueSet when stack is Dynamo. Pinned at v0.1.0-alpha.12-rc2 or later: earlier releases append Grove's own env vars after template env instead of prepending them (grove#753), which blocks Modelplane from aliasing MODELPLANE_LEADER_ADDRESS to them. minLength: 1 maxLength: 32 kaiScheduler: type: string default: "v0.16.8" description: >- KAI Scheduler chart version. Grove hands a PodCliqueSet's gang-scheduling to KAI, which binds each gang all-or-nothing against a Queue. minLength: 1 maxLength: 32 modelExpress: type: string default: "0.4.1" description: >- ModelExpress server image tag. The metadata-only server coordinates peer-to-peer weight transfer between engine pods that opt into --load-format modelexpress. minLength: 1 maxLength: 32 gateway: type: object description: >- Configuration for the cluster's inference traffic gateway. properties: className: type: string default: envoy description: >- GatewayClass name. Override if the cluster already has a GatewayClass named envoy. minLength: 1 maxLength: 63 hostname: type: string description: >- The name this cluster's gateway is reached by, projected from the InferenceCluster. The gateway serves a certificate for it, so an InferenceGateway can originate TLS and know it reached the right cluster. Without it the gateway serves plain HTTP and carries no traffic, since an InferenceGateway addresses a cluster by name. minLength: 1 maxLength: 253 clientCAs: type: array description: >- PEM certificates of the CAs whose client certificates this gateway accepts, one per InferenceGateway in the fleet. Projected from the InferenceCluster, which reads them from each gateway's status. Presenting one of these is how a caller proves it is a fleet gateway. Requests without one are refused, which is what makes a fleet gateway the only thing that can reach the engines behind this cluster's gateway. maxItems: 32 items: type: object required: [name, certificate] properties: name: type: string description: The InferenceGateway this CA belongs to. minLength: 1 maxLength: 253 certificate: type: string description: The CA certificate, PEM encoded. minLength: 1 maxLength: 16384 listeners: type: array description: >- Gateway listeners. Defaults to a single HTTP listener on port 80 if not specified. maxItems: 8 x-kubernetes-list-type: map x-kubernetes-list-map-keys: - name items: type: object required: - name - port - protocol properties: name: type: string description: Unique listener name. maxLength: 63 minLength: 1 port: type: integer description: Port number for this listener. minimum: 1 maximum: 65535 protocol: type: string description: Protocol for this listener. enum: - HTTP - TCP type: object status: description: ServingStackStatus defines the observed state of ServingStack. properties: gateway: type: object description: Status of the cluster's inference gateway. properties: address: type: string maxLength: 256 description: >- The gateway's external address, once assigned by the cloud load balancer. caCertificate: type: string maxLength: 16384 description: >- PEM certificate of the CA that signed this gateway's serving certificate. An InferenceGateway validates the gateway against it, so it reaches the cluster it meant to and not whatever answers on that address. type: object required: - spec type: object served: true ```