Feature: Support Deployment Stacks

[dtzar]

Support Azure deployment stacks.

[matthchr]

Is this just "we want support for arbitrary resources" (tracking as #4028), or do you actually think that there's a use for the Stacks grouping mechanism in Kubernetes? AFAIK the problem Stacks is solving is "it's hard to delete resources in ARM because nobody likes Complete deployment mode". That same problem doesn't exist in Kubernetes (and by extension doesn't exist in ASO).

if you want to delete a resource, you just kubectl delete it (or delete it from Flux or whatever).

[bmoore-msft]

DeploymentStacks enables a different lifecycle boundary for an arbitrary set of resources. CRUD on the resources as a group regardless of where they reside (e.g. different subs/rgs) is part of it - RBAC/deny is another. But the grouping is the key.

[matthchr]

I guess the question is, does that same grouping make sense in Kubernetes?

I'm not sure that it does because:

1. The namespace already exists
2. If you want grouping within a namespace, you would just do it with your own labels and/or YAML groupings locally, i.e. group1.yaml and group2.yaml (== 2 Helm charts or 2 Kustomize roots), then you can manage those resources independently even though they go into the same namespace.

I don't know that "Azure keeps track of which resources are grouped" is all that compelling in a k8s native world. Something like hierarchical namespaces feels more native?

One of the challenges with Deployment Stacks is that they're pretty coupled to BICEP/ARM templates, which ASO doesn't use, and they don't (at least at first glance) compose well with the CRD model in k8s. I suppose that you could annotate a resource as belonging to a deployment stack and somehow do management of the resource through the stack, but surface the state on the individual CRs... but it would be difficult to coordinate.

[dtzar]

If deployment stacks requires someone to create and maintain a BICEP/ARM template, then I'd have to lean towards this doesn't make the most sense. My thinking is there might be a way to represent a "stack" of resources via helm/K8s YAML provisioned by ASO and then put them into an Azure "deployment stack" to gain the top three why bullets as advertised.

Based on the API, do you think this would be reasonably possible?
e.g. I see the az stack group create --template-file parameter, but this parameter is not required. However, assuming you can provision resources in ASO and then assign them to a deployment stack the question then would be can you do anything useful/meaningful afterwards without a BICEP file?

[matthchr]

I do think that there is a bit of an impedance mismatch between how ARM is doing things here versus how Kubernetes "wants" things/how ASO does them. We think it makes sense to keep this open as there may be a way to reconcile the two such as using an annotation on the existing CRs to "group" them into a stack somehow - but definitely the templating part of stacks, and the part that allows resources embedded directly in it, don't really make sense in the context of Kubernetes.