Add support multiple protocol on the same service

[jspdown]

Feature Request

Proposal

Scope:

This proposal is about supporting multiple traffic-type on the same service. It doesn't cover setting different middlwares on ports.

Background

Since the begining of Traefik Mesh it's possible to specify which type traffic a service is exposing. This is achieved using the mesh.traefik.io/traffic-type annotation, which can be either http, tcp or udp. If no value is given, its traffic-type will considered to be the one defined by the --defaultMode flag.

Kubernetes allows a service to expose multiple ports with different protocols (either UDP or TCP). However, this annotation, since we can have just one traffic-type per service, forces the user to create muliple services for exposing multiple ports with different protocols.

Proposal:

Stop relying on annotations to specify the L4 protocol and use the builtin protocol field. Adopt appProtocol field for L7 protocol definition. For now we are only going to support "http" but this could be extended later with grpc

As HTTP is the traffic type most users will want, we need to ease it without requiring the user to change all its services configuration. For that I suggest we add a new option on the controller: --defaultAppProtocol=http. If this option is set and the user wants to use plain TCP he would have to specify a appProtocol: "".

The defaultAppProtocol flag is not expected to be enforced by a mutating webhook. Instead it's just expected to be used when appProtocol is not set.

The value of appProtocol, as defined in the API reference, is expected to be a IANA standard service names. In our case "http". In term of validation, IANA defines service names as lowercase so we must expect a lowercased "http" string.

AppProtocol KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/20191227-app-protocol.md
Status:

• 1.18: alpha, not enabled by default
• 1.19: beta, enabled by default
• 1.20: GA expected, still enabled by default

Example:

apiVersion: v1
kind: Service
metadata:
  name: test
  annotations:
    mesh.traefik.io/ratelimit-average: 12
    mesh.traefik.io/ratelimit-burst: 20
spec:
  ports:
    - name: whoami
      port: 8080
      targetPort: 80
      protocol: TCP
      appProtocol: HTTP

    - name: sql
      port: 2638
      protocol: TCP

    - name: vlc
      port: 1234
      protocol: UDP
      
  selector:
    app: test

Using appProtocol would even allow us to also get rid of the mesh.traefik.io/scheme annotation. But IANA doesn't define the h2c name from what I'm aware of. It would make a lot of sense from my point of view, but supporting that would force us to diverge from the IANA list.

Workarounds

There's currently no workarounds.

[kevinpollet]

The proposal looks really good to me and diverging from the IANA list is not a big issue 👍
My only concern is that Traefik Mesh is supposed to support the latest three minor of Kubernetes.

How will we provide the same functionality for the old versions of Kubernetes?

[jspdown]

For backward compatibility we could use an annotation like suggested in that proposal: #756

Something like mesh.traefik.io/ports.whoami.appProtocol: http

[SantoDE]

Yeah, I agree with @jspdown. For backward compatibility, we should support it by using "old" annotations but immediately deprecate it. Because then, we can get rid of it once the last 3 k8s version support this :)