Let's Encrypt support for gateways

Service Mesh Manager provides a streamlined way to use Let’s Encrypt with Istio-native Gateways. Compared to relying on Istio’s Ingress, this solution gives you greater flexibility when defining your own Ingress gateway.

Background

Service Mesh Manager solves this issue by allowing Cert Manager to issue TLS certificates directly to an Istio Gateway resource.

Prerequisites

To set up TLS certificates based on Let’s Encrypt for an Istio Gateway, the following requirements must be met:

  • cert-manager must be enabled for the deployment. Verify that the cert-manager namespace exits.
  • The deployment must be able to allocate a LoadBalancer type service, in addition to the existing Gateway
  • The deployment must expose an http endpoint on port 80
  • The DNS must be set up properly to point to the existing MeshGateway’s Service (we recommend using external-dns).

For example, if you have a namespace called my-app, where the traffic is exposed over TCP port 80 using http with the following MeshGateway and Gateway setup:

apiVersion: servicemesh.cisco.com/v1alpha1
kind: IstioMeshGateway
metadata:
    labels:
        app.kubernetes.io/instance: app
        app.kubernetes.io/name: app-ingress
    name: app-ingress
    namespace: my-app
spec:
    istioControlPlane:
        name: cp-v113x
        namespace: istio-system
    deployment:
       metadata:
          labels:
              app.kubernetes.io/instance: app
              app.kubernetes.io/name: app-ingress
              gateway-name: app-ingress
              gateway-type: ingress
       replicas:
          min: 1
          max: 1
          count: 1
    service:
      metadata:
        annotations:
          external-dns.alpha.kubernetes.io/hostname: my-app.example.org
      ports:
      - name: http2
        port: 80
        protocol: TCP
        targetPort: 8080
      type: LoadBalancer
    runAsRoot: true
    type: ingress
---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  labels:
    app.kubernetes.io/instance: app
    app.kubernetes.io/name: app-ingress
  name: app-ingress
  namespace: my-app
spec:
  selector:
    app.kubernetes.io/instance: app
    app.kubernetes.io/name: app-ingress
  servers:
  - hosts:
    - '*'
    port:
      name: http
      number: 80
      protocol: HTTP

Note: The VirtualService and other Istio resources are omitted from this example, but those are also required for the application to work properly.

Add TLS endpoints to the Gateway

Reconfigure the Gateway and the MeshGateway to support SSL.

  1. Add a new port needs to the MeshGateway resource to accept incoming traffic on port 443 (see the .spec.ports.[name=https] part):

    apiVersion: servicemesh.cisco.com/v1alpha1
    kind: IstioMeshGateway
    metadata:
      labels:
         app.kubernetes.io/instance: app
         app.kubernetes.io/name: app-ingress
      name: app-ingress
      namespace: my-app
    spec:
      istioControlPlane:
         name: cp-v113x
         namespace: istio-system
      deployment:
         metadata:
            labels:
               app.kubernetes.io/instance: app
               app.kubernetes.io/name: app-ingress
               gateway-name: app-ingress
               gateway-type: ingress
         replicas:
            min: 1
            max: 1
            count: 1
      service:
         metadata:
            annotations:
               external-dns.alpha.kubernetes.io/hostname: my-app.example.org
         ports:
         - name: http2
           port: 80
           protocol: TCP
           targetPort: 8080
         - name: https
           port: 443
           protocol: TCP
           targetPort: 8443
         type: LoadBalancer
      runAsRoot: true
      type: ingress
    
  2. Configure the Gateway to accept HTTPS traffic. You have to:

    • Instruct Istio to redirect any non-https traffic to the https endpoint (for details, see Known limitations):

          tls:
            httpsRedirect: true
      
    • And set up a new https endpoint:

        - hosts:
          - '*'
          port:
            name: https
            number: 443
            protocol: HTTPS
          tls:
            credentialName: app-ingressgateway-tls
            mode: SIMPLE
      

    The modified Gateway resource should look something like this:

    apiVersion: networking.istio.io/v1beta1
    kind: Gateway
    metadata:
      labels:
        app.kubernetes.io/instance: app
        app.kubernetes.io/name: app-ingress
      name: app-ingress
      namespace: my-app
    spec:
      selector:
        app.kubernetes.io/instance: app
        app.kubernetes.io/name: app-ingress
      servers:
      - hosts:
        - '*'
        port:
          name: http
          number: 80
          protocol: HTTP
        tls:
          httpsRedirect: true
      - hosts:
        - '*'
        port:
          name: https
          number: 443
          protocol: HTTPS
        tls:
          credentialName: app-ingressgateway-tls
          mode: SIMPLE
    

    Note: The secret in the example does not exist yet. This does not cause a configuration issue, but the TLS endpoint is not yet operational.

Configure cert-manager to provide TLS certificates

Configure Cert Manager to issue the TLS credentials into the app-ingressgateway-tls Secret.

  1. Create an Issuer that represents a connection to the Let’s Encrypt service:

    cat > issuer.yaml <<EOF
    ---
    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      labels:
        app.kubernetes.io/instance: app
        app.kubernetes.io/name: app-issuer
      name: app-issuer
      namespace: my-app
    spec:
      acme:
        email: noreply@cisco.com
        preferredChain: ""
        privateKeySecretRef:
          name: app-letsencrypt-issuer
        server: https://acme-v02.api.letsencrypt.org/directory
        solvers:
        - http01:
            ingress:
              class: nginx
    EOF
    kubectl apply -f issuer.yaml
    

    Note: You can also use a cluster-wide issuer called ClusterIssuer if multiple Certificates are required for multiple namespaces.

  2. Check that the Issuer is working properly. Run the kubectl get issuer command. The Issuer should be in ready state:

    NAME          READY   AGE
    app-issuer    True    19h
    
  3. Create the Certificate object that instructs Cert Manager to maintain a certificate for a given domain name, and automatically refresh it if needed. You can use the following example, but note the following points:

    • The dnsNames section must contain all the host names for which TLS certificates must be present in the resulting secret. For each domain name listed here, there must be a properly set up DNS record pointing to the LoadBalancer’s external IP for the MeshGateway.

        dnsNames:
        - my-app.example.org
      
    • For Service Mesh Manager to know which Gateway to attach this certificate to, the following annotation must be put on the Certificate. This is a label matcher, that the Let’s Encrypt Operator uses to find the Gateways needed to be configured for TLS.

      metadata:
        annotations:
          acme.smm.cisco.com/gateway-selector: |
            {
              "app.kubernetes.io/instance": "app",
              "app.kubernetes.io/name": "app-ingress"
            }      
      
    • The secretName field must match the value of the previously configured TLS secret in the Gateway resource.

    The following yaml includes all three requirements:

    cat > certificate.yaml <<EOF
    ---
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      annotations:
        acme.smm.cisco.com/gateway-selector: |
          {
            "app.kubernetes.io/instance": "app",
            "app.kubernetes.io/name": "app-ingress"
          }
      labels:
        app.kubernetes.io/instance: app
        app.kubernetes.io/name: app-certificate
      name: app-certificate
      namespace: my-app
    spec:
      dnsNames:
      - my-app.example.org
      duration: 2160h0m0s
      issuerRef:
        group: cert-manager.io
        kind: Issuer
        name: app-issuer
      privateKey:
        algorithm: RSA
        encoding: PKCS1
        size: 2048
      renewBefore: 360h0m0s
      secretName: app-ingressgateway-tls
      usages:
      - server auth
      - client auth
    EOF
    kubectl apply -f issuer.yaml
    
  4. Verify that the Certificate has been issued, by running the kubectl get certificate command. The secret should be in ready state:

    NAME                          READY   SECRET                   AGE
    app-certificate               True    app-ingressgateway-tls   19h
    

Known limitations

Istio’s http to https redirection allows for little configurability. As a result, the Let’s Encrypt challenge verification fails in the following case:

  • The gateway has named hosts (for example, '*' is not in the hosts list of the Gateway resource) AND
  • Automatic redirection to HTTPS is enabled in the tls settings of the http endpoint

In this case, try to use one of the following workarounds:

  • Use a Gateway that has '*' specified in its hosts list (this might mean that you need external load balancers).
  • Use the Kubernetes Ingress implementation IngressClass of Istio.
  • Disable the automatic https redirection and use application logic to do the redirect.