1 - Circuit Breaking

Circuit Breaking is a pattern for creating resilient microservices applications. In microservices architecture, services are deployed across multiple nodes or clusters and have different response times or failure rate. Downstream clients need to be protected from excessive slowness of upstream services. Upstream services, in turn, must be protected from being overloaded by a backlog of requests.

A circuit breaker can have three states:

  • Closed: requests succeed or fail until the number of failures reach a predetermined threshold, with no interference from the breaker. When the threshold is reached, the circuit breaker opens.
  • Open: the circuit breaker trips the requests, which means that it returns an error without attempting to execute the call
  • Half open: the failing service is given time to recover from its broken behavior. If requests continue to fail in this state, then the circuit breaker is opened again and keeps tripping requests. Otherwise, if the requests succeed in the half open state, then the circuit breaker will close and the service will be allowed to handle requests again.

Circuit breaking in Istio Circuit breaking in Istio

Service Mesh Manager is using Istio’s - and therefore Envoy’s - circuit breaking feature under the hood.

Circuit breaking using the UI

Set circuit breaking

To configure a circuit breaker for a service, you don’t need to create or edit a Destination Rule resource manually, just complete the following steps.

  1. Select the service on the TOPOLOGY or the SERVICES page

  2. Select CIRCUIT BREAKER, then click CONFIGURE. If you have already configured a circuit breaker for the service and want to modify it, click Edit .

  3. Configure the parameters of the circuit breaker.

    Circuit Breaking set Circuit Breaking set

Monitor circuit breaking

With this configuration you’ve just set, when traffic begins to flow from two connections simultaneously, the circuit breaker starts to trip requests. In the Service Mesh Manager UI, you can see this the graph’s red edges on the MENU > TOPOLOGY page. Click on the service to see two live Grafana dashboards which specifically show the circuit breaker trips and help you learn more about the errors involved.

The first dashboard details the percentage of total requests that were tripped by the circuit breaker. When there are no circuit breaker errors, and your service works as expected, this graph shows 0%. Otherwise, it shows the percentage of the requests that were tripped by the circuit breaker.

The second dashboard provides a breakdown of the trips caused by the circuit breaker by source. If no circuit breaker trips occurred, there are no spikes in this graph. Otherwise, it shows which service caused the circuit breaker to trip, when, and how many times. Malicious clients can be tracked by checking this graph.

Circuit Breaking trip Circuit Breaking trip

These are live Grafana dashboards customized in order to display circuit breaker-related information.

Remove circuit breaking

To remove circuit breaking, navigate to the service, then select CIRCUIT BREAKER > Delete .

2 - External Services

An Istio service mesh has a few different ways of reaching services that are external to the mesh. External services are everything that are not defined in Istio’s internal service registry, that is, services which are outside of the mesh. By default Istio permits requests to unknown or external services. While using permissive configuration for testing purposes is ok, in a production environment a stricter configuration might be necessary.

Control access to external services Control access to external services

Note: Service Mesh Manager is using Istio’s - and therefore Envoy’s - egress control feature under the hood.

Change the default policy

You can change the default policy for outbound traffic by running the smm istio outbound-traffic-policy <setting> command.

  • To restrict outbound traffic to known endpoints, run the following command.

    smm istio outbound-traffic-policy restricted
    

    Expected output:

    mesh wide outbound traffic policy is set to 'REGISTRY_ONLY'
    

    To permit access to an external service, see Allow access only to registered services.

  • To permit all outbound traffic without restrictions, run the following command. (This is the default setting.)

    smm istio outbound-traffic-policy allowed
    

    Expected output:

    mesh wide outbound traffic policy is set to 'ALLOW_ANY'
    

    Note: Running smm istio outbound-traffic-policy returns the current setting of the traffic policy. If you haven’t changed the outbound traffic policy yet, it returns “mesh wide outbound traffic policy is not found”, which means that the default Istio setting is used, which is ALLOW_ANY (permits outbound traffic without any restrictions).

Allow access only to registered services

To allow access only to registered external services, complete the following steps.

Note: Accessing external HTTPS services comes with a few constrains.

  • All the HTTP-related information like method, URL path, response code, is encrypted so Istio cannot see and cannot monitor that information for HTTPS.
  • Service Mesh Manager’s dashboard shows HTTPS as TCP since detailed HTTP-related information is not available.
  1. Change the default outbound traffic policy to block unknown services.

    smm istio outbound-traffic-policy restricted
    
  2. Create ServiceEntry resources for the services you want to permit access to.

    ServiceEntry resources add additional entries into Istio’s internal service registry, so that auto-discovered services in the mesh can access/route to these manually specified services. A service entry describes the properties of a service (DNS name, VIPs, ports, protocols, endpoints). These services could be external to the mesh (for example, web APIs) or mesh-internal services that are not part of the platform’s service registry. For more information, see the documentation of the ServiceEntry resource.

    For example, the following command creates a ServiceEntry resource that allows HTTP access to the httpbin.org site from the smm-demo namespace.

    kubectl apply -f - <<EOF
    apiVersion: networking.istio.io/v1alpha3
    kind: ServiceEntry
    metadata:
      name: httpbin.org
      namespace: smm-demo
    spec:
      hosts:
      - httpbin.org
      - www.httpbin.org
      ports:
      - number: 80
        name: http
        protocol: HTTP
      resolution: DNS
      location: MESH_EXTERNAL
      EOF
    
  3. (Optional) Test that your pods can access the external service. For example, if you have installed the SMM demo application, you can change the notifications-v1 deployment by running:

    kubectl -n smm-demo set env deployment/notifications-v1 'REQUESTS=http://httpbin.org/get#1'
    

    Expected output:

    deployment.extensions/notifications-v1 env updated
    

    Once the notifications pods are restarted, the Service Mesh Manager Dashboard displays outgoing calls to httpbin.org

Note: To route outgoing traffic through an egress gateway, see Create egress gateway.

Remove access to an external service

To remove access to an external service, delete the ServiceEntry resource of the service, for example:

kubectl delete serviceentry -n smm-demo httpbin.org

Expected output:

serviceentry.networking.istio.io "httpbin.org" deleted

Traffic optimization with SD-WAN

Modern microservices applications rely on an efficient network. Also, these microservices often communicate not only among themselves but also with external services, which in many cases are offered by a third party. Optimizing the network between the local application components and the remote services they might be consuming is critical.

Fortunately, in the Istio service mesh these external dependencies are well defined, making it possible for modern Software-Defined Wide Area Network (SD-WAN) solutions to automatically consume information about those external application dependencies and optimize the connectivity between the service mesh and the external services.

Integrate Istio with SD-WAN Integrate Istio with SD-WAN

Why use SD-WAN with Istio

Istio allows you to improve the security of your infrastructure by managing access to external services. In addition to that, integrating Istio with an SD-WAN solution provides the following benefits:

  • Traffic optimization: automatically select the best path for the traffic
  • Minimized external service latency: Latency can be optimized per service and per location
  • Increased external service availability: SD-WAN provides transparent path failover in case of an error
  • No extra Istio configuration is needed: After an initial setup everything is automatic, based on the Istio ServiceEntry custom resources with MESH_EXTERNAL values

Integrate Istio with SD-WAN

To integrate your Service Mesh Manager deployment with Cisco SD-WAN, follow the documentation of the open source Egress-Watcher. The integration consists of the following high level steps:

  1. Configure Cisco vManage (a component of Cisco SD-WAN).
  2. Download and install Egress-Watcher on your primary Service Mesh Manager cluster.
  3. Configure Egress-Watcher.

3 - Fault Injection

Fault injection is a system testing method which involves the deliberate introduction of network faults and errors into a system. It can be used to identify design or configuration weaknesses, and to ensure that the system can handle faults and recover from error conditions.

With Service Mesh Manager, you can inject failures at the application layer to test the resiliency of the services. You can configure faults to be injected into requests that match specific conditions to simulate service failures and higher latency between services. There are two types of failures:

  • Delay adds a time delay before forwarding the requests, emulating various failures such as network issues, an overloaded upstream service, and so on.

  • Abort aborts the HTTP request attempts and returns error codes to a downstream service, giving the impression that the upstream service is faulty.

Service Mesh Manager uses Istio’s - and therefore Envoy’s - fault injection feature under the hood.

Fault injection using the UI

To configure fault injection on the UI, complete the following steps.

  1. Select the service on the MENU > SERVICES or the MENU > TOPOLOGY page.

    Traffic management tab Traffic management tab

  2. Select TRAFFIC MANAGEMENT > CREATE NEW.

  3. Set the destination PORT NUMBER of the service where you want to add the rule.

  4. By default, the new rule matches every incoming request. Click ADD CUSTOM MATCH to select only specific traffic for the rule based on scheme, method, URI, host, port, or authority.

    Inject faults into the traffic Inject faults into the traffic

  5. Scroll down, select FAULT INJECTION, and set the percentage of traffic you want to add faults to, and the amount of delay or the HTTP error code to return.

    Fault injection parameters Fault injection parameters

  6. Click Apply, then check the status of the service on the MENU > TOPOLOGY page. You should see the effects of the injected faults (for example, a delay in the response times).

    Faults in the service Faults in the service

4 - Ingress gateways

Ingress gateways define an entry point into your Istio mesh for incoming traffic.

Multiple ingress gateways in Istio

You can configure gateways using the Gateway and VirtualService custom resources of Istio, and the IstioMeshGateway CR of Service Mesh Manager.

  • The Gateway resource describes the port configuration of the gateway deployment that operates at the edge of the mesh and receives incoming or outgoing HTTP/TCP connections. The specification describes a set of ports that should be exposed, the type of protocol to use, TLS configuration – if any – of the exposed ports, and so on. For more information about the gateway resource, see the Istio documentation.
  • The VirtualService resource defines a set of traffic routing rules to apply when a host is addressed. Each routing rule defines matching criteria for the traffic of a specific protocol. If the traffic matches a routing rule, then it is sent to a named destination service defined in the registry. For example, it can route requests to different versions of a service or to a completely different service than was requested. Requests can be routed based on the request source and destination, HTTP paths and header fields, and weights associated with individual service versions. For more information about VirtualServices, see the Istio documentation.
  • Service Mesh Manager provides a custom resource called IstioMeshGateway and uses a separate controller to reconcile gateways, allowing you to use multiple gateways in multiple namespaces. That way you can also control who can manage gateways, without having permissions to modify other parts of the Istio mesh configuration.

Using IstioMeshGateway, you can add Istio ingress or egress gateways in the mesh and configure them. When you create a new IstioMeshGateway CR, Service Mesh Manager takes care of configuring and reconciling the necessary resources, including the Envoy deployment and its related Kubernetes service.

Note: Service Mesh Manager automatically creates an ingress gateway called smm-ingressgateway and an istio-meshexpansion-cp-v115x. The smm-ingressgateway serves as the main entry point for the services of Service Mesh Manager, for example, the dashboard and the API, while the meshexpansion gateway is used in multi-cluster setups to ensure communication between clusters for the Istio control plane and the user services.

Do not use this gateway for user workloads, because it is managed by Service Mesh Manager, and any change to its port configuration will be overwritten. Instead, create a new mesh gateway using the IstioMeshGateway custom resource.

Service Mesh Manager provides gateway management and observability, allowing you to:

For details on how to set up gateways using Service Mesh Manager, Gateways.

5 - Mirroring

Traffic mirroring or shadowing is a feature that can be used to test new versions of a service with real traffic before rolling it out to the users with minimal risk or to monitor and audit traffic of existing services. Mirroring sends a copy of live traffic to a mirrored service.

Mirroring using the UI

To configure mirroring from the dashboard, complete the following steps.

  1. Select the service on the MENU > SERVICES or the MENU > TOPOLOGY page.

  2. Select TRAFFIC MANAGEMENT > CREATE NEW.

    Traffic management

  3. Scroll down and select MIRRORING.

  4. Set the destination of the mirrored traffic. You must set at least the destination host.

    Configure Traffic Management Configure Traffic Management

  5. Click Apply. The mirroring rule is listed in the ACTIONS field.

    Mirroring rule Mirroring rule

6 - Restrict Outbound Traffic of Workloads

By default, each Envoy proxy receives information about every workload in the mesh. This can result in high memory usage in the Envoy proxies. Service Mesh Manager can help you limit the allowed outbound connections of a workload or a whole namespace to reduce the memory requirements, especially in larger meshes.

You can set a restriction manually, or you can rely on Service Mesh Manager to give you a recommended configuration based on the current network traffic. For details on how this works, see our blog post about the sidecar resource.

Service Mesh Manager is using Istio’s - and therefore Envoy’s - sidecar feature under the hood.

Restrict outbound traffic from the command line

The following sections describe how to manage outbound traffic restrictions using the smm command-line tool. If you want to use the Service Mesh Manager web interface instead, see Restrict outbound traffic using the UI.

Get an outbound traffic restriction recommendation

  1. To get a recommendation for a specific workload based on the current traffic, run the following command. (Replace the smm-demo namespace and the name of the workload as needed for your environment.)

    smm sidecar-proxy egress recommend smm-demo --workload payments-v1
    

    Sample output:

    Recommended sidecar egress rules for smm-demo/payments-v1
    
    Sidecar               Selector        Hosts                                                        Bind  Port  Capture Mode
    smm-demo-rmoy8  app="payments"  ./notifications.smm-demo.svc.cluster.local                   -
                        version="v1"    istio-system/istio-telemetry.istio-system.svc.cluster.local
    

    In this case, the recommended configuration only allows connections from the smm-demo/payments-v1 workload to the istio-telemetry service in the istio-system namespace and to the notifications service in the current namespace (from the perspective of the workload).

  2. To get a recommendation for the whole namespace (smm-demo in this case), run the following command.

    smm sidecar-proxy egress recommend smm-demo
    

    Sample output:

    Recommended sidecar egress rules for namespace smm-demo
    
    Sidecar               Selector  Hosts           Bind  Port  Capture Mode
    smm-demo-zy8fq            istio-system/*        -
                                    ./*
    

    In this case, the recommendation restricts connections to the current and istio-system namespaces.

  3. To apply the recommendations, run the same command again with the --apply switch, for example:

    smm sidecar-proxy egress recommend smm-demo --workload payments-v1 --apply
    

    or

    smm sidecar-proxy egress recommend smm-demo --apply
    

Restrict outbound traffic using the UI

To restrict outbound traffic using the Service Mesh Manager web interface, complete the following steps. If you want to use the Service Mesh Manager command line tool instead, see Restrict outbound traffic from the command line.

  1. Navigate to MENU > TOPOLOGY, or to MENU > WORKLOADS.

    • To restrict outbound traffic for a workload, select a workload.
    • To set outbound traffic restrictions for a namespace, click the name of the namespace (shown in capitals, for example, SMM-DEMO.

    Restrict outbound traffic for a namespace Restrict outbound traffic for a namespace

  2. Click PROXY CONFIG > Override rule .

  3. To get rule recommendations based on live traffic, click Automatic recommendation.

    Proxy config &gt; Automatic recommendation Proxy config &gt; Automatic recommendation

  4. To add new rule manually, click Add, then select the destination namespace and service where you want to permit traffic.

  5. To activate your changes, click Apply. The restrictions you configured are shown on the PROXY CONFIG page.

    Egress proxy rules Egress proxy rules

7 - Routing

Note: This section describes the routing rules of in-mesh services. To configure routing rules for ingress gateways, see Routes and traffic management.

One of the top features of Service Mesh Manager is the ability to fully configure how traffic flows in the service mesh. This kind of routing works in the application layer, and lets you configure sophisticated rules based on URIs, ports, or headers.

In Istio, routing is mostly described in Virtual Services, and then translated to Envoy configuration. Service Mesh Manager covers almost everything that you can describe with Virtual Services, but comes with an easy to understand structure of request matches, routes, and different actions.

You can add routing or redirect rules for requests that match certain criteria, and configure different rules like retry policies, request timeouts, or fault injection.

Rule precedence

Note the following points about how Service Mesh Manager evaluates the routing rules:

  • Rules are evaluated in top-down order.
  • Rules that match on any traffic are always the last to help avoid rule shadowing.
  • Changing the order of rules is not supported in Service Mesh Manager.

When you specify multiple MATCH arguments, they have a logical OR relationship: the rule matches any request that fits one of the match rules. Within a match rule, you can specify multiple rules that have an AND relation. That way you can match requests against a specific URL and an HTTP method, for example.

Set routing using the UI

To create a new routing rule from the dashboard, complete the following steps. You can also edit or delete rules, and you can also view the full YAML configuration of the virtual service.

Note: Rules are evaluated in top-down order. For more details, see Rule precedence.

  1. Select the service on the MENU > SERVICES or the MENU > TOPOLOGY page.

  2. Select TRAFFIC MANAGEMENT > CREATE NEW.

    Configure routing routes Configure routing routes

  3. By default, the new rule matches every incoming request. Click ADD CUSTOM MATCH to select only specific traffic for the rule based on scheme, method, URI, host, port, or authority.

    When you specify multiple MATCH arguments, they have a logical OR relationship: the rule matches any request that fits one of the match rules. Within a match rule, you can specify multiple rules that have an AND relation. That way you can match requests against a specific URL and an HTTP method, for example.

    For example, the following rule matches GET requests, and PUT requests received on port 8080.

    Match requests Match requests

  4. Set the action you want to execute on the matching requests.

    • You can Route the requests to a specific service. To route a portion of the traffic to a different destination, select ADD DESTINATION and use the WEIGHT parameter to split the traffic between multiple destination services.

      Multiple destinations Multiple destinations

      Note: If you want to mirror the traffic (that is, send the same requests to multiple destinations), see Mirroring.

    • Alternatively, you can Redirect the traffic to a specific URI. Redirect rules overwrite the Path portion of the URL with this value. Note that the entire path is replaced, irrespective of the request URI being matched as an exact path or prefix. To overwrite the Authority/Host portion of the URL, set the AUTHORITY FIELD as well.

      Redirect traffic Redirect traffic

  5. Set TIMEOUT and RETRY options as needed for your environment.

    Set timeout and retry parameters Set timeout and retry parameters

  6. Click Apply. The new rule appears on the TRAFFIC MANAGEMENT tab.

    Edit routing rule Edit routing rule

    You can later edit or delete the routing rule by clicking the Edit or Delete icons, respectively.

8 - Sidecar injection

Service Mesh Manager allows you to configure automatic sidecar injection on the namespace level.

For details on configuring the sidecar proxies, see Restrict Outbound Traffic of Workloads.

Set sidecar injection using the UI

To set automatic sidecar injection using the web interface, complete the following steps.

  1. Navigate to MENU > TOPOLOGY.

  2. Click the name of the namespace you want to modify, for example, SMM-DEMO. A sidebar opens.

    Enable automatic sidecar injection Enable automatic sidecar injection

  3. Click DISABLE or ENABLE to disable or enable automatic sidecar injection.

    Note: Automatic sidecar injection happens when the pod is created. Changing this setting does not affect existing pods. To update existing pods, delete them manually, or update all deployments of the namespace by running kubectl rollout restart deployment -n <name-of-namespace>

Set sidecar injection from the command line

To enable automatic sidecar injection on a namespace, run the following command:

smm sidecar-proxy auto-inject on <name-of-namespace>

Expected output:

INFO[0006] auto sidecar injection successfully set to namespace

Note: Automatic sidecar injection happens when the pod is created. Changing this setting does not affect existing pods. To update existing pods, delete them manually, or update all deployments of the namespace by running kubectl rollout restart deployment -n <name-of-namespace>

CAUTION:

Adding the istio-injection label to the namespace does not trigger sidecar injection, because Service Mesh Manager uses versioned control planes. We recommend using the smm sidecar-proxy auto-inject command. Alternatively, you can set the istio.io/rev=cp-v115x.istio-system label manually by running:

kubectl label ns <namespace-to-label> istio.io/rev=cp-v115x.istio-system

To disable automatic sidecar injection on a namespace, run the following command:

smm sidecar-proxy auto-inject off <name-of-namespace>

Expected output:

INFO[0006] auto sidecar injection removed from to namespace