CHAOS ENGINEERING

with

SERVICE MESH

Julien Bisconti

SRE / Data Engineer

contact

g.dev/julien

slides: bisconti.cloud

Outline

Genesis
Service mesh: architecture and features
Demo of Envoy and Istio
Chaos Engineering: concepts & origin
Demo of fault-injection
Q&A

Questions to the audience

Who knows what is a service mesh ?
Who knows what is a SLI, SLO, SLA ?
Who knows what is Chaos Engineering ?
Who already did Chaos Engineering ?

Service Mesh

CONNECT

SECURE

CONTROL

OBSERVE

source: istio.io

VIDEO: Istio a la carte by Dan Ciruli

8 fallacies of distributed computing

The network is reliable.
Latency is zero.
Bandwidth is infinite.
The network is secure.

Topology doesn't change.
There is one administrator.
Transport cost is zero.
The network is homogeneous.

source (wikipedia)
RFC 1925 ( 12 Networking Truths )

using a library, framework or service for:

circuit breaking
timeouts
retries
service discovery
client-side loadbalancing
metrics
traffic shaping
rate limiting
...

for which languages?

drawbacks

combination language/framework/version/feature
maintain, upgrade, migrate, retire
code pollution and complexity (+ testing)
deployment / rolling update
language/framework/version lock-in
debugging

➡️ ️ move it to the infrastructure

What is a service mesh

What problems does it solve

Communication between services

A network for services, not bytes

How does it solve inter service communication

The overall architecture of an Istio-based application.

source

What's in the code


details = {
    "name" : "http://details:9080",
    "endpoint" : "details",
    "children" : []
}
ratings = {
    "name" : "http://ratings:9080",
    "endpoint" : "ratings",
    "children" : []
}

source code

Traffic Management


apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
  ...
spec:
  hosts:
  - reviews
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v1

Resiliency


apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v2
      retries:
        attempts: 3
        perTryTimeout: 2s

Security

namespace-level and service-level policies
mutual TLS Authentication
role-based access control (RBAC)

Observability

Metrics (prometheus)
Logs (fluentd)
Tracing (jaeger)
Cluster traffic (kiali)

DEMO

QUESTIONS about service mesh

List of service meshes

Comparison: Consult vs Istio

Resources

CHAOS ENGINEERING

Chaos Engineering is the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.
— principlesofchaos.org

Thoughtful, planned experiments designed to reveal the weakness in our systems.
— Kolton Andrus (cofounder and CEO of Gremlin Inc.)

Chaos Engineering isn't done to cause problems; it is done to reveal them.
— Nora Jones (Netflix)

Chaos Engineering is exploratory testing of non-functional requirements where ‘non-functional requirements’ are the requirements that if not met render a service non-functional.
— @littleidea