This post will cover a demo working setup of a service mesh architecture using Envoy using a demo application. In this service mesh architecture, we will be using Envoy proxy for both control and data plane. The setup is deployed in a Kubernetes cluster using Amazon EKS.
Pre-requisites
We will be deploying an echo-grpc test application provided by Google in their article related to gRPC load balancing and was used as a reference to test the service mesh setup with Envoy. The article covers setting up Envoy as an edge proxy only. This is a simple gRPC application that exposes a unary method that takes a string in the content request field and responds with the content unaltered. Repo: grpc-gke-nlb-tutorial
- Clone this repo.
- Go to the echo-grpc directory.
- Using the Dockerfile provided in the folder, we would have to build the image and push it to the Docker registry of choice. Since we are not using GCP, Docker Hub is used as the registry.
- Run docker login and login with your hub credentials.
- Build the image docker build -t echo-grpc .
- Tag the image docker tag echo-grpc /echo-grpc
- Push the image docker push /echo-grpc
- Create a separate folder to put all the YAML files.
- Create namespace in k8s:
- Install grpcurl tool which is similar to curl but for gRPC for testing:
Sidecar Deployment
Configuration of envoy for the sidecar deployment:
envoy-echo.yaml:
A couple things to note here.
- We are exposing sidecar on 8786 port on the container.
- Filter envoy.http_connection_manager handles the HTTP traffic.
- route_config is used to define the routes for each domain to their respective clusters. Here we are keeping the domain as , allowing all domains to pass-through.
- A cluster is envoy defines the services that will be called based on the route.
- In the cluster, the lb_policy defines the algorithm for load balancing, keeping as ROUND_ROBIN, with type STATIC because it is a sidecar and needs to communicate to only one pod always which leads to the reason for keeping the address in socket_address as localhost while port_value is what will be exposed by that particular service’s deployment.
Run:
Deployment of echo-grpc application with 3 replicas. The config contains two containers, one for application and another being the Envoy image.
Here, echo-grpc is test application and envoy is being deployed in the same pod. Config volumes are mounted so that the envoy can read the configmaps.
Run:
Headless Service Configuration
We are using headless service for echo-grpc. Using service as headless will expose the Pods IP to the DNS server of kubernetes which will be used by Envoy to do service discovery for the pods.
echo-service.yaml
In the above config file, we are exposing two ports, one for envoy sidecar (this is the same port we mentioned in the config map of sidecar envoy) and one for the service itself.
Run:
Front Envoy Configuration
Creating a service of type LoadBalancer so that client can access the backend service.
envoy-service.yaml:
Creating self-signed certificates
Run:
Since we are deploying front envoy LoadBalancer on port 443, we have to create a self-signed certificate to make it terminate SSL/TLS connection.
-
Get the external IP:
-
Copy the LoadBalancer address in the EXTERNAL-IP section and do a nslookup and copy the IP address:
-
Create a self-signed cert and key:
-
Create a Kubernetes TLS Secret called envoy-certs that contains the self-signed SSL/TLS certificate and key:
Edge Envoy configuration
Configuration for the edge Envoy:
envoy-configmap.yaml
Since we will be offloading HTTPS, we are using port_value of 443. Most of the configurations are same as of sidecar envoy except for three things:
- A tls_context config is required to mention the tls certifications needed for authentication purposes.
- In clusters, the type has been to STATIC to STRICT_DNS which is a kind of service discovery mechanism making use of Headless service we deployed earlier.
- The socket_address’s address value has been changed to the FQDN of the service.
Run:
Deployment Configuration
envoy-deployment.yaml
Run:
Testing
Proto file for the echo-grpc service:
ccho.proto:
Run the following command to call the server:
The output will be similar to something like this:
Run the above command multiple times and check the value of the hostname field every time which will contain the pod name of one of the 3 pods deployed.
References
