13. Envoy Acceleration on OCTEON
13.1. Overview
Envoy is a high-performance, open-source edge and service proxy tailored for cloud-native environments, especially microservices architectures. It serves as a universal data plane for service meshes like Istio, and can also function as an internal load balancer or ingress/egress proxy. Envoy simplifies traffic management, load balancing, and observability, making network operations more transparent and troubleshooting more efficient.
13.2. Architecture
At its core, Envoy Proxy operates at layers 3 and 4 (L3/L4), using a configurable set of filters to manage TCP/UDP traffic. It also supports layer 7 (L7) HTTP filters, which are essential for cloud-native applications, including features like TLS termination. Envoy offers advanced load balancing capabilities such as circuit breaking, automatic retries, and gRPC routing. Its configuration is dynamically manageable via APIs, allowing updates to be pushed in real time without restarting the cluster.
Envoy uses a multi-threaded architecture within a single process. A primary thread oversees coordination tasks, while worker threads handle connection processing, filtering, and forwarding. When a listener accepts a new connection, a worker thread is assigned to manage it until the connection is closed.
13.3. Accelerating Envoy on Marvell OCTEON Platform
13.4. Accelerating TLS handshake
Envoy uses BoringSSL as its default TLS library, where TLS handshakes are traditionally handled synchronously blocking the worker thread until the handshake completes. While this approach is straightforward, it becomes a performance bottleneck when integrating with hardware accelerators like OCTEON CPT, which are designed to offload cryptographic operations to dedicated hardware. The key advantage of such accelerators lies in their ability to perform encryption and decryption in parallel, allowing Envoy to continue processing other workloads. However, synchronous handshakes prevent Envoy from leveraging this parallelism, limiting scalability and efficiency.
13.5. Private Key Provider Extension
To address this, Envoy introduces the Private Key Provider extension. This mechanism utilizes BoringSSL’s private key method hooks, enabling external handlers to perform cryptographic operations such as ECDSA signing and RSA signing/decryption. Crucially, it supports asynchronous processing, allowing the handshake function to return immediately. A polling mechanism monitors the status of the cryptographic task, and once complete, a callback is triggered to resume and finalize the handshake. This design decouples cryptographic processing from thread execution, enabling non-blocking behavior and unlocking the benefits of hardware acceleration.
13.6. DPDK Crypto API Integration
A significant enhancement to this model is the integration of the Private Key Provider with DPDK crypto APIs. This integration transforms TLS handshake handling by offloading cryptographic workloads to hardware accelerators through DPDK. As a result, Envoy’s worker threads remain unblocked during handshakes and can continue servicing other requests. This leads to:
Higher TLS connection throughput
Improved system responsiveness
Reduced CPU utilization and power consumption
The DPDK crypto APIs are generic and portable, making them suitable for deployment across a wide range of DPDK-supported platforms without requiring platform-specific adaptations.
13.7. Benefits
Together, asynchronous handshake processing and DPDK-based hardware acceleration significantly enhance Envoy’s performance, scalability, and energy efficiency—making it a robust solution for modern, high-performance, and cloud-native networking environments.
13.8. Deployment
Envoy can be deployed within a Kubernetes environment on a dedicated node that runs on a Data Processing Unit (DPU). This setup enables enhanced performance by offloading cryptographic operations to specialized crypto hardware (CPT) available on the OCTEON platform, which supports DPDK-based crypto acceleration.
Envoy acceleration architecture on OCTEON platform