Benchmark Overview & Methodology
In this video, we'll try to find the fastest Go web framework. We'll compare the Go standard library with fasthttp, Fiber, and gNet frameworks.
In the first test, we'll compare just the frameworks themselves. We'll start with latency, measuring it from the client side. I think it's one of the most important metrics for any framework. Then, we'll measure the throughput of each framework by counting how many requests per second each can handle.
We'll also measure CPU usage, memory usage, availability or error rate, as well as CPU throttling since we'll run these applications in Kubernetes.
In the second test, I'll add a PostgreSQL database and instrument each application with Prometheus metrics to measure latency for insert operations. We'll also measure the connection pool size, as well as the database CPU usage itself.
I think I found a major limitation in one of the frameworks I tested.
Test Environment on AWS EKS
I use AWS to run my tests. I create an EKS cluster with two instance groups. I use 4xlarge Graviton instances to deploy my monitoring components, and I use large instances for each application I deploy, ensuring each app runs on its own VM.
For the second test, I deploy PostgreSQL on a 2xlarge instance with 8 CPUs and 64GB of memory, and I use pgTune to optimize my database performance.
A Note on Channel Support
To support my channel and pay for the infrastructure, I'd like to offer one-on-one sessions. You can find more information in the video description.
Alright, let's go ahead and run the test. So, as in many other tests where I compared the Go standard library with other frameworks, in this one as well, the standard library has the lowest latency from the start. But it also has the highest CPU usage.
At around 15,000 requests per second and 32 percent CPU usage, the standard library loses its advantage and becomes the slowest framework. So, if you use it, pay attention to this CPU usage threshold and scale it horizontally.
On the other hand, we have Fiber and fasthttp, which are very close to each other. This is because Fiber is a framework built on top of fasthttp. Fasthttp doesn't really have a user-friendly API, so Fiber is a tool that wraps the fasthttp framework and adds not only a nice API but other middleware as well.
And gNet is supposed to be the fastest framework based on the benchmarks I've seen, but it has a very low-level API with almost non-existent documentation. However, it has an incredibly small memory footprint and the lowest CPU usage with the potential to achieve the highest throughput in the end.
Test 1 Results: Under Heavy Load
At around 55,000 requests per second, you can start to notice that the standard library is falling behind all other frameworks; this is very consistent with all other benchmarks I've run in AWS using Go. Also, you can notice that gNet has one of the lowest latencies so far under heavy load.
At around 90,000 requests per second, health checks on the Fiber and standard library applications started to fail, and in Kubernetes, if your pod fails too quickly, it transitions to a crashloop state. In production, to recover, you would need to double the capacity and then adjust based on the load.
Test 1 Results: fasthttp vs. gNet
So now we have fasthttp and gNet fighting for the title of the fastest framework.
It looks like fasthttp is much more stable at the end and able to handle more than 100,000 requests per second, which is a huge difference compared to the standard library, while gNet is slightly behind with around 98,000 requests per second.
Test 1 Graph Analysis
Alright, let me open each graph for the entire test duration.
First, we have requests per second.
Next, we have GET latency, which is measured using the 99th percentile.
Next, we have CPU usage.
Availability graph.
Memory usage.
And finally, the CPU throttling graph.
So, as expected, the standard library in this test would be the slowest, and fasthttp actually performed better than any other framework.
Test 2 with PostgreSQL: gNet's Major Limitation
In the second test, when the client sends a POST request with a JSON payload, each application needs to parse it, convert it to a Go struct, and then use the PostgreSQL driver to save it in the database.
So here we're getting latency from the application when it inserts a new record into the database. The source code is available in my public GitHub repository, and you can try to improve it if you can.
For this test, I also used the PostgreSQL exporter to track the pool size created by each application and found a major issue with gNet. I did enable multi-core support since we're running on a VM with two CPUs, and gNet created two event loops.
On my Mac, for example, with 10 CPU cores, it creates 10 event loops. It's best to run as many threads or processes as you have CPU cores, similar to other frameworks like Ruby on Rails. However, the connection pool is limited to the number of threads. So, I have the same 100 connection limit per application, but gNet only creates as many connections as it has event loops. Maybe there's a way to adjust this, but they don't really have any documentation.
So, the number of connections to the database will be a limiting factor for gNet. This doesn't only affect the database driver; it could also affect the AWS SDK and how many connections it can create.
Test 2 Graph Analysis
For this test, I used a 2xlarge instance for PostgreSQL, which actually became a bottleneck. I might use larger instances for future tests it just becomes very expensive.
Alright, let me open each graph for the entire duration.
First, we have the requests per second graph. gNet was way behind all other applications due to its small database pool size.
Next, we have POST latency for the entire request.
Then, we have database insert latency. As in the previous test, the standard library performs better than other frameworks.
Next, we have CPU usage.
Availability graph.
PostgreSQL database CPU usage.
Connection pool size.
And Memory usage.
Final Verdict & Key Takeaways
So, it looks like the standard library has the best latency performance in all tests but also has the highest CPU usage. It's up to you to decide what's more important: latency or throughput.
Thank you for watching, and I have other benchmarks on my channel that you may find interesting. Thank you, and see you in the next video.
