Backing data for the summary in ../soak-results.md (procedure: ../../soak/README.md) and for
grpc/grpc-java#13012.
Setup: talos-main (arm64), identical 1-CPU/1-Gi Guaranteed pods, one arm per
4-core worker, sequential runs, k6 ramping-arrival-rate 200→2400 req/s in
2-minute steps (10 s climb + 110 s hold), tiny unary echo, warm JVM client on
its own worker. Each row: the step's 110 s hold window. grpc-java 1.83.1,
non-shaded Netty 4.2.16.Final, Clojure 1.12.5 via clj-grpc 0.1.4.
Known artifacts, disclosed: the first step of every JVM-based run is JIT warmup (the arm rolls immediately before its run). Run 1's arm was liveness-killed at ~00:06:41Z and again near ramp top (probes later relaxed to failureThreshold 60). Run 3's arm was liveness-killed at ~00:56Z. Run 5's 1400-offered window shows a transient inversion (dropped-iterations spike that window) inconsistent with its neighbors.
Capacity ramps (200→2400 req/s, 2-min steps, one arm at a time, same pods): max sustainable goodput per identical 1-CPU pod —
| configuration | plateau | overload behavior |
|---|---|---|
gRPC JVM :direct | ~2,140 req/s | graceful (h2 flow-control backpressure) |
| gRPC JVM VT default | ~2,060 req/s | graceful, tails degrade |
| gRPC native VT | ~1,550 req/s | liveness-starvation kills before probe tuning |
| REST (Jetty, HTTP/1.1) | ~960 req/s | queue-death: p50 pins at 1.7 s, then collapse |
At equal resources gRPC sustains 2.2× (JVM) / 1.6× (native) REST's
throughput. The executor trade inverts with load: VT protects tails ~2×
at ~10% utilization; above ~75% utilization :direct wins goodput and
tails (p99 382 vs 662 ms at 2000 req/s). Peak heaps under saturation:
native 81 MB, gRPC-JVM 61 MB, REST 267 MB.
A grpc-netty patch adding a server-side WriteQueue.drainNow() (upstream
prototype) reproduced its JMH signature on-cluster: p99 −20–23% across
1200–2200 req/s and −64% at 2400 (1994→717 ms), goodput unchanged.
| offered | delivered | p50ms | p99ms | dropped/s |
|---|---|---|---|---|
| 200 | 200.3 | 2.6 | 9.1 | 0 |
| 400 | 400.5 | 2.7 | 13.1 | 0 |
| 600 | 600.0 | 2.9 | 23.7 | 0 |
| 800 | 800.8 | 3.8 | 48.9 | 0 |
| 1000 | 999.5 | 11.0 | 81.3 | 0.3 |
| 1200 | 1197.6 | 30.9 | 187.8 | 2.8 |
| 1400 | 1379.3 | 82.3 | 457.7 | 20.8 |
| 1600 | 1499.7 | 426.0 | 2226.4 | 91.9 |
| 1800 | 1580.6 | 760.3 | 2455.1 | 187.6 |
| 2000 | 1567.6 | 832.9 | 2465.4 | 432.4 |
| 2200 | 1607.7 | 823.3 | 2459.5 | 598.8 |
| 2400 | 1524.8 | 840.1 | 2496.5 | 818.0 |
| offered | delivered | p50ms | p99ms | dropped/s |
|---|---|---|---|---|
| 200 | 207.6 | 3.9 | 4326.1 | 0.0 |
| 400 | 400.2 | 2.8 | 12.8 | 0.0 |
| 600 | 600.2 | 3.0 | 24.9 | 0.0 |
| 800 | 800.4 | 3.3 | 44.5 | 0.0 |
| 1000 | 1000.6 | 3.9 | 59.5 | 0.0 |
| 1200 | 1201.8 | 4.9 | 86.8 | 0.0 |
| 1400 | 1393.0 | 10.7 | 827.9 | 6.9 |
| 1600 | 1600.7 | 16.0 | 238.7 | 0.0 |
| 1800 | 1791.7 | 34.9 | 324.9 | 0.0 |
| 2000 | 1991.9 | 69.2 | 661.8 | 10.8 |
| 2200 | 2059.3 | 126.6 | 972.5 | 132.4 |
| 2400 | 1984.3 | 254.6 | 2069.8 | 339.7 |
| offered | delivered | p50ms | p99ms | dropped/s |
|---|---|---|---|---|
| 200 | 201.9 | 207.1 | 10000.0 | 3.8 |
| 400 | 396.8 | 2692.3 | 7685.4 | 9.5 |
| 600 | 601.4 | 4.6 | 239.9 | 0.0 |
| 800 | 799.9 | 9.1 | 95.5 | 0.0 |
| 1000 | 962.5 | 1634.6 | 2486.2 | 30.0 |
| 1200 | 960.7 | 1697.2 | 2487.9 | 239.1 |
| 1400 | 959.1 | 1700.3 | 2488.8 | 441.0 |
| 1600 | 958.3 | 1696.7 | 2488.3 | 642.1 |
| 1800 | 957.1 | 1689.7 | 2490.3 | 843.1 |
| 2000 | 43.2 | 0.0 | 9696.5 | 1942.6 |
| 2200 | 244.6 | 6572.0 | 10000.0 | 1954.6 |
| 2400 | 348.1 | 5500.6 | 7460.2 | 1995.7 |
| offered | delivered | p50ms | p99ms | dropped/s |
|---|---|---|---|---|
| 200 | 206.6 | 3.1 | 4448.4 | 0.0 |
| 400 | 400.4 | 2.6 | 9.6 | 0.0 |
| 600 | 599.5 | 2.8 | 21.1 | 0.0 |
| 800 | 800.3 | 3.0 | 38.3 | 0.0 |
| 1000 | 1000.2 | 3.4 | 50.4 | 0.0 |
| 1200 | 1200.7 | 4.0 | 86.6 | 0.0 |
| 1400 | 1403.1 | 4.9 | 113.0 | 0.0 |
| 1600 | 1602.3 | 9.4 | 222.5 | 0.0 |
| 1800 | 1797.5 | 17.1 | 289.3 | 3.7 |
| 2000 | 1991.0 | 32.4 | 381.9 | 4.0 |
| 2200 | 2137.4 | 60.3 | 649.7 | 78.0 |
| 2400 | 1978.8 | 94.4 | 1993.9 | 340.7 |
grpc-netty classes replaced with the wedge/server-drain-now-1.83.1 build
(v1.83.1 + server-side WriteQueue.drainNow() in
NettyServerHandler.channelReadComplete); everything else identical to run 4,
same node.
| offered | delivered | p50ms | p99ms | dropped/s |
|---|---|---|---|---|
| 200 | 203.9 | 2.9 | 2125.2 | 0.0 |
| 400 | 400.3 | 2.6 | 9.6 | 0.0 |
| 600 | 600.6 | 2.7 | 18.4 | 0.0 |
| 800 | 799.9 | 2.9 | 32.4 | 0.0 |
| 1000 | 1000.8 | 3.4 | 49.0 | 0.0 |
| 1200 | 1196.3 | 4.0 | 66.6 | 0.0 |
| 1400 | 1397.7 | 5.6 | 260.1 | 1.1 |
| 1600 | 1594.6 | 10.1 | 222.0 | 5.4 |
| 1800 | 1796.0 | 20.3 | 230.7 | 1.0 |
| 2000 | 1986.6 | 43.5 | 391.6 | 6.5 |
| 2200 | 2116.4 | 75.5 | 498.1 | 77.4 |
| 2400 | 2013.4 | 104.5 | 717.1 | 318.1 |
| arm | idle | peak under saturation |
|---|---|---|
| grpc-native | 6 MB | 81 MB |
| grpc-jvm | 11 MB | 61 MB |
| rest | 30 MB | 267 MB |
Can you improve this documentation?Edit on GitHub
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |