Hello,

Flamegraph profiling on stunnel has shown that most time is NOT spent
in encryption/decryption,
but in sending/receiving data.

Experimental setup:
```
CPU: 2x 18-core Intel Xeon Gold 6354
NIC: Intel E810-XXV, 100 Gbit/s
Kernel: 6.14.8-300.fc42.x86_64 x86_64
Mem: 251.32 GiB
OS: Fedora Linux 42
openssl version: OpenSSL 3.2.4 11 Feb 2025 (Library: OpenSSL 3.2.4 11 Feb 2025)
stunnel: 5.75 from git
```

Network, encryption and memory copy speeds converted into Gbit/s on
the above HW:
```
iperf3: 40Gbit/s
openssl speed -evp aes-256-gcm: 72Gbit/s
perf bench memcpy: 100Gbit/s
```

Under ideal conditions we should be able to achieve at most ~20.5Gbit/s,
however I only measured ~7.6Gbit/s.

The scripts that I used to perform measurements are included in the
attached patch 3.

See also related discussion on curl: https://github.com/curl/curl/pull/17548

Enabling readahead, avoiding excessive alloc/free of buffers,
increasing buffer sizes,
and improving buffer handling can all improve performance.

For now I'm sending just the first small patches as proofs of concept,
each of them one-liners.
I hope you can include these in some form in the next release.

Eventually these should probably be exposed as configuration flags in
stunnel.conf,
I can help implementing that, or I can leave implementing that to you
if you prefer.

In fact readahead was enabled in previous versions of stunnel, and
then disabled again,
so I assume you ran into some bugs with it on certain protocols?

Increasing the buffer size is not a clear win (as with curl), due to
the excessive use of memmove/memset,
and that SSL_read/SSL_write only still processes 1 TLS record at a time.
I have some further changes that can improve that, but the patches are
larger and I haven't finished testing them for correctness yet.
Please let me know if you'd want these as patches, or if you'd rather
implement them yourself.

Edwin Török (3):
  openssl: enable readahead
  openssl: disable SSL_MODE_RELEASE_BUFFERS
  Benchmark scripts

 src/ctx.c                    |   5 +-
 tests/benchmark/benchmark.sh | 120 +++++++++++++++++++++++
 tests/benchmark/launch.sh    | 185 +++++++++++++++++++++++++++++++++++
 3 files changed, 309 insertions(+), 1 deletion(-)
 create mode 100644 tests/benchmark/benchmark.sh
 create mode 100755 tests/benchmark/launch.sh

--
2.43.5
_______________________________________________
stunnel-users mailing list -- stunnel-users@stunnel.org
To unsubscribe send an email to stunnel-users-leave@stunnel.org