Hi Edwin,
On 6/11/25 2:32 PM, Edwin Torok via stunnel-users wrote:
The use case is live migrating VMs, where the connection between 2 hosts is encrypted using stunnel.
Having an untrusted network that fast is quite impressive. I assume you've already considered using a trusted VLAN or MPLS instead of encryption.
Of course migrating more than one VM at a time (and thus using more than 1 HTTPS connection at a time) can work around this limitation up to a point, but would be good to fix the performance bottlenecks that are easily fixable by tweaking OpenSSL settings and buffer sizes.
As long as it doesn’t affect data integrity, a 17% improvement is (barely) noticeable. stunnel uses non-blocking sockets, so data may be moved within buffers while a zero-copy networking stack still performs DMA to/from the NIC.
We offer commercial support for stunnel, including performance tuning: https://www.stunnel.org/support.html
I'll know more once I finished the rest of my patches, but comparing nginx vs stunnel as a server (with curl as a client) shows that nginx can do 20Gbit/s (on a single stream), and a patched stunnel can do ~18Gbit/s, so there is potentially more performance to be gained by improving stunnel (also with a now patched curl client I should be able to achieve more on both).
Are you sure your nginx is configured as a proxy? I can't see it in your script. Otherwise, nginx only transfers half of the data transferred by stunnel, which explains better performance. If nginx is configured as a proxy, a 10% difference may be barely noticeable, but it might indicate some room for further optimization.
Have you tried:
socket = l:TCP_NODELAY=0 socket = r:TCP_NODELAY=0
? Nagle’s algorithm improves latency for small writes but increases the number of packets, which reduces throughput. It improves interactive connections, but it deteriorates bulk data transfer. This is why it's enabled by default and disabling it with TCP_NODELAY=1 is only useful for interactive protocols.
Another idea to consider:
socket = l:SO_RCVBUF=1048576 socket = l:SO_SNDBUF=1048576 socket = r:SO_RCVBUF=1048576 socket = r:SO_SNDBUF=1048576
Best regards, Mike