Hey!
I'm having a problem with a stunnel setup I have, and was wondering if anyone could help?
I have two machines, one is acting as a simulator of clients. It uses the MINA library from apache to simulate a bunch of clients of the server. And the other acting as a server. Without stunnel in place, the client and server chat with no trouble... even with as many as 10k simulated clients. The protocol they communicate is just compressed binary data over standard TCP/IP. 20-50 bytes each direction per exchange.
The behavior changes once stunnel is in place. The server behaves just fine, but the client acts very strange... in ways that make me think that stunnel is hitting some OS limit. (The files ulimit is set waaaaaay higher than the number of clients)
On starting the client application, it begins making connections through the stunnel tunnel. The first 1500 or so are fine, but after that the load on the machine begins to climb. It gets to 5 or so around 2000. If I only do 2000 clients, the load then drops down into the 0.5 range, and the system runs okay. However, if I set the number of simulated clients higher... say 4000, the load continues to climb. When it gets to 3500 or so, the load goes above 500. The machine becomes unusable, and after a bit, stunnel dies with the message KILLED.
I tried changing how quickly the clients connect, but that seemed to make no difference. Also, running two stunnels for the clients, and sending 2000 to each, for instance, causes it to still break down around 3500 or so. (Which is why I think it is an os limit)
Any suggestions on this? Thanks!
Peace, Gary
version Stunnel 4.20
startup info 2007.09.24 12:09:50 LOG7[26537:3086939840]: RAND_status claims sufficient entropy for the PRNG 2007.09.24 12:09:50 LOG7[26537:3086939840]: PRNG seeded successfully 2007.09.24 12:09:50 LOG4[26537:3086939840]: Wrong permissions on hcs-key-and-cert-1024.pem 2007.09.24 12:09:50 LOG7[26537:3086939840]: Certificate: hcs-key-and-cert-1024.pem 2007.09.24 12:09:50 LOG7[26537:3086939840]: Certificate loaded 2007.09.24 12:09:50 LOG7[26537:3086939840]: Key file: hcs-key-and-cert-1024.pem 2007.09.24 12:09:50 LOG7[26537:3086939840]: Private key loaded 2007.09.24 12:09:50 LOG7[26537:3086939840]: SSL context initialized for service hcsuno 2007.09.24 12:09:50 LOG5[26537:3086939840]: stunnel 4.20 on i686-pc-linux-gnu with OpenSSL 0.9.8d 28 Sep 2006 2007.09.24 12:09:50 LOG5[26537:3086939840]: Threading:PTHREAD SSL:ENGINE Sockets:POLL,IPv4 Auth:LIBWRAP 2007.09.24 12:09:50 LOG6[26537:3086939840]: file ulimit = 80000 (can be changed with 'ulimit -n') 2007.09.24 12:09:50 LOG6[26537:3086939840]: poll() used - no FD_SETSIZE limit for file descriptors 2007.09.24 12:09:50 LOG5[26537:3086939840]: 39062 clients allowed 2007.09.24 12:09:50 LOG7[26537:3086939840]: FD 3 in non-blocking mode 2007.09.24 12:09:50 LOG7[26537:3086939840]: FD 4 in non-blocking mode 2007.09.24 12:09:50 LOG7[26537:3086939840]: FD 5 in non-blocking mode 2007.09.24 12:09:50 LOG7[26537:3086939840]: SO_REUSEADDR option set on accept socket 2007.09.24 12:09:50 LOG7[26537:3086939840]: hcsuno bound to 0.0.0.0:4090 2007.09.24 12:09:50 LOG7[26537:3086939840]: Created pid file /tmp/stunnel1.pid
uname -a Linux testbed-client1 2.6.9-42.ELsmp #1 SMP Sat Aug 12 09:39:11 CDT 2006 i686 i686 i386 GNU/Linux libc 2.3.4
stunnel -version stunnel 4.20 on i686-pc-linux-gnu with OpenSSL 0.9.8d 28 Sep 2006 Threading:PTHREAD SSL:ENGINE Sockets:POLL,IPv4 Auth:LIBWRAP
Global options debug = 5 pid = /usr/local/var/run/stunnel/stunnel.pid RNDbytes = 64 RNDfile = /dev/urandom RNDoverwrite = yes
Service-level options cert = /usr/local/etc/stunnel/stunnel.pem ciphers = ALL:!ADH:+RC4:@STRENGTH key = /usr/local/etc/stunnel/stunnel.pem session = 300 seconds sslVersion = SSLv3 for client, all for server TIMEOUTbusy = 300 seconds TIMEOUTclose = 60 seconds TIMEOUTconnect = 10 seconds TIMEOUTidle = 43200 seconds verify = none
gcc -v Reading specs from /usr/lib/gcc/i386-redhat-linux/3.4.6/specs Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-java-awt=gtk --host=i386-redhat-linux Thread model: posix gcc version 3.4.6 20060404 (Red Hat 3.4.6-3)
openssl version OpenSSL 0.9.7a Feb 19 2003
In the config, I have:
client=yes debug=debug socket = l:TCP_NODELAY=1 pid = /tmp/stunnel1.pid foreground=yes
[hcsuno] accept=4090 connect = testbed-hcs1:4094 cert = hcs-key-and-cert-1024.pem key = hcs-key-and-cert-1024.pem
********* ***** ** Gary Coulbourne *************************.* Bear Activist ****** *********** ** *******o ******* ********* **** ****`- Systems Administrator ******* ********* ***** http://www.bears.org ****** ********** **** [email protected] ## ***** ***** ## **** KB3INA ### ***** ### **** Animal Conservation/Preservation #,,, ***,,, ##,,, **,,,
Howdy!
I was wondering -- is stunnel asymmetric between client and server use? If I use stunnel to create a tunnel, with one stunnel doing the client side and another doing the server side, is it reasonable that the client side would more heavily load the CPU than the server side given the same hardware? Thanks!
Peace, Gary
********* ***** ** Gary Coulbourne *************************.* Bear Activist ****** *********** ** *******o ******* ********* **** ****`- Systems Administrator ******* ********* ***** http://www.bears.org ****** ********** **** [email protected] ## ***** ***** ## **** KB3INA ### ***** ### **** Animal Conservation/Preservation #,,, ***,,, ##,,, **,,,
On Wednesday 26 September 2007 18:07, [email protected] wrote:
I was wondering -- is stunnel asymmetric between client and server use? If I use stunnel to create a tunnel, with one stunnel doing the client side and another doing the server side, is it reasonable that the client side would more heavily load the CPU than the server side given the same hardware? Thanks!
There is an asymmetry of CPU utilization during the initial key negotiation. Fortunately due to SSL session caching it has no practical impact, as subsequent connections can reuse negotiated keys.
Best regards, Mike
On Monday 24 September 2007 18:24, [email protected] wrote:
I'm having a problem with a stunnel setup I have, and was wondering if anyone could help?
[cut]
stunnel 4.20 on i686-pc-linux-gnu with OpenSSL 0.9.8d 28 Sep 2006 Threading:PTHREAD SSL:ENGINE Sockets:POLL,IPv4 Auth:LIBWRAP
Try to disable libwrap first: ./configure --disable-libwrap make clean make make install It is a bottleneck for this kind of tests.
Best regards, Mike
On Wed, 26 Sep 2007, Michal Trojnara wrote:
Try to disable libwrap first: ./configure --disable-libwrap make clean make make install It is a bottleneck for this kind of tests.
PERFECT! Thank you very much! That seems to have sorted out my troubles. The load average stayed well within what I expected. I've got a hardware firewall protecting the machine, so libwrap wasn't helping me anyhow.
Peace, Gary
********* ***** ** Gary Coulbourne *************************.* Bear Activist ****** *********** ** *******o ******* ********* **** ****`- Systems Administrator ******* ********* ***** http://www.bears.org ****** ********** **** [email protected] ## ***** ***** ## **** KB3INA ### ***** ### **** Animal Conservation/Preservation #,,, ***,,, ##,,, **,,,
-
bear＠bears.org -
Michal Trojnara