[stunnel-users] transparent = source, not working

Sat Aug 3 20:24:11 CEST 2013

Hi,

I'm trying to setup yet another service of this kind. I've seen this has 
been largely discussed several times on this list (but without valid 
solutions), and I'm writting this email as some kind of last resort 
after hours of testing and debugging.

Pretty simple configuration:

# cat /etc/stunnel/stunnel.conf
cert = /etc/ssl/certs/stunnel4/my-cert.crt
key = /etc/ssl/certs/stunnel4/my-cert.key

sslVersion = SSLv3
foreground = yes
pid = /tmp/stunnel4.pid

socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1

debug = 7
output = /var/log/stunnel.log

[service]
accept  = 195.78.X.X:6697
connect = 195.78.X.X:1357
transparent = source

Accept and connect IP's are the same.

# /usr/local/bin/stunnel -version
stunnel 4.56 on i686-pc-linux-gnu platform
Compiled/running with OpenSSL 0.9.8o 01 Jun 2010
Threading:PTHREAD Sockets:POLL,IPv6 SSL:ENGINE,OCSP

Global options:
debug                  = daemon.notice
pid                    = /usr/local/var/run/stunnel/stunnel.pid
RNDbytes               = 64
RNDfile                = /dev/urandom
RNDoverwrite           = yes

Service-level options:
ciphers                = ALL:!SSLv2:!aNULL:!EXP:!LOW:-MEDIUM:RC4:+HIGH
curve                  = prime256v1
sessionCacheSize       = 1000
sessionCacheTimeout    = 300 seconds
sslVersion             = TLSv1 for client, all for server
stack                  = 65536 bytes
TIMEOUTbusy            = 300 seconds
TIMEOUTclose           = 60 seconds
TIMEOUTconnect         = 10 seconds
TIMEOUTidle            = 43200 seconds
verify                 = none

# /usr/local/bin/stunnel -sockets
stunnel 4.56 on i686-pc-linux-gnu platform
Compiled/running with OpenSSL 0.9.8o 01 Jun 2010
Threading:PTHREAD Sockets:POLL,IPv6 SSL:ENGINE,OCSP

Socket option defaults:
     Option Name     |  Accept  |   Local  |  Remote  |OS default
     ----------------+----------+----------+----------+----------
     SO_DEBUG        |    --    |    --    |    --    |        no
     SO_DONTROUTE    |    --    |    --    |    --    |        no
     SO_KEEPALIVE    |    --    |    --    |    --    |        no
     SO_LINGER       |    --    |    --    |    --    |       0:0
     SO_OOBINLINE    |    --    |    --    |    --    |        no
     SO_RCVBUF       |    --    |    --    |    --    |     87380
     SO_SNDBUF       |    --    |    --    |    --    |     65536
     SO_RCVLOWAT     |    --    |    --    |    --    |         1
     SO_SNDLOWAT     |    --    |    --    |    --    |         1
     SO_RCVTIMEO     |    --    |    --    |    --    |       0:0
     SO_SNDTIMEO     |    --    |    --    |    --    |       0:0
     SO_REUSEADDR    |       yes|    --    |    --    |        no
     SO_BINDTODEVICE |    --    |    --    |    --    |write-only
     TCP_KEEPCNT     |    --    |    --    |    --    |         9
     TCP_KEEPIDLE    |    --    |    --    |    --    |      7200
     TCP_KEEPINTVL   |    --    |    --    |    --    |        75
     IP_TOS          |    --    |    --    |    --    |         0
     IP_TTL          |    --    |    --    |    --    |        64
     TCP_NODELAY     |    --    |       yes|       yes|        no
     IP_FREEBIND     |    --    |    --    |    --    |        no

And the timeout, the same I've seen suffering to other people:

2013.08.03 19:52:12 LOG7[18496:3074533056]: Service [service] accepted 
(FD=3) from MY_HOME_ADDRESS:34836
2013.08.03 19:52:12 LOG7[18496:3074530160]: Service [service] started
2013.08.03 19:52:12 LOG5[18496:3074530160]: Service [service] accepted 
connection from MY_HOME_ADDRESS:34836
(blah blah ssl stuff)
2013.08.03 19:52:12 LOG6[18496:3074530160]: Negotiated TLSv1/SSLv3 
ciphersuite: DHE-RSA-AES256-SHA (256-bit encryption)
2013.08.03 19:52:12 LOG6[18496:3074530160]: Compression: null, 
expansion: null
2013.08.03 19:52:12 LOG6[18496:3074530160]: IP_TRANSPARENT socket option set
2013.08.03 19:52:12 LOG6[18496:3074530160]: local_bind succeeded on the 
original port
2013.08.03 19:52:12 LOG6[18496:3074530160]: connect_blocking: connecting 
195.78.X.X:1357
2013.08.03 19:52:12 LOG7[18496:3074530160]: connect_blocking: 
s_poll_wait 195.78.X.X:1357: waiting 60 seconds
2013.08.03 19:52:21 LOG3[18496:3074530160]: connect_blocking: connect 
195.78.X.X:1357: Connection timed out (110)
2013.08.03 19:52:21 LOG5[18496:3074530160]: Connection reset: 0 byte(s) 
sent to SSL, 0 byte(s) sent to socket

iptables stuff and that, is in place:

# cat /proc/sys/net/ipv4/conf/all/rp_filter
0
# cat /proc/sys/net/ipv4/ip_forward
1

And did the iptables part:

iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100

Debugging the whole thing, it can be seen that stunnel tries to connect:

[pid 16823] connect(9, {sa_family=AF_INET, sin_port=htons(1357), 
sin_addr=inet_addr("195.78.X.X")}, 16) = -1 EINPROGRESS (Operation now 
in progress)

BUT the service running in 1357 does this:

# tcpdump -i eth1 -n port 1357
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
19:52:52.586773 IP 195.78.X.X.1357 > MY_HOME_ADDRESS.34853: Flags [S.], 
seq 2655966098, ack 546202865, win 5840, options [mss 
1460,nop,nop,sackOK], length 0

And, according to that, it looks obvious to me that this setup will 
never work, since that ACK packet goes to my home box, and not to the 
local connection.

So either I'm doing something wrong (I hope I am!!) or this thing 
definitely doesn't work...

Any help/ideas/whatever, please?

Thanks,

Rubén.