Hi All,
I've been playing with this for a little while now and before the all in front of me and my head meet I was wondering if someone else could shed some light on this for me please.
First off let me give you a over view of the network setup: My Virtual Machine has two interfaces eth0 has two addresses 192.168.82.9/18and 192.168.82.10/18. 192.168.82.9 is going to be for the management of the server and 192.168.82.10 is what the website will respond too. eth1 has a network of its own 10.0.0.0/24 and I have two Debian Apache web servers that are connected to this network 10.0.0.10 and 10.0.0.20. If this was a real world setup I might have put the management address (192.168.82.9) on a different port but I didn't think of that at the time of setting this up but I may do that later.
Second what have I done! Well this is a brand new install of Centos6.2 minimal fully patched and both HAProxy and STunnel downloaded and installed. I'm using the devel version of HAProxy but thats a different story and version 4.53 of STunnel.
Now the problem. Both seem to work perfectly as-long as I don't want to know who is accessing my site which make it next to useless. However, I can get HAProxy to report the IP Address of the visitor as long as you visit the HTTP page on port 80 as per my configuration file. I can also get STunnel to work with HAProxy but as soon as I enable 'protocol = proxy' the HTTPS side breaks and all I get in my browser is '400 Bad Request Your browser sent an invalid request'. I've played with everything I can thing of and I still cant get a Transparent STunnel>HAProxy solution working correctly.
Config Files:
stunnel.conf ====================================== chroot = /usr/local/var/lib/stunnel/ #setuid = nobody setgid = nobody pid = /stunnel.pid cert = /usr/local/etc/stunnel/stunnel.pem key = /usr/local/etc/stunnel/stunnel.pem options = NO_SSLv2 debug = 7 #fips = no [https] accept = 192.168.82.10:443 connect = 192.168.82.10:80 protocol = proxy
haproxy.cfg ====================================== global daemon log /dev/log local4 maxconn 40000 ulimit-n 81000 defaults log global mode http contimeout 4000 clitimeout 42000 srvtimeout 43000
listen http1 bind 192.168.82.10:80 mode http option http-server-close option forwardfor source 0.0.0.0 usesrc clientip balance roundrobin server http1_1 10.0.0.10:80 cookie http1_1 check inter 2000 rise 2 fall 3 server http1_1 10.0.0.20:80 cookie http1_1 check inter 2000 rise 2 fall 3
Log file ====================================== Apr 5 12:37:32 lbmaster haproxy[1351]: Proxy http1 started. Apr 5 12:37:33 lbmaster stunnel: LOG5[1353:140163149080512]: stunnel 4.53 on x86_64-unknown-linux-gnu platform Apr 5 12:37:33 lbmaster stunnel: LOG5[1353:140163149080512]: Compiled/running with OpenSSL 1.0.0-fips 29 Mar 2010 Apr 5 12:37:33 lbmaster stunnel: LOG5[1353:140163149080512]: Threading:PTHREAD SSL:+ENGINE+OCSP+FIPS Auth:none Sockets:POLL+IPv6 Apr 5 12:37:33 lbmaster stunnel: LOG5[1353:140163149080512]: Reading configuration from file /usr/local/etc/stunnel/stunnel.conf Apr 5 12:37:33 lbmaster stunnel: LOG5[1353:140163149080512]: FIPS mode is enabled Apr 5 12:37:33 lbmaster stunnel: LOG6[1353:140163149080512]: Initializing service section [https] Apr 5 12:37:33 lbmaster stunnel: LOG5[1353:140163149080512]: Configuration successful Apr 5 12:37:41 lbmaster stunnel: LOG5[1354:140163149076224]: Service [https] accepted connection from 192.168.64.10:53149 Apr 5 12:37:41 lbmaster stunnel: LOG6[1354:140163149076224]: connect_blocking: connecting 192.168.82.10:80 Apr 5 12:37:41 lbmaster haproxy[1352]: Connect from 192.168.82.10:47570 to 192.168.82.10:80 (http1/HTTP) Apr 5 12:37:41 lbmaster stunnel: LOG5[1354:140163149076224]: connect_blocking: connected 192.168.82.10:80 Apr 5 12:37:41 lbmaster stunnel: LOG5[1354:140163149076224]: Service [https] connected remote server from 192.168.82.10:47570 Apr 5 12:37:41 lbmaster stunnel: LOG6[1354:140163149076224]: Server-mode proxy protocol negotiations started Apr 5 12:37:41 lbmaster stunnel: LOG6[1354:140163149076224]: Server-mode proxy protocol negotiations succeeded Apr 5 12:37:41 lbmaster stunnel: LOG6[1354:140163149076224]: SSL accepted: new session negotiated Apr 5 12:37:41 lbmaster stunnel: LOG6[1354:140163149076224]: Negotiated TLSv1/SSLv3 ciphersuite: DHE-RSA-AES256-SHA (256-bit encryption) Apr 5 12:37:41 lbmaster stunnel: LOG6[1354:140163149076224]: Compression: null, expansion: null Apr 5 12:37:41 lbmaster stunnel: LOG6[1354:140163149076224]: SSL_shutdown successfully sent close_notify alert Apr 5 12:37:41 lbmaster stunnel: LOG5[1354:140163149076224]: Error detected on socket (read) file descriptor: Broken pipe (32) Apr 5 12:37:41 lbmaster stunnel: LOG5[1354:140163149076224]: Connection reset: 187 byte(s) sent to SSL, 1 byte(s) sent to socket
Any help would be most gracefully received and welcome.
~Yours, Scott
Scott McKeown wrote:
Both seem to work perfectly as-long as I don't want to know who is accessing my site which make it next to useless. However, I can get HAProxy to report the IP Address of the visitor as long as you visit the HTTP page on port 80 as per my configuration file. I can also get STunnel to work with HAProxy but as soon as I enable 'protocol = proxy' the HTTPS side breaks and all I get in my browser is '400 Bad Request Your browser sent an invalid request'. I've played with everything I can thing of and I still cant get a Transparent STunnel>HAProxy solution working correctly.
[cut]
haproxy.cfg
global daemon log /dev/log local4 maxconn 40000 ulimit-n 81000 defaults log global mode http contimeout 4000 clitimeout 42000 srvtimeout 43000
listen http1 bind 192.168.82.10:80 mode http option http-server-close option forwardfor source 0.0.0.0 usesrc clientip balance roundrobin server http1_1 10.0.0.10:80 cookie http1_1 check inter 2000 rise 2 fall 3 server http1_1 10.0.0.20:80 cookie http1_1 check inter 2000 rise 2 fall 3
I'm not a haproxy expert, but it looks like you forgot to specify "accept-proxy" setting in the "bind" option.
http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt
3. Implementations
Haproxy 1.5 implements the PROXY protocol on both sides : - the listening sockets accept the protocol when the "accept-proxy" setting is passed to the "bind" keyword. Connections accepted on such listeners will behave just as if the source really was the one advertised in the protocol. This is true for logging, ACLs, content filtering, transparent proxying, etc...
- the protocol may be used to connect to servers if the "send- proxy" setting is present on the "server" line. It is enabled on a per-server basis, so it is possible to have it enabled for remote servers only and still have local ones behave differently. If the incoming connection was accepted with the "accept-proxy", then the relayed information is the one advertised in this connection's PROXY line.
Mike
-
Michal Trojnara -
Scott McKeown