Dear Michal, Dear all, I have continued to work on my WCE version of stunnel 4.34, after having received some request for help to compile for other TARGET CPU than ARM, typically WCE/X86.
That led me to a few more compilation bug fixes and some improvments in makefiles, proper usage of winsock2 library, automatisation and management of "make/build" process for multiple targets WCE (ARM|X86...) and Win32 (with MS VC and Gnu-mingw32 !). I added also, as a practical feature, the inclusion of "version/copyright" properties in the exe files, properties that are displayable in the Windows Explorer or by right-clicking on the exefile. This is not only a "beautifying" feature: this allows automatic version checking by external tools without having to run the stunnel program, particularly useful when the targetcpu does not match the host cpu...
I also worked to have "as close as possible" evc.mak, vc.mak, and mingw.mak files. A lot of work and test-fail-adapt cycles have been performed to have a common code working with the various flavors of C-PREPROCESSORS ! because there are subtle differences between MS EVC, MS VC, Borland BCC and GNU-GCC preprocessors.
THE ATTACHMENT has been sent in zip format, because it contains a few scripts and other files that exceed the 100KB limit of the mailing list. it contains the files : version.h : contains the #define version numbers, major and minor version.rc : version displayable by Windows Explorer property sheet makew32.bat : an helper for build wih MS VC
Michal: patched2X86.txt : a FULL diff -cr between your last 4.34 official version and my last patched patched2X86_incr.txt : an INCREMENTAL diff -cr between my previous patched version sent on 27/09/2010 and the present one.
For the impatient, my patched version of stunnel 4.34 is available here : http://delaage.pierre.free.fr/contrib/stunnel/wce/stunnel434_WCEpatched2X86....
IT COVERS both WCE and Win32 versions of stunnel.
NEW FILES have been added to the sources: see attachments: - version.h is NOW the CENTRAL point to define major and minor versions of stunnel (NO MORE definition is makefile, which is logical : you modify the version of the product, without necessarily modifying the production process). - version.rc is the resource file required by windows as the repository of the stunnel.exe properties displayed in Windows Explorer. Compiled with evc for wce or with VC for win32, this works perfectly. But compiled and linked with gcc, those properties are unfortunately not displayed by Windows Explorer, although embedded in the exe file. I have established that it is a bug in "gnu-link" stage. - makew32 : the equivalent of makece for Win32, which avoid repetitive and error prone pollution of env vars when repetitively compiling for w32. It is very useful in frequent recompilation at dev/test time.
*** SOME PRACTICAL INFO : - now to compile, or clean up things, for a particular WCE TARGET CPU : just type things like : makece ARMV4, eg for ARMV4 CPU makece ARMV4 clean makece X86, eg for ARMV4 CPU makece X86clean ===> this create ../bin/ARMV4/stunnel.exe and ../obj/ARMV4/all objs files, or the like with X86 cpu and so on. See list of all supported target cpu in evc.mak. ARMV4 and X86 go well; more cpu may be checked by experts vs compilation flags... This is very practical to compile for various targets without modifying, deleting, renaming, anything and with no side effects between targets.
- For Win32, yes for Windows PC, just type : makew32 makew32 clean ===> this create ../bin/W32/stunnel.exe and ../obj/W32/all objs files. This tool uses MS-VC 2008 express as compiler/linker.
- For Win32, but with MINGW32 tools (and GNU-WIN32 coreutils !!!), just type : mingw32-make -f mingw.mak mingw32-make -f mingw.mak clean ===> this create ../bin/MGW32/stunnel.exe and ../obj/MGW32/objs files. GOODIE: I have tricked the mingw.mak to work EITHER on a Windows compilation HOST MACHINE, OR a LINUX/UN*X compilation HOST. On windows mingw.mak properly checks and warns about potential missing of gnu-win32 tools. Note: the make.bat file is just a shortcut for mingw32-make -f mingw.mak.
*** MODIFICATIONS of FILES :
So, since my previous version of the patch, here are my adds :
1/ evc.mak : 1.1/linking with a recompiled version of Essemer/wcecompat: this library has been customized to compile in SEPARATE folders for MULTIPLE targets. So path to proper libs have been changed. My customized version of wcecompat is available here : http://delaage.pierre.free.fr/contrib/wcecompat/wcecompat12_patched2X86.zip
IT HAS ALSO been adapted to have separate folders for distinct targets pre-compiled versions. Please note that this philosophy is shared with openssl.
1.2/ linking with winsock2 library instead of winsock1: at first to be consistent with the C code, and second because winsock2 is more close to socket standard than ws1.
1.3/ linking with COREDLL.lib and CORELIBC.lib ONLY: whatever /MC or /MT or other flag you put for CLxxx command, the set of default libs expected to be linked is WRONG. This is a bug of CLxxx tools, inconsitent with MS doc. SO we have to overwrite the default lib list by always the same ones. You can verify that this is the right way to do it by searching for a libc.lib in various evc/libs : you will NOT find any although clxxx /? talks about it.
1.3/ change in VERSION management : now there is a version.h and version.rc souce file for this. One that wants to change version has just to edit version.h and then version info will be propagated as usual in resource.h, AND also in the exe itself to be displayable in Windows Explorer. Now people will have a simple mean to check the version of stunnel.
Michal: have you noticed that there was a vestige "version.h" file mentioned in vc.mak ! now it makes sense in evc.mak, vc.mak and mingw.mak, for wce or win32 targets.
1.4/ link /machine : IX86 is improper with WCE, it is X86.
1.5/ Multiple WCE CPU targets supported by giving targetcpu on the make command line like this : makece ARMV4 all targetcpu is now mandatory. makece without options, as usual, will show the new syntax (very simple).
1.6/ improved clean target with some rmdirs and proper redirection of error messages. proper use of make flags -@.
1.7/ added explicit "inference rules" that are compatible MS nmake/ Borland make, to be sure of the proper options, and to manage multiple subdirs management matching TARGETCPU. Now stunnel.exe is build in stunnel/bin/TARGETCPU subdir objs are build in stunnel/obj/TARGETCPU
I really think it is practical and good policy : on my host machine I have been able to test efficiently at the same time three flavors of stunnel : win32, wce/arm, wce/x86 without erasing/renaming/recompiling each time and wondering for what platform was my stunnel.Exe built for.
1.8/ HOST flag put in $(DEFINES) for CC and RC to add more info to exe info property shhet in Windows Explorer "this stunnel is for THIS type of target"
2/ makece.bat: protection against pollution of environment when makece is repeatedly run with the same targetcpu. + Env is updated only when one is switching targetcpu. This is not perfect, but a step in the good direction. To understand what I mean, perform "set" to see all the env vars after each makece call...
PRACTICALLY : you can type makece ARMV4 and then makece X86 in the same command line window, without polluting "too much" the environment, only when changing target. No need, reasonably, to have multiple command line windows opened.
3/ vc.mak: for win32 "multiple target" support by putting w32 obj and exe files in obj/w32 and bin/w32 subdirs, version.rc support, winsock2 lib integration.
crypt32 was missing .lib SUFFIX !
improper use of winsock1 : it is better to take winsock2 lib, and it is what the code requires (#includes...) !
version management through version.h /.rc instead of /DVERSION
general logic now close to that of WCE evc.mak, as can be (evc and vc are very close in their syntax, flags...). HOST flag to add more info to exe info property sheet in Windows Explorer "this stunnel is for THIS type of target"
4/ created a makew32.bat script as a wrapper to setenv vars for MSVC 2008 Express, and nmake call. This has to be improved in the future: presently it cannot be called after a previous compilation for WCE : pollution of PATH and INCLUDE env vars requires to reopen a shell via cmd.exe. One day I will check if there a simple way to clean env vars from some path (not so simple).
5/ adapted mingw.mak to link with WINSOCK 2 (!!) and use of version.rc/.h + general processing made similar to that of vc.mak. I tested with mingw for windows and it works. Someone should test with gcc under linux, this should work... HOST flag to add more info to exe info property sheet in Windows Explorer "this stunnel is for THIS type of target" Some rewriting to have a mingw.mak close to vc.mak, to ease maintenance... I checked gnu make syntax, and it is close to ms nmake and borland make, except for ifdef directives and inference rules syntax, unfortunately. I am somehow skilled in this boring exercise of comparing various make tools and syntax.
MINGW.MAK bugs: incorrect reference to "Makefile.w32", incorrect INC path for openssl/win32 (this is NOT "outinc" but "inc32"! since a long time).
NOTE : to compile on windows I commented "-l zdll" lib in the mingw.mak. Michal, may you restore it if it is useful (I do not see why anyway).
Added a testenv pseudo-target to check for gnu-win32 proper installation. Added some tricks to enable proper execution with the same makefile either on a Windows HOST or a Linux HOST. When I say host, I mean "production machine" , and NOT TARGET machine which is always win32 with mingw.mak.
6/ make.bat: this script is ONLY meaningful with mingw.mak and MINGW32 gnu environment. Due to various improvments I MADE in mingw.mak, mingw.mak is now only compatible with mingw32-make, and not just "make.exe", which is Borland make on Windows...
TODO : I did not deal with stunnel making for linux, and/or rpm packaging. But I guess that makefile.in etc...should be updated... Anyway, on linux, NOT including version.rc is not a problem, as this file is useless in that target environment. But I think that having ../obj/UNX and ../bin/UNX folders should be a good practice...
7/ common.h : Eureka ! I finally understood and cleaned up the mess around the redefinition of various Socket Error codes. This comes from the fact that, previously, those codes were defined in MS winsock1 header files, but disappeared in winsock2 header files (they are commented) Apparently, stunnel code was including winsock1 for w32 but winsock2 for WCE, although winsock2 is in fact available on both platforms. So I decided to move to winsock2 for WCE, and re-declaring (again) SOME of those constants. Some clean up of the makefile has been done also to REALLY link to ws2 lib instead of ws1 (because there was also a confusion there!).
8/ ctx.c : htons is requesting "short". so a cast was required to avoid warnings from the compiler.
9/ resolver.c : here we are: many routines have been reimplemented when they are in fact available in winsock2 both for w32 AND WCE. Michal: I expect that you review this with particular attention. In fact I have modified the logic of "masking" local_routines when already defined in winsock2. the previous code was HIDING winsock2 instead of the local_routines. It seemed not logical. It is important to note that the three routines "getaddrinfo, freeaddrinfo, getnameinfo" are FOR INSTANT used only in resolver.c, BUT the previous code was preventing them from being used by other modules. Now, with my code, it is possible. My code also avoid compilation warnings (because, as getaddrinfo REALLY exist in win32, such a "#define getaddrinfo localgetaddrinfo" inevitably raises a warning of the compiler for redef of an existing symbol; and hence it reveals a problem of conflict..., while my solution avoids all this pitfalls).
Removed also the restriction "AND IPV6" on win32 conditional #IF: winsock2 supports ipv6. Michal: a review on these conditional #IF should be useful...
I wanted to add that those routines are also parts of the POSIX standard API, so they are available almost everywhere... so I hardly understand their purpose today.
10/ version.h : I have used some preprocessor tricks to have a VERSION string correctly defined BOTH with MS evc preprocessor, MS vc preprocessor: AND GNU preprocessor. those preprocessors have some differences...particularly when called from a rc compiler through a "pipe" like gnu windres. In ms VC and EVC the rc preprocessing is NOT the same as that of the compiler... Now it works with every build system, and either in .c compilation or .rc compilation.
USEFUL : openssl team is desperately ignoring any WCE patch, so if you want an openssl 100a source tree compatible with WCE, just download mine there : http://delaage.pierre.free.fr/contrib/openssl/wce/openssl100a_WCEpatched3.zi... FYI, my patch to openssl is registered here : http://rt.openssl.org/index.html?q=2350 (login guest/password guest).
TODO : gui.c contains code to dynamically load ws2 getaddrinfo why not rely only on static linking for that, as ws2 is available everywhere since w95 and wce4.1 ? we have to harmonize the code there with resolver.c and various makefiles In resolver.c: code remains to load dynamically ws2 getaddrinfo etc..at run-time only on win32, and not in CE while it is ALSO available! I suggest to have "static" linking to ws2.lib both in w32 and wce . but more code has to be modified to be definitely clean and clear. When I say "static linking" I mean "Load-Time Dynamic Linking", ie link with ws2.lib stub at build time, not "link with static lib"...
I thank you for your good work and attention, Yours sincerely,
Pierre Delaage
Le 27/09/2010 00:40, Pierre DELAAGE a écrit :
Dear All, Please find enclosed a patch in "diff -cr orig patched" format, applying to stunnel4.34 as found here ftp://stunnel.mirt.net/stunnel/. This patch mainly addresses compilation and unicode issues for Windows CE targets + ONE critical issue preventing stunnel to service anything (!).
I use MS EVC 4sp2 compiler with WCE 420 SDK, on a vista host platform. Once debugged the code works fine on WM6 HTC smartphones. Should work on WM5. It needs a windows CE openssl lib (I recompiled MY patched version of openssl 1.0.0a successfully: I will have to log a patch to those gentlemen, hoping that they are open to integrate it, something not so obvious in the past).
I will later post here a link to my patch for openssl 100a pointing to the openssl mailing list. THIS OPENSSL PATCH IS ESSENTIAL : with my older version of openssl 098pre_j for wce, stunnel does not work anymore (it crashes after first cnx).
The present stunnel patch addresses the following issues :
I] COMPILATION FAILURES
1/ on common.h "EINVAL redefinition" : in fact the compile options WX (see evc.mak) does not tolerate any warning : The MS evc420 compiler ALWAYS issues a warning on this symbol redefinition. Michal and I are dealing with it since month... But, considering that NEITHER Exxx error codes are used in stunnel, I suggest that in fact ALL these redef should be suppressed.
TODO: In the future a proper rewrite of code in log.c, where hardcoded values of WSA error codes are used instead of symbolic constant, will have to be done.
2/ client.c CreateProcess is a UNICODE function: code was ascii only. Code has been adapted to work in unicode.
3/ evc.mak path to new openssl lib has been redefined. proper location for the libs in openssl tree has been set in the make-install block (although this block is presently USELESS and needs a complete rewrite).
4/ gui.c All code using hmainmenu is OFF under winCE. So some new lines involving it were not compiling.
5/ options.c
linger_val.l_linger and linger_val.l_onoff require ushort values. Strict type checking of the compiler required a proper cast to ushort of the r-values.
stralloc uses "strdup": on Win32 and winCE strdup is _strdup (!).
6/ prototypes.h : _beginthread was defined as returning an int, an a long in sthread.c. After some checking in various documentations, long is the correct type, in particular because _beginthread can return the special error value "-1L". So that unsigned types are illogical. This simplifies the code in some locations, by saving some -now useless- casts.
7/ sthread.c
create_client: suppressed some useless cast to (long) for beginthread.
II] OPERATIONAL ERRORS (at run-time) This is a reminder of previous notification to the mailing list : due to beginthread improper code and/or use, It is presently IMPOSSIBLE for stunnel 434 to be used in production on WCE: it is just unable to service any...service. Nothing happens if, for example, you use it to connect to an https server. No log either !
I corrected the code with both Michal patch to gui.c and mine to sthread.c:
gui.c (Michal patch) : _beginthread was called with 0 as stacksize which, cumulated to an improper coding of sthread/beginthread, led to FAILURE of service thread creation.
sthread.c: _beginthread : now supports "0" stacksize value, leading to default linker stack size (1MB by default unless specified differently on the linker command line, something we do not yet perform but could).
III] MINOR IMPROVMENT _beginthread: added a log on critical event of "failure" to create a service thread. because I think it is a critical event, isn't it?
I hope you will find this patch useful. Thank you for your excellent work, Yours sincerely,
Pierre Delaage
Note : I use stunnel to establish a simple "vpn" between smartphones and a corporate linux server mainly for HTTPS/POPS/SMTPS support. Stunnel is very relevant in that matter, over solutions based on SSH (although we use also ssh), from a communication cost point of view : ssh establishes permanent socket between client and server, so that the communication is charged by the mobile network provider : and these charges are very expensive. On the contrary stunnel only establishes ssl sockets on demand so that financial charges are limited to strict necessary. Please note that stunnel brings "client based certificate authentication" to POP/SMTP mobile mail user agents which only BASICALLY supports SSL with server authentication, but NO client authentication, such as M$ Outlook for Mobile (unless you pay for an exchange server and exchange client licence). Here again stunnel is very relevant.
Note 2 : the strdup and unicode bug fixes should benefit also to the win32 (for PC) stunnel version for PC.