Workarounds and fixes for wrapped executables on various platforms

regarding the handling of wrapped dynamic libraries.

The basic flow of operation is to copy such libraries into a temp
file, hand them to the OS loader for processing, and then to delete
them immediately, to prevent them from being accessible to other
executables. On platforms where that is not possible the library is
left in place and things are arranged to delete it on regular process
exit.

An example of the latter are older revisions of HPUX which report that
the file is busy when trying to delete it. Younger revisions of HPUX
have changed to allow the deletion, but are also buggy, the OS loader
mangles its data structures so that a second library loaded in this
manner fails.

More recently it was found that Linux which is usually ok with
deleting the file and gets everything right shows the same trouble as
modern HPUX when the "docker" containerization system is involved, or
more specifically the AUFS in use there. Deleting the loaded library
file mangles data structures and breaks loading of the following
libraries. For a demonstration which does not involve Tcl at all see
the ticket

     https://github.com/dotcloud/docker/issues/1911

in the docker tracker.

This of course breaks the use of wrapped executables within docker
containers.

This commit introduces the function TclSkipUnlink() which centralizes
the handling of such exceptions to unlinking the library after unload,
and provides code handling the known cases. IOW HPUX is generally
forced to not unlink, and ditto when we detect that the copied library
file resides within an AUFS.

The latter must however be explicitly activated by setting the define
-DTCL_TEMPLOAD_NO_UNLINK during build. We still need proper configure
tests to set it on the relevant platforms (i.e. Linux).

The AUFS detection and handling can be overridden by the environment
variable TCL_TEMPLOAD_NO_UNLINK which can force the behaviour either
way (skip or not). In case the user knows best, or wishes to test if
the problem with AUFS has been fixed.

1 files changed, 86 insertions, 1 deletions

diff --git a/generic/tclIOUtil.c b/generic/tclIOUtil.c
index 295e313..3d33992 100644
--- a/generic/tclIOUtil.c
+++ b/generic/tclIOUtil.c
@@ -24,6 +24,12 @@
 #endif
 #include "tclFileSystem.h"
 
+#ifdef TCL_TEMPLOAD_NO_UNLINK
+#ifndef NO_FSTATFS
+#include <sys/statfs.h>
+#endif
+#endif
+
 /*
  * struct FilesystemRecord --
  *
@@ -3149,6 +3155,83 @@ Tcl_FSLoadFile(
 typedef int (Tcl_FSLoadFileProc2) (Tcl_Interp *interp, Tcl_Obj *pathPtr,
 	Tcl_LoadHandle *handlePtr, Tcl_FSUnloadFileProc **unloadProcPtr, int flags);
 
+/*
+ * Workaround for issue with modern HPUX which do allow the unlink (no ETXTBSY
+ * error) yet somehow trash some internal data structures which prevents the
+ * second and further shared libraries from getting properly loaded. Only the
+ * first is ok. We try to get around the issue by not unlinking,
+ * i.e. emulating the behaviour of the older HPUX which denied removal.
+ *
+ * Doing the unlink is also an issue within docker containers, whose AUFS
+ * bungles this as well, see
+ *     https://github.com/dotcloud/docker/issues/1911
+ *
+ * For these situations the change below makes the execution of the unlink
+ * semi-controllable at runtime.
+ *
+ *     An AUFS filesystem (if it can be detected) will force avoidance of
+ *     unlink. The env variable TCL_TEMPLOAD_NO_UNLINK allows detection of a
+ *     users general request (unlink and not.
+ *
+ * By default the unlink is done (if not in AUFS). However if the variable is
+ * present and set to true (any integer > 0) then the unlink is skipped.
+ */
+
+int
+TclSkipUnlink (Tcl_Obj* shlibFile)
+{
+    /* Order of testing:
+     * 1. On hpux we generally want to skip unlink in general
+     *
+     * Outside of hpux then:
+     * 2. For a general user request   (TCL_TEMPLOAD_NO_UNLINK present, non-empty, => int)
+     * 3. For general AUFS environment (statfs, if available).
+     *
+     * Ad 2: This variable can disable/override the AUFS detection, i.e. for
+     * testing if a newer AUFS does not have the bug any more.
+     * 
+     * Ad 3: This is conditionally compiled in. Condition currently must be set manually.
+     *       This part needs proper tests in the configure(.in).
+     */
+
+#ifdef hpux
+    return 1;
+#else
+    int   skip = 0;
+    char* skipstr;
+
+    skipstr = getenv ("TCL_TEMPLOAD_NO_UNLINK");
+    if (skipstr && (skipstr[0] != '\0')) {
+	return atoi(skipstr);
+    }
+
+#ifdef TCL_TEMPLOAD_NO_UNLINK
+#ifndef NO_FSTATFS
+    {
+	struct statfs fs;
+	/* Have fstatfs. May not have the AUFS super magic ... Indeed our build
+	 * box is too old to have it directly in the headers. Define taken from
+	 *     http://mooon.googlecode.com/svn/trunk/linux_include/linux/aufs_type.h
+	 *     http://aufs.sourceforge.net/
+	 * Better reference will be gladly taken.
+	 */
+#ifndef AUFS_SUPER_MAGIC
+#define AUFS_SUPER_MAGIC ('a' << 24 | 'u' << 16 | 'f' << 8 | 's')
+#endif /* AUFS_SUPER_MAGIC */
+	if ((statfs(Tcl_GetString (shlibFile), &fs) == 0) &&
+	    (fs.f_type == AUFS_SUPER_MAGIC)) {
+	    return 1;
+	}
+    }
+#endif /* ... NO_FSTATFS */
+#endif /* ... TCL_TEMPLOAD_NO_UNLINK */
+
+    /* Fallback: !hpux, no EV override, no AUFS (detection, nor detected):
+     * Don't skip */
+    return 0;
+#endif /* hpux */
+}
+
 int
 TclLoadFile(
     Tcl_Interp *interp,		/* Used for error reporting. */
@@ -3353,7 +3436,9 @@ TclLoadFile(
      * avoids any worries about leaving the copy laying around on exit.
      */
 
-    if (Tcl_FSDeleteFile(copyToPtr) == TCL_OK) {
+    if (
+	!TclSkipUnlink (copyToPtr) &&
+	(Tcl_FSDeleteFile(copyToPtr) == TCL_OK)) {
 	Tcl_DecrRefCount(copyToPtr);
 
 	/*