Hdf5 merge issue 537 v110 (#545)

* close #195. (#196)

* Update HDF5PluginMacros.cmake

* Update HDF5PluginMacros.cmake

* Modify temporary rpath for testing in java example scripts. (#230)

* Fix undefined left shifting of negative numbers (#338)

Undefined Bahavior Sanitizer errored here about left shifting negative numbers.

* Update license url (#332)

* Modify temporary rpath for testing in java example scripts.

* Update URL in source file Copyright headers for web copy of COPYING
file - src and test directories.

* Cleans up a couple of MSVC warnings in testhdf5 (#475)
* Fixes a few testhdf5 warnings raised in Visual Studio

Visual Studio is grumpier about treating pointers like integers than
gcc.

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Fix typos and grammar errors. (#476)

* Suppresses the tcheck_version test's abort dialog on Windows (#477)

* Suppresses the tcheck_version test's abort dialog on Windows

Windows raises a modal abort/retry/ignore dialog box when CRT
calls abort(). This change installs a report hook that suppresses
the dialog so that the CMake tests don't time out waiting for a
nonexistent user to click a dialog box.

* Committing clang-format changes

* Removes __cdecl from callback

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* More warning fixes (#400)

* Fixed -Wunused-local-typedef warning

* Fixed -Wformat warnings

In one case also removed a `z` character. There was a `z%d` that I think was supposed to be `%zd`

* Fixed -Wshorten-64-to-32 warnings

* Fixed -Wself-assign warnings

* Fixed -Wreserved-id-macro warnings

* Commit format changes from clang-format, clang version 10.0.1.

* Fixed -Wself-assign warnings

* Fixed -Wunused-local-typedef warning

* Fixed -Wformat warnings

In two cases also removed a `z` character. There was a `z%d` that was supposed to be `%zd`.

* Fixed -Wshorten-64-to-32 warnings

* Fixed -Wreserved-id-macro warnings

* Fixed -Wself-assign warnings

* Format source.

* Remove blank lines to pass format check.

Co-authored-by: Larry Knox <lrknox@hdfgroup.org>

* Changes found during merge to 1.12 (#414)

* OESS-98 convert plugin option to FetchContent, add tests

* Fixes for pkcfg files because of plugin option

* OESS-98 fix tools test for plugins

* Keep doxygen comments under 100 chars long - format hint

* Whitespace

* HDFFV-11144 - Reclassify CMake messages

* HDFFV-11099/11100 added help text

* Reworked switch statement to compare string instead

* Fix typo

* Update CDash mode

* Correct name of threadsafe

* Correct option name

* Undo accidental commit

* Note LLVM 10 to 11 format default changes

* Update format plugin

* Undo clang-format version 11 changes

* One more correction

* Update supported platforms

* Revert whitespace changes

* Correct whitespace

* Changes from PR#3

* HDFFV-11213 added option to control gcc10 warnings diagnostics

* HDFFV-11212 Use the new references correctly in JNI utility and tests

* format source

* Fix typo

* Add new test file

* HDFFV-11212 - update test and remove unused arg

* Minor non-space formatting changes

* Use H5I_INVALID_ID instead of "-1"

* source formatting

* add missing testfile, update jni function

* Undo commit of debug code

* remove mislocated file

* Fix h5repack test for handling of fapls and id close

* Update h5diff test files usage text

* HDFFV-11212 add new ref tests for JNI export dataset

* src format update

* Remove blank line typo

* src format typo

* long double requires %Lg

* Another long double foramt specifer S.B. %Lg

* issue with t128bit test

* Windows issue with h5dump and type.

* Fix review issues

* refactor function nesting and fix error checks

* format fixes

* Remove untested functions and javadoc quiet comments

* Restore TRY block.

* Change string append errors to memory exception

* revert to H5_JNI_FATAL_ERROR - support functions need work

* Add assertion error for h5util functions

* remove duplicate function

* format fix

* Revert HD function error handling

* Update copyright comments

* GH #386 java folder copyright corrections

* Whitespace

* GH #359 implement and fix tools 1.6 API usage

* remove excessive comments

* Flip inits to correct ifdef section

* rework ifdef to be simpler

* format issue

* Reformat ifdef inits

* remove static attribute

* format compliance

* Update names

* Revert because logic relies on float not being int

* Changes noticed from creating merge of #412

* Double underscore change

* Correct compiler version variable used

* Remove header guard underscores

* Whitespace cleanup

* File locks now work on Windows (#480)

* File locks now work on Windows

Uses LockFileEx() and UnlockFileEx(). Fixes HDFFV-10191 (partial).

* Committing clang-format changes

* Committing clang-format changes

* Fixes commenting in h5repack

* Reworks H5Fis_accessible()

H5Fis_accessible() created a new file handle and attempted to read
through it, which will fail when a file has been opened with an
exclusive lock on operating systems that have mandatory locks.

This change uses the same scheme we use in H5Fopen() to check if
the file is already open and only tries to read the file signature
if the file has not already been opened.

Also adds a test for this behavior.

* Committing clang-format changes

* Trivial change to force github to run actions

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* small edits (#481)

* fixed missed closing of a dataset

* fixed missed closing of a dataset

* fixed typo in error return

* Committing clang-format changes

* minor edits

* code format

* Committing clang-format changes

* code format

* minor edit

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Adds an implementation for nanosleep on Windows (#483)

* Adds an implementation for nanosleep on Windows

* Committing clang-format changes

* Fix insane formatter issue

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Updates HSYS_GOTO_ERROR to emit GetLastError() values on Win32 (#492)

* Committing clang-format changes

* Updates H5SYS_GOTO_ERROR to emit Win32's GetLastError()

* Committing clang-format changes

* Format source changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Updates that allow accum SWMR tests to work on Windows (#493)

* Committing clang-format changes

* Updates the accum test to work on Windows

Uses CreateProcess() on Windows

* Source formatting

* Fix for Unix child result

* Fixes process return code bug on Windows

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Disable long double tests and remove CMake flag groups (#494)

* OESS-98 convert plugin option to FetchContent, add tests

* Fixes for pkcfg files because of plugin option

* OESS-98 fix tools test for plugins

* Keep doxygen comments under 100 chars long - format hint

* Whitespace

* HDFFV-11144 - Reclassify CMake messages

* HDFFV-11099/11100 added help text

* Reworked switch statement to compare string instead

* Fix typo

* Update CDash mode

* Correct name of threadsafe

* Correct option name

* Undo accidental commit

* Note LLVM 10 to 11 format default changes

* Update format plugin

* Undo clang-format version 11 changes

* One more correction

* Update supported platforms

* Revert whitespace changes

* Correct whitespace

* Changes from PR#3

* HDFFV-11213 added option to control gcc10 warnings diagnostics

* HDFFV-11212 Use the new references correctly in JNI utility and tests

* format source

* Fix typo

* Add new test file

* HDFFV-11212 - update test and remove unused arg

* Minor non-space formatting changes

* Use H5I_INVALID_ID instead of "-1"

* source formatting

* add missing testfile, update jni function

* Undo commit of debug code

* remove mislocated file

* Fix h5repack test for handling of fapls and id close

* Update h5diff test files usage text

* HDFFV-11212 add new ref tests for JNI export dataset

* src format update

* Remove blank line typo

* src format typo

* long double requires %Lg

* Another long double foramt specifer S.B. %Lg

* issue with t128bit test

* Windows issue with h5dump and type.

* Fix review issues

* refactor function nesting and fix error checks

* format fixes

* Remove untested functions and javadoc quiet comments

* Restore TRY block.

* Change string append errors to memory exception

* revert to H5_JNI_FATAL_ERROR - support functions need work

* Add assertion error for h5util functions

* remove duplicate function

* format fix

* Revert HD function error handling

* Update copyright comments

* GH #386 java folder copyright corrections

* Whitespace

* GH #359 implement and fix tools 1.6 API usage

* remove excessive comments

* Flip inits to correct ifdef section

* rework ifdef to be simpler

* format issue

* Reformat ifdef inits

* remove static attribute

* format compliance

* Update names

* Revert because logic relies on float not being int

* Changes noticed from creating merge of #412

* Double underscore change

* Correct compiler version variable used

* Remove header guard underscores

* Whitespace cleanup

* Split format source and commit changes on repo push

* remove pre-split setting

* Change windows TS to use older VS.

* correct window os name

* HDFFV-11212 JNI export util and Javadoc

* Suggested review changes

* Another change found

* Committing clang-format changes

* HDFFV-11113 long double in tools

* HDFFV-11113 add note

* Disable long double tests for now

* HDFFV-11228 remove arbitrary CMake warning groups.

* Make each flag a string

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Misc warning fixes (#495)

* Committing clang-format changes

* Misc warning fixes from Visual Studio

* Committing clang-format changes

* Fixes warnings in swmr.c test

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Correct merge errors.

* Committing clang-format changes

Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org>
Co-authored-by: Sean McBride <sean@rogue-research.com>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: Scot Breitenfeld <brtnfld@hdfgroup.org>

4 files changed, 106 insertions, 61 deletions

diff --git a/src/H5Eprivate.h b/src/H5Eprivate.h
index 58010a3..c3c440f 100644
--- a/src/H5Eprivate.h
+++ b/src/H5Eprivate.h
@@ -111,6 +111,7 @@ typedef struct H5E_t H5E_t;
 /* Retrieve the error code description string and push it onto the error
  * stack.
  */
+#ifndef H5_HAVE_WIN32_API
 #define HSYS_DONE_ERROR(majorcode, minorcode, retcode, str)                                                  \
     {                                                                                                        \
         int myerrno = errno;                                                                                 \
@@ -129,6 +130,35 @@ typedef struct H5E_t H5E_t;
         HGOTO_ERROR(majorcode, minorcode, retcode, "%s, errno = %d, error message = '%s'", str, myerrno,     \
                     HDstrerror(myerrno));                                                                    \
     }
+#else /* H5_HAVE_WIN32_API */
+/* On Windows we also emit the result of GetLastError(). This call returns a DWORD, which is always a
+ * 32-bit unsigned type. Note that on Windows, either errno or GetLastError() (but probably not both) will
+ * be useful depending on whether a C/POSIX or Win32 call failed. The other value will likely be zero,
+ * though I wouldn't count on that.
+ */
+#define HSYS_DONE_ERROR(majorcode, minorcode, retcode, str)                                                  \
+    {                                                                                                        \
+        int   myerrno   = errno;                                                                             \
+        DWORD win_error = GetLastError();                                                                    \
+        /* Other projects may rely on the description format to get the errno and any changes should be      \
+         * considered as an API change                                                                       \
+         */                                                                                                  \
+        HDONE_ERROR(majorcode, minorcode, retcode,                                                           \
+                    "%s, errno = %d, error message = '%s', Win32 GetLastError() = %" PRIu32 "", str,         \
+                    myerrno, HDstrerror(myerrno), win_error);                                                \
+    }
+#define HSYS_GOTO_ERROR(majorcode, minorcode, retcode, str)                                                  \
+    {                                                                                                        \
+        int   myerrno   = errno;                                                                             \
+        DWORD win_error = GetLastError();                                                                    \
+        /* Other projects may rely on the description format to get the errno and any changes should be      \
+         * considered as an API change                                                                       \
+         */                                                                                                  \
+        HGOTO_ERROR(majorcode, minorcode, retcode,                                                           \
+                    "%s, errno = %d, error message = '%s', Win32 GetLastError() = %" PRIu32 "", str,         \
+                    myerrno, HDstrerror(myerrno), win_error);                                                \
+    }
+#endif /* H5_HAVE_WIN32_API */
 
 #ifdef H5_HAVE_PARALLEL
 /*
diff --git a/src/H5Fint.c b/src/H5Fint.c
index 9ddd3d7..c8e2cad 100644
--- a/src/H5Fint.c
+++ b/src/H5Fint.c
@@ -918,9 +918,10 @@ done:
 htri_t
 H5F__is_hdf5(const char *name)
 {
-    H5FD_t *file      = NULL;        /* Low-level file struct                    */
-    haddr_t sig_addr  = HADDR_UNDEF; /* Addess of hdf5 file signature            */
-    htri_t  ret_value = FAIL;        /* Return value                             */
+    H5FD_t *      file      = NULL;        /* Low-level file struct            */
+    H5F_shared_t *shared    = NULL;        /* Shared part of file              */
+    haddr_t       sig_addr  = HADDR_UNDEF; /* Addess of hdf5 file signature    */
+    htri_t        ret_value = FAIL;        /* Return value                     */
 
     FUNC_ENTER_PACKAGE
 
@@ -928,10 +929,20 @@ H5F__is_hdf5(const char *name)
     if (NULL == (file = H5FD_open(name, H5F_ACC_RDONLY, H5P_FILE_ACCESS_DEFAULT, HADDR_UNDEF)))
         HGOTO_ERROR(H5E_FILE, H5E_CANTINIT, FAIL, "unable to open file")
 
-    /* The file is an hdf5 file if the hdf5 file signature can be found */
-    if (H5FD_locate_signature(file, &sig_addr) < 0)
-        HGOTO_ERROR(H5E_FILE, H5E_NOTHDF5, FAIL, "error while trying to locate file signature")
-    ret_value = (HADDR_UNDEF != sig_addr);
+    /* If the file is already open, it's an HDF5 file
+     *
+     * If the file is open with an exclusive lock on an operating system that enforces
+     * mandatory file locks (like Windows), creating a new file handle and attempting
+     * to read through it will fail so we have to try this first.
+     */
+    if ((shared = H5F__sfile_search(file)) != NULL)
+        ret_value = TRUE;
+    else {
+        /* The file is an HDF5 file if the HDF5 file signature can be found */
+        if (H5FD_locate_signature(file, &sig_addr) < 0)
+            HGOTO_ERROR(H5E_FILE, H5E_NOTHDF5, FAIL, "error while trying to locate file signature")
+        ret_value = (HADDR_UNDEF != sig_addr);
+    }
 
 done:
     /* Close the file */
@@ -1603,7 +1614,7 @@ H5F_open(const char *name, unsigned flags, hid_t fcpl_id, hid_t fapl_id)
     FUNC_ENTER_NOAPI(NULL)
 
     /*
-     * If the driver has a `cmp' method then the driver is capable of
+     * If the driver has a 'cmp' method then the driver is capable of
      * determining when two file handles refer to the same file and the
      * library can insure that when the application opens a file twice
      * that the two handles coordinate their operations appropriately.
@@ -3546,10 +3557,17 @@ H5F__start_swmr_write(H5F_t *f)
     setup = TRUE;
 
     /* Place an advisory lock on the file */
-    if (H5F_USE_FILE_LOCKING(f))
+    if (H5F_USE_FILE_LOCKING(f)) {
+        /* Have to unlock on Windows as Win32 doesn't support changing the lock
+         * type (exclusive vs shared) with a second call.
+         */
+        if (H5FD_unlock(f->shared->lf) < 0) {
+            HGOTO_ERROR(H5E_FILE, H5E_CANTUNLOCKFILE, FAIL, "unable to unlock the file")
+        }
         if (H5FD_lock(f->shared->lf, TRUE) < 0) {
             HGOTO_ERROR(H5E_FILE, H5E_CANTLOCKFILE, FAIL, "unable to lock the file")
         }
+    }
 
     /* Mark superblock as dirty */
     if (H5F_super_dirty(f) < 0)
diff --git a/src/H5system.c b/src/H5system.c
index 26054e7..99a6f4d 100644
--- a/src/H5system.c
+++ b/src/H5system.c
@@ -924,8 +924,8 @@ Wgetlogin(void)
 {
 
 #ifdef H5_HAVE_WINSOCK2_H
-    long bufferCount = WloginBuffer_count;
-    if (GetUserName(Wlogin_buffer, &bufferCount) == 0)
+    DWORD bufferCount = WloginBuffer_count;
+    if (GetUserName(Wlogin_buffer, &bufferCount) != 0)
         return (Wlogin_buffer);
     else
 #endif /* H5_HAVE_WINSOCK2_H */
@@ -969,61 +969,45 @@ c99_vsnprintf(char *str, size_t size, const char *format, va_list ap)
  *-------------------------------------------------------------------------
  */
 int
-Wflock(int H5_ATTR_UNUSED fd, int H5_ATTR_UNUSED operation)
+Wflock(int fd, int operation)
 {
 
-/* This is a no-op while we implement a Win32 VFD */
-#if 0
-int
-Wflock(int fd, int operation) {
-
-    HANDLE          hFile;
-    DWORD           dwFlags = LOCKFILE_FAIL_IMMEDIATELY;
-    DWORD           dwReserved = 0;
-                    /* MAXDWORD for entire file */
-    DWORD           nNumberOfBytesToLockLow = MAXDWORD;
-    DWORD           nNumberOfBytesToLockHigh = MAXDWORD;
-                    /* Must initialize OVERLAPPED struct */
-    OVERLAPPED      overlapped = {0};
+    HANDLE hFile;
+    DWORD  dwFlags    = LOCKFILE_FAIL_IMMEDIATELY;
+    DWORD  dwReserved = 0;
+    /* MAXDWORD locks the entire file */
+    DWORD nNumberOfBytesToLockLow  = MAXDWORD;
+    DWORD nNumberOfBytesToLockHigh = MAXDWORD;
+    /* Must initialize OVERLAPPED struct */
+    OVERLAPPED overlapped = {0};
 
     /* Get Windows HANDLE */
-    hFile = _get_osfhandle(fd);
+    if (INVALID_HANDLE_VALUE == (hFile = (HANDLE)_get_osfhandle(fd)))
+        return -1;
 
     /* Convert to Windows flags */
-    if(operation & LOCK_EX)
+    if (operation & LOCK_EX)
         dwFlags |= LOCKFILE_EXCLUSIVE_LOCK;
 
     /* Lock or unlock */
-    if(operation & LOCK_UN)
-        if(0 == UnlockFileEx(hFile, dwReserved, nNumberOfBytesToLockLow,
-                            nNumberOfBytesToLockHigh, &overlapped))
-            return -1;
-    else
-        if(0 == LockFileEx(hFile, dwFlags, dwReserved, nNumberOfBytesToLockLow,
-                            nNumberOfBytesToLockHigh, &overlapped))
+    if (operation & LOCK_UN) {
+        if (0 ==
+            UnlockFileEx(hFile, dwReserved, nNumberOfBytesToLockLow, nNumberOfBytesToLockHigh, &overlapped)) {
+            /* Attempting to unlock an already unlocked file will fail and this can happen
+             * in H5Fstart_swmr_write(). For now, just ignore the "error" (error code: 0x9e / 158).
+             */
+            if (GetLastError() != 158)
+                return -1;
+        }
+    }
+    else {
+        if (0 == LockFileEx(hFile, dwFlags, dwReserved, nNumberOfBytesToLockLow, nNumberOfBytesToLockHigh,
+                            &overlapped))
             return -1;
-#endif /* 0 */
-    return 0;
-} /* end Wflock() */
+    }
 
-/*--------------------------------------------------------------------------
- * Function:    Wnanosleep
- *
- * Purpose:     Sleep for a given # of nanoseconds (Windows version)
- *
- * Return:      SUCCEED/FAIL
- *
- * Programmer:  Dana Robinson
- *              Fall 2016
- *--------------------------------------------------------------------------
- */
-int
-Wnanosleep(const struct timespec *req, struct timespec *rem)
-{
-    /* XXX: Currently just a placeholder */
     return 0;
-
-} /* end Wnanosleep() */
+} /* end Wflock() */
 
 /*-------------------------------------------------------------------------
  * Function:    Wllround, Wllroundf, Wlround, Wlroundf, Wround, Wroundf
@@ -1403,6 +1387,9 @@ done:
  *
  * Purpose:     Sleep for a given # of nanoseconds
  *
+ *              Note that commodity hardware is probably going to have a
+ *              resolution of milliseconds, not nanoseconds.
+ *
  * Return:      SUCCEED/FAIL
  *
  * Programmer:  Quincey Koziol
@@ -1412,15 +1399,26 @@ done:
 void
 H5_nanosleep(uint64_t nanosec)
 {
-    struct timespec sleeptime; /* Struct to hold time to sleep */
-
     FUNC_ENTER_NOAPI_NOINIT_NOERR
 
-    /* Set up time to sleep */
-    sleeptime.tv_sec  = 0;
-    sleeptime.tv_nsec = (long)nanosec;
+#ifdef H5_HAVE_WIN32_API
+
+    /* On Windows, Sleep() is in milliseconds. Passing 0 to Sleep()
+     * causes the thread to relinquish the rest of its time slice.
+     */
+    Sleep(nanosec / (1000 * 1000));
+
+#else
+    {
+        struct timespec sleeptime; /* Struct to hold time to sleep */
+
+        /* Set up time to sleep */
+        sleeptime.tv_sec = 0;
+        sleeptime.tv_nsec = (long)nanosec;
 
-    HDnanosleep(&sleeptime, NULL);
+        HDnanosleep(&sleeptime, NULL);
+    }
+#endif
 
     FUNC_LEAVE_NOAPI_VOID
 } /* end H5_nanosleep() */
diff --git a/src/H5win32defs.h b/src/H5win32defs.h
index 8f3947a..9cd0afb 100644
--- a/src/H5win32defs.h
+++ b/src/H5win32defs.h
@@ -174,7 +174,6 @@ H5_DLL int    Wflock(int fd, int operation);
 H5_DLL char * Wgetlogin(void);
 H5_DLL int    c99_snprintf(char *str, size_t size, const char *format, ...);
 H5_DLL int    c99_vsnprintf(char *str, size_t size, const char *format, va_list ap);
-H5_DLL int    Wnanosleep(const struct timespec *req, struct timespec *rem);
 H5_DLL herr_t H5_expand_windows_env_vars(char **env_var);
 H5_DLL wchar_t *H5_get_utf16_str(const char *s);
 H5_DLL int      Wopen_utf8(const char *path, int oflag, ...);