FindCUDAToolkit: Improve usage, library set, and tests

Refined the initial design of FindCUDAToolkit and improve it
by adding more library support, more toolkit information and
tests.

10 files changed, 282 insertions, 73 deletions

diff --git a/Modules/FindCUDA.cmake b/Modules/FindCUDA.cmake
index ad7fe99..0e2f551 100644
--- a/Modules/FindCUDA.cmake
+++ b/Modules/FindCUDA.cmake
@@ -5,17 +5,22 @@ FindCUDA
 .. deprecated:: 3.10
 
   Superseded by first-class support for the CUDA language in CMake.
+  Superseded by the :module:`FindCUDAToolkit` for CUDA toolkit libraries.
 
 Replacement
 ^^^^^^^^^^^
 
-It is no longer necessary to use this module or call ``find_package(CUDA)``.
-Instead, list ``CUDA`` among the languages named in the top-level
-call to the :command:`project` command, or call the
+It is no longer necessary to use this module or call ``find_package(CUDA)``
+for compiling CUDA code. Instead, list ``CUDA`` among the languages named
+in the top-level call to the :command:`project` command, or call the
 :command:`enable_language` command with ``CUDA``.
 Then one can add CUDA (``.cu``) sources to programs directly
 in calls to :command:`add_library` and :command:`add_executable`.
 
+To find and use the CUDA toolkit libraries the :module:`FindCUDAToolkit`
+module has superseded this module.  It works whether or not the ``CUDA``
+language is enabled.
+
 Documentation of Deprecated Usage
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
diff --git a/Modules/FindCUDAToolkit.cmake b/Modules/FindCUDAToolkit.cmake
index 384f734..c3ce38a 100644
--- a/Modules/FindCUDAToolkit.cmake
+++ b/Modules/FindCUDAToolkit.cmake
@@ -15,9 +15,12 @@ Search Behavior
 Finding the CUDA Toolkit requires finding the ``nvcc`` executable, which is
 searched for in the following order:
 
-1. If the ``CUDAToolkit_ROOT`` cmake configuration variable (e.g.,
+1. If the ``CUDA`` language has been enabled we will use the directory
+   containing the compiler as the first search location for ``nvcc``.
+
+2. If the ``CUDAToolkit_ROOT`` cmake configuration variable (e.g.,
    ``-DCUDAToolkit_ROOT=/some/path``) *or* environment variable is defined, it
-   will be searched first.  If both an environment variable **and** a
+   will be searched.  If both an environment variable **and** a
    configuration variable are specified, the *configuration* variable takes
    precedence.
 
@@ -27,16 +30,16 @@ searched for in the following order:
    package is marked as **not** found.  No subsequent search attempts are
    performed.
 
-2. The user's path is searched for ``nvcc`` using :command:`find_program`.  If
+3. The user's path is searched for ``nvcc`` using :command:`find_program`.  If
    this is found, no subsequent search attempts are performed.  Users are
    responsible for ensuring that the first ``nvcc`` to show up in the path is
    the desired path in the event that multiple CUDA Toolkits are installed.
 
-3. On Unix systems, if the symbolic link ``/usr/local/cuda`` exists, this is
+4. On Unix systems, if the symbolic link ``/usr/local/cuda`` exists, this is
    used.  No subsequent search attempts are performed.  No default symbolic link
    location exists for the Windows platform.
 
-4. The platform specific default install locations are searched.  If exactly one
+5. The platform specific default install locations are searched.  If exactly one
    candidate is found, this is used.  The default CUDA Toolkit install locations
    searched are:
 
@@ -88,13 +91,15 @@ Options
 Imported targets
 ^^^^^^^^^^^^^^^^
 
+An :ref:`imported target <Imported targets>` named ``CUDA::toolkit`` is provided.
+
 This module defines :prop_tgt:`IMPORTED` targets for each
 of the following libraries that are part of the CUDAToolkit:
 
-- :ref:`CUDA Runtime Libraries<cuda_toolkit_rt_libs>`
+- :ref:`CUDA Runtime Library<cuda_toolkit_rt_lib>`
+- :ref:`CUDA Driver Library<cuda_toolkit_driver_lib>`
 - :ref:`cuBLAS<cuda_toolkit_cuBLAS>`
 - :ref:`cuFFT<cuda_toolkit_cuFFT>`
-- :ref:`cuLIBOS<cuda_toolkit_cuLIBOS>`
 - :ref:`cuRAND<cuda_toolkit_cuRAND>`
 - :ref:`cuSOLVER<cuda_toolkit_cuSOLVER>`
 - :ref:`cuSPARSE<cuda_toolkit_cuSPARSE>`
@@ -102,15 +107,19 @@ of the following libraries that are part of the CUDAToolkit:
 - :ref:`nvBLAS<cuda_toolkit_nvBLAS>`
 - :ref:`nvGRAPH<cuda_toolkit_nvGRAPH>`
 - :ref:`nvJPEG<cuda_toolkit_nvJPEG>`
+- :ref:`nvidia-ML<cuda_toolkit_nvML>`
+- :ref:`nvRTC<cuda_toolkit_nvRTC>`
 - :ref:`nvToolsExt<cuda_toolkit_nvToolsExt>`
+- :ref:`OpenCL<cuda_toolkit_opencl>`
+- :ref:`cuLIBOS<cuda_toolkit_cuLIBOS>`
 
-.. _`cuda_toolkit_rt_libs`:
+.. _`cuda_toolkit_rt_lib`:
 
-CUDA Runtime Libraries
-""""""""""""""""""""""
+CUDA Runtime Library
+""""""""""""""""""""
 
-The CUDA Runtime libraries (cudart) are what most applications will typically
-need to link against to make any calls such as `cudaMalloc` and `cudaFree`.
+The CUDA Runtime library (cudart) are what most applications will typically
+need to link against to make any calls such as `cudaMalloc`, and `cudaFree`.
 They are an explicit dependency of almost every library.
 
 Targets Created:
@@ -118,6 +127,20 @@ Targets Created:
 - ``CUDA::cudart``
 - ``CUDA::cudart_static``
 
+.. _`cuda_toolkit_driver_lib`:
+
+CUDA Driver Library
+""""""""""""""""""""
+
+The CUDA Driver library (cuda) are used by applications that use calls
+such as `cuMemAlloc`, and `cuMemFree`. This is generally used by advanced
+
+
+Targets Created:
+
+- ``CUDA::cuda_driver``
+- ``CUDA::cuda_driver``
+
 .. _`cuda_toolkit_cuBLAS`:
 
 cuBLAS
@@ -144,24 +167,6 @@ Targets Created:
 - ``CUDA::cufft_static``
 - ``CUDA::cufftw_static``
 
-.. _`cuda_toolkit_cuLIBOS`:
-
-cuLIBOS
-"""""""
-
-The cuLIBOS library is a backend thread abstraction layer library which is
-static only.  The ``CUDA::cublas_static``, ``CUDA::cusparse_static``,
-``CUDA::cufft_static``, ``CUDA::curand_static``, and (when implemented) NPP
-libraries all automatically have this dependency linked.
-
-Target Created:
-
-- ``CUDA::culibos``
-
-**Note**: direct usage of this target by consumers should not be necessary.
-
-.. _`cuda_toolkit_cuRAND`:
-
 cuRAND
 """"""
 
@@ -293,9 +298,10 @@ Targets Created:
 .. _`cuda_toolkit_nvJPEG`:
 
 nvJPEG
-"""""""
+""""""
 
 The `nvJPEG <https://docs.nvidia.com/cuda/nvjpeg/index.html>`_ library.
+Introduced in CUDA 10.
 
 Targets Created:
 
@@ -314,6 +320,20 @@ Targets Created:
 
 - ``CUDA::nvrtc``
 
+.. _`cuda_toolkit_nvml`:
+
+nvidia-ML
+"""""""""
+
+The `NVIDIA Management Library <https://developer.nvidia.com/nvidia-management-library-nvml>`_.
+This is a shared library only.
+
+Targets Created:
+
+- ``CUDA::nvml``
+
+.. _`cuda_toolkit_opencl`:
+
 .. _`cuda_toolkit_nvToolsExt`:
 
 nvToolsExt
@@ -326,6 +346,35 @@ Targets Created:
 
 - ``CUDA::nvToolsExt``
 
+OpenCL
+""""""
+
+The `NVIDIA OpenCL Library <https://developer.nvidia.com/opencl>`_.
+This is a shared library only.
+
+Targets Created:
+
+- ``CUDA::OpenCL``
+
+.. _`cuda_toolkit_cuLIBOS`:
+
+cuLIBOS
+"""""""
+
+The cuLIBOS library is a backend thread abstraction layer library which is
+static only.  The ``CUDA::cublas_static``, ``CUDA::cusparse_static``,
+``CUDA::cufft_static``, ``CUDA::curand_static``, and (when implemented) NPP
+libraries all automatically have this dependency linked.
+
+Target Created:
+
+- ``CUDA::culibos``
+
+**Note**: direct usage of this target by consumers should not be necessary.
+
+.. _`cuda_toolkit_cuRAND`:
+
+
 
 Result variables
 ^^^^^^^^^^^^^^^^
@@ -337,6 +386,15 @@ Result variables
     The exact version of the CUDA Toolkit found (as reported by
     ``nvcc --version``).
 
+``CUDAToolkit_VERSION_MAJOR``
+    The major version of the CUDA Toolkit.
+
+``CUDAToolkit_VERSION_MAJOR``
+    The minor version of the CUDA Toolkit.
+
+``CUDAToolkit_VERSION_PATCH``
+    The patch version of the CUDA Toolkit.
+
 ``CUDAToolkit_BIN_DIR``
     The path to the CUDA Toolkit library directory that contains the CUDA
     executable ``nvcc``.
@@ -350,7 +408,7 @@ Result variables
     Runtime library ``cudart``.
 
 ``CUDAToolkit_NVCC_EXECUTABLE``
-    The path to the NVIDIA CUDA compiler ``nvcc``.  Note that this path may not
+    The path to the NVIDIA CUDA compiler ``nvcc``.  Note that this path may
     **not** be the same as
     :variable:`CMAKE_CUDA_COMPILER <CMAKE_<LANG>_COMPILER>`.  ``nvcc`` must be
     found to determine the CUDA Toolkit version as well as determining other
@@ -396,15 +454,21 @@ Result variables
 #
 ###############################################################################
 
-# Attempt 1: try user provided paths first.
-find_path(CUDAToolkit_BIN_DIR
-  NAMES nvcc nvcc.exe
-  PATHS
-    ${CUDAToolkit_ROOT}
-    ENV CUDAToolkit_ROOT
-  PATH_SUFFIXES bin bin64
-  NO_DEFAULT_PATH
-)
+if(CMAKE_CUDA_COMPILER)
+  get_filename_component(cuda_dir "${CMAKE_CUDA_COMPILER}" DIRECTORY)
+  # use the already detected cuda compiler
+  set(CUDAToolkit_BIN_DIR "${cuda_dir}" CACHE PATH "")
+else()
+  # Try user provided paths first.
+  find_path(CUDAToolkit_BIN_DIR
+    NAMES nvcc nvcc.exe
+    PATHS
+      ${CUDAToolkit_ROOT}
+      ENV CUDAToolkit_ROOT
+    PATH_SUFFIXES bin bin64
+    NO_DEFAULT_PATH
+  )
+endif()
 
 # If the user specified CUDAToolkit_ROOT but nvcc could not be found, this is an error.
 if (NOT CUDAToolkit_BIN_DIR AND (DEFINED CUDAToolkit_ROOT OR DEFINED ENV{CUDAToolkit_ROOT}))
@@ -544,21 +608,27 @@ find_program(CUDAToolkit_NVCC_EXECUTABLE
   PATHS ${CUDAToolkit_BIN_DIR}
   NO_DEFAULT_PATH
 )
-# Compute the version.
-execute_process(
-  COMMAND ${CUDAToolkit_NVCC_EXECUTABLE} "--version"
-  OUTPUT_VARIABLE NVCC_OUT
-)
-string(
-  REGEX REPLACE ".*release ([0-9]+)\\.([0-9]+).*" "\\1"
-  CUDAToolkit_VERSION_MAJOR ${NVCC_OUT}
-)
-string(
-  REGEX REPLACE ".*release ([0-9]+)\\.([0-9]+).*" "\\2"
-  CUDAToolkit_VERSION_MINOR ${NVCC_OUT}
-)
-set(CUDAToolkit_VERSION "${CUDAToolkit_VERSION_MAJOR}.${CUDAToolkit_VERSION_MINOR}")
-unset(NVCC_OUT)
+if(CUDAToolkit_NVCC_EXECUTABLE AND
+   CUDAToolkit_NVCC_EXECUTABLE STREQUAL CMAKE_CUDA_COMPILER)
+  # Need to set these based off the already computed CMAKE_CUDA_COMPILER_VERSION value
+  # This if statement will always match, but is used to provide variables for MATCH 1,2,3...
+  if(CMAKE_CUDA_COMPILER_VERSION MATCHES [=[([0-9]+)\.([0-9]+)\.([0-9]+)]=])
+    set(CUDAToolkit_VERSION_MAJOR "${CMAKE_MATCH_1}")
+    set(CUDAToolkit_VERSION_MINOR "${CMAKE_MATCH_2}")
+    set(CUDAToolkit_VERSION_PATCH "${CMAKE_MATCH_3}")
+    set(CUDAToolkit_VERSION "${CMAKE_CUDA_COMPILER_VERSION}")
+  endif()
+else()
+  # Compute the version by invoking nvcc
+  execute_process (COMMAND ${CUDAToolkit_NVCC_EXECUTABLE} "--version" OUTPUT_VARIABLE NVCC_OUT)
+  if(NVCC_OUT MATCHES [=[ V([0-9]+)\.([0-9]+)\.([0-9]+)]=])
+    set(CUDAToolkit_VERSION_MAJOR "${CMAKE_MATCH_1}")
+    set(CUDAToolkit_VERSION_MINOR "${CMAKE_MATCH_2}")
+    set(CUDAToolkit_VERSION_PATCH "${CMAKE_MATCH_3}")
+    set(CUDAToolkit_VERSION  "${CMAKE_MATCH_1}.${CMAKE_MATCH_2}.${CMAKE_MATCH_3}")
+  endif()
+  unset(NVCC_OUT)
+endif()
 
 
 get_filename_component(CUDAToolkit_ROOT_DIR ${CUDAToolkit_BIN_DIR} DIRECTORY ABSOLUTE)
@@ -599,27 +669,33 @@ find_package_handle_standard_args(CUDAToolkit
 
 #-----------------------------------------------------------------------------
 # Construct result variables
-if(CUDAToolkit)
- set(CUDAToolkit_ROOT_DIR )
+if(CUDAToolkit_FOUND)
  set(CUDAToolkit_INCLUDE_DIRS ${CUDAToolkit_INCLUDE_DIR})
  get_filename_component(CUDAToolkit_LIBRARY_DIR ${CUDA_CUDART} DIRECTORY ABSOLUTE)
 endif()
 
 #-----------------------------------------------------------------------------
 # Construct import targets
-if(CUDAToolkit)
+if(CUDAToolkit_FOUND)
 
   function(find_and_add_cuda_import_lib lib_name)
 
-    string(TOUPPER ${lib_name} LIB_NAME)
-    find_library(CUDA_${LIB_NAME} ${lib_name} PATHS ${CUDAToolkit_LIBRARY_DIR})
+    if(ARGC GREATER 1)
+      set(search_names ${ARGN})
+    else()
+      set(search_names ${lib_name})
+    endif()
+
+    find_library(CUDA_${lib_name}_LIBRARY
+      NAMES ${search_names}
+      PATHS ${CUDAToolkit_LIBRARY_DIR}
+    )
 
-    if (NOT CUDA::${lib_name} AND CUDA_${LIB_NAME})
+    if (NOT CUDA::${lib_name} AND CUDA_${lib_name}_LIBRARY)
       add_library(CUDA::${lib_name} IMPORTED INTERFACE)
-      target_include_directories(CUDA::${lib_name} INTERFACE "${CUDA_INCLUDE_DIRS}")
-      target_link_libraries(CUDA::${lib_name} INTERFACE "${CUDA_${LIB_NAME}}")
+      target_include_directories(CUDA::${lib_name} SYSTEM INTERFACE "${CUDAToolkit_INCLUDE_DIRS}")
+      target_link_libraries(CUDA::${lib_name} INTERFACE "${CUDA_${lib_name}_LIBRARY}")
     endif()
-
   endfunction()
 
   function(add_cuda_link_dependency lib_name)
@@ -628,6 +704,13 @@ if(CUDAToolkit)
     endforeach()
   endfunction()
 
+  add_library(CUDA::toolkit IMPORTED INTERFACE)
+  target_include_directories(CUDA::toolkit SYSTEM INTERFACE "${CUDAToolkit_INCLUDE_DIRS}")
+  target_link_directories(CUDA::toolkit INTERFACE "${CUDAToolkit_LIBRARY_DIR}")
+
+
+  find_and_add_cuda_import_lib(cuda_driver cuda)
+
   find_and_add_cuda_import_lib(cudart)
   find_and_add_cuda_import_lib(cudart_static)
 
@@ -662,11 +745,15 @@ if(CUDAToolkit)
   endforeach()
 
   find_and_add_cuda_import_lib(nvrtc)
-  add_cuda_link_dependency(nvrtc cuda)
+  add_cuda_link_dependency(nvrtc cuda_driver)
+
+  find_and_add_cuda_import_lib(nvml nvidia-ml nvml)
 
   find_and_add_cuda_import_lib(nvToolsExt)
   add_cuda_link_dependency(nvToolsExt cudart)
 
+  find_and_add_cuda_import_lib(OpenCL)
+
   find_and_add_cuda_import_lib(culibos)
   foreach (cuda_lib cublas cufft cusparse curand nvjpeg)
     add_cuda_link_dependency(${cuda_lib}_static culibos)
diff --git a/Tests/Cuda/CMakeLists.txt b/Tests/Cuda/CMakeLists.txt
index 3b6a611..35b9022 100644
--- a/Tests/Cuda/CMakeLists.txt
+++ b/Tests/Cuda/CMakeLists.txt
@@ -9,7 +9,8 @@ ADD_TEST_MACRO(Cuda.MixedStandardLevels3 MixedStandardLevels3)
 ADD_TEST_MACRO(Cuda.MixedStandardLevels4 MixedStandardLevels4)
 ADD_TEST_MACRO(Cuda.MixedStandardLevels5 MixedStandardLevels5)
 ADD_TEST_MACRO(Cuda.NotEnabled CudaNotEnabled)
-ADD_TEST_MACRO(Cuda.ToolkitInclude CudaToolkitInclude)
+ADD_TEST_MACRO(Cuda.Toolkit Toolkit)
+ADD_TEST_MACRO(Cuda.IncludePathNoToolkit IncludePathNoToolkit)
 ADD_TEST_MACRO(Cuda.ProperDeviceLibraries ProperDeviceLibraries)
 ADD_TEST_MACRO(Cuda.ProperLinkFlags ProperLinkFlags)
 ADD_TEST_MACRO(Cuda.WithC CudaWithC)
diff --git a/Tests/Cuda/ToolkitInclude/CMakeLists.txt b/Tests/Cuda/IncludePathNoToolkit/CMakeLists.txt
index f246b54..7be1561 100644
--- a/Tests/Cuda/ToolkitInclude/CMakeLists.txt
+++ b/Tests/Cuda/IncludePathNoToolkit/CMakeLists.txt
@@ -1,11 +1,11 @@
 cmake_minimum_required(VERSION 3.8)
-project (ToolkitInclude CXX CUDA)
+project (IncludePathNoToolkit CXX CUDA)
 
 #Goal for this example:
 # Validate that between the CXX implicit include directories and the
 # CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES directories we can find
 # the cuda runtime headers
 
-add_executable(CudaToolkitInclude main.cpp)
-target_include_directories(CudaToolkitInclude PRIVATE
+add_executable(IncludePathNoToolkit main.cpp)
+target_include_directories(IncludePathNoToolkit PRIVATE
                            ${CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES})
diff --git a/Tests/Cuda/ToolkitInclude/main.cpp b/Tests/Cuda/IncludePathNoToolkit/main.cpp
index c8d5c6b..c8d5c6b 100644
--- a/Tests/Cuda/ToolkitInclude/main.cpp
+++ b/Tests/Cuda/IncludePathNoToolkit/main.cpp
diff --git a/Tests/Cuda/Toolkit/CMakeLists.txt b/Tests/Cuda/Toolkit/CMakeLists.txt
new file mode 100644
index 0000000..45b700b
--- /dev/null
+++ b/Tests/Cuda/Toolkit/CMakeLists.txt
@@ -0,0 +1,45 @@
+cmake_minimum_required(VERSION 3.15)
+project(Toolkit CXX)
+
+#Goal for this example:
+# Validate that we can use CUDAToolkit to find cuda include paths
+find_package(CUDAToolkit REQUIRED)
+
+message(STATUS "CUDAToolkit_VERSION: ${CUDAToolkit_VERSION}")
+message(STATUS "CUDAToolkit_VERSION_MAJOR: ${CUDAToolkit_VERSION_MAJOR}")
+message(STATUS "CUDAToolkit_VERSION_MINOR: ${CUDAToolkit_VERSION_MINOR}")
+message(STATUS "CUDAToolkit_VERSION_PATCH: ${CUDAToolkit_VERSION_PATCH}")
+message(STATUS "CUDAToolkit_BIN_DIR: ${CUDAToolkit_BIN_DIR}")
+message(STATUS "CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}")
+message(STATUS "CUDAToolkit_LIBRARY_DIR: ${CUDAToolkit_LIBRARY_DIR}")
+message(STATUS "CUDAToolkit_NVCC_EXECUTABLE ${CUDAToolkit_NVCC_EXECUTABLE}")
+
+# Verify that all the CUDA:: targets exist even when the CUDA language isn't enabled
+
+foreach (cuda_lib cudart cuda_driver cublas cufft cufftw curand cusolver cusparse nvgraph)
+  if(NOT TARGET CUDA::${cuda_lib})
+    message(FATAL_ERROR "The CUDA::${cuda_lib} target was expected but couldn't be found")
+  endif()
+endforeach()
+
+foreach (cuda_lib nppc nppial nppicc nppidei nppif nppig nppim nppist nppitc npps nppicom nppisu)
+  if(NOT TARGET CUDA::${cuda_lib})
+    message(FATAL_ERROR "The CUDA::${cuda_lib} target was expected but couldn't be found")
+  endif()
+endforeach()
+
+foreach (cuda_lib nvrtc nvToolsExt OpenCL culibos)
+  if(NOT TARGET CUDA::${cuda_lib})
+    message(FATAL_ERROR "The CUDA::${cuda_lib} target was expected but couldn't be found")
+  endif()
+endforeach()
+
+#libraries added CUDA 10
+if(CUDAToolkit_VERSION_MAJOR VERSION_GREATER 9)
+  if(NOT TARGET CUDA::nvjpeg)
+    message(FATAL_ERROR "The CUDA::nvjpeg target was expected but couldn't be found")
+  endif()
+endif()
+
+add_executable(Toolkit main.cpp)
+target_link_libraries(Toolkit PRIVATE CUDA::toolkit)
diff --git a/Tests/Cuda/Toolkit/main.cpp b/Tests/Cuda/Toolkit/main.cpp
new file mode 100644
index 0000000..c8d5c6b
--- /dev/null
+++ b/Tests/Cuda/Toolkit/main.cpp
@@ -0,0 +1,8 @@
+// Only thing we care about is that these headers are found
+#include <cuda.h>
+#include <cuda_runtime_api.h>
+
+int main()
+{
+  return 0;
+}
diff --git a/Tests/CudaOnly/CMakeLists.txt b/Tests/CudaOnly/CMakeLists.txt
index 0dd3309..a0575cd 100644
--- a/Tests/CudaOnly/CMakeLists.txt
+++ b/Tests/CudaOnly/CMakeLists.txt
@@ -6,6 +6,7 @@ ADD_TEST_MACRO(CudaOnly.GPUDebugFlag CudaOnlyGPUDebugFlag)
 ADD_TEST_MACRO(CudaOnly.ResolveDeviceSymbols CudaOnlyResolveDeviceSymbols)
 ADD_TEST_MACRO(CudaOnly.SeparateCompilation CudaOnlySeparateCompilation)
 ADD_TEST_MACRO(CudaOnly.Standard98 CudaOnlyStandard98)
+ADD_TEST_MACRO(CudaOnly.Toolkit CudaOnlyToolkit)
 ADD_TEST_MACRO(CudaOnly.WithDefs CudaOnlyWithDefs)
 
 add_test(NAME CudaOnly.DontResolveDeviceSymbols COMMAND
diff --git a/Tests/CudaOnly/Toolkit/CMakeLists.txt b/Tests/CudaOnly/Toolkit/CMakeLists.txt
new file mode 100644
index 0000000..682e56e
--- /dev/null
+++ b/Tests/CudaOnly/Toolkit/CMakeLists.txt
@@ -0,0 +1,54 @@
+cmake_minimum_required(VERSION 3.15)
+project(CudaOnlyToolkit CUDA)
+find_package(CUDAToolkit REQUIRED)
+
+message(STATUS "CUDAToolkit_VERSION: ${CUDAToolkit_VERSION}")
+message(STATUS "CUDAToolkit_VERSION_MAJOR: ${CUDAToolkit_VERSION_MAJOR}")
+message(STATUS "CUDAToolkit_VERSION_MINOR: ${CUDAToolkit_VERSION_MINOR}")
+message(STATUS "CUDAToolkit_VERSION_PATCH: ${CUDAToolkit_VERSION_PATCH}")
+message(STATUS "CUDAToolkit_BIN_DIR: ${CUDAToolkit_BIN_DIR}")
+message(STATUS "CUDAToolkit_INCLUDE_DIRS: ${CUDAToolkit_INCLUDE_DIRS}")
+message(STATUS "CUDAToolkit_LIBRARY_DIR: ${CUDAToolkit_LIBRARY_DIR}")
+message(STATUS "CUDAToolkit_NVCC_EXECUTABLE ${CUDAToolkit_NVCC_EXECUTABLE}")
+
+# Verify that all the CUDA:: targets and variables exist
+foreach (cuda_lib cudart cuda_driver cublas cufft cufftw curand cusolver cusparse nvgraph)
+  if(NOT CUDA_${cuda_lib}_LIBRARY)
+    message(FATAL_ERROR "expected CUDAToolkit variable CUDA_${cuda_lib}_LIBRARY not found")
+  endif()
+  if(NOT TARGET CUDA::${cuda_lib})
+    message(FATAL_ERROR "expected CUDAToolkit target CUDA::${cuda_lib} not found")
+  endif()
+endforeach()
+
+foreach (cuda_lib nppc nppial nppicc nppidei nppif nppig nppim nppist nppitc npps nppicom nppisu)
+  if(NOT CUDA_${cuda_lib}_LIBRARY)
+    message(FATAL_ERROR "expected CUDAToolkit variable CUDA_${cuda_lib}_LIBRARY not found")
+  endif()
+  if(NOT TARGET CUDA::${cuda_lib})
+    message(FATAL_ERROR "expected CUDAToolkit target CUDA::${cuda_lib} not found")
+  endif()
+endforeach()
+
+foreach (cuda_lib nvrtc nvToolsExt OpenCL culibos)
+  if(NOT CUDA_${cuda_lib}_LIBRARY)
+    message(FATAL_ERROR "expected CUDAToolkit variable CUDA_${cuda_lib}_LIBRARY not found")
+  endif()
+
+  if(NOT TARGET CUDA::${cuda_lib})
+    message(FATAL_ERROR "expected CUDAToolkit target CUDA::${cuda_lib} not found")
+  endif()
+endforeach()
+
+#libraries added as CUDA 10
+if(CUDAToolkit_VERSION_MAJOR VERSION_GREATER 9)
+  if(NOT CUDA_nvjpeg_LIBRARY)
+    message(FATAL_ERROR "expected CUDAToolkit variable CUDA_nvjpeg_LIBRARY not found")
+  endif()
+  if(NOT TARGET CUDA::nvjpeg)
+    message(FATAL_ERROR "The CUDA::nvjpeg target was expected but couldn't be found")
+  endif()
+endif()
+
+add_executable(CudaOnlyToolkit main.cu)
+target_link_libraries(CudaOnlyToolkit PRIVATE CUDA::toolkit)
diff --git a/Tests/CudaOnly/Toolkit/main.cu b/Tests/CudaOnly/Toolkit/main.cu
new file mode 100644
index 0000000..0f3ccdc
--- /dev/null
+++ b/Tests/CudaOnly/Toolkit/main.cu
@@ -0,0 +1,8 @@
+// Only thing we care about is that these headers are found
+#include <cuda.h>
+#include <cuda_runtime_api.h>
+
+int main(int argc, char** argv)
+{
+  return 0;
+}