CMake doesn't have a package manager built into it, but does include support for manually installing packages and consuming them with the find_package command.
The idiom is that write something like “find_package(boost)” and after that point have access to modern CMake targets (such as libraries and other entities) sitting in an documented namespace, such as “boost”.
In a more traditional language, that would look like this:
Boost = find_package(Boost) # Not real CMake code
...
# Add a dependencies for the Boost header libraries to the library I'm building
target_link_libraries(my_library PUBLIC Boost::boost)
Unfortunately, the CMake language allows functions to create variables in the scope of the caller, which is what find_package does:
but instead it's like this:
find_package(Boost REQUIRED)
# Read the docs and know that a varable named `Boost` was just
# magically brought into scope by the find_package call above
target_link_libraries(my_library PUBLIC Boost::boost)
Ewww.
The second problem is that the exact variables created by find_package can be different for every single package. For example, to figure out what imported targets find_package(BOOST) brings into scope you can look at the Cmake docs. But what about for other libraries? Everyone is free to create package configuration files that do whatever they want. In fact, many find_package files don't even bring the modern imported, namespaced targets into parent scope, but instead bring global variables into the root namespace.
In short, there's no standard way of knowing how find_package will behave.
This problem is compounded due a third issue: many popular libraries don't have official CMake find_package support. Instead, multiple third parties may have written unofficial CMake packages and there's no clear way to choose which ones you will use.
Let's take the SDL2 libraries. If you want to use SDL2 with CMake, you could use the Bincrafters packages for Conan, or tcbrindle's CMake scripts, or the CGet compatable CMake scripts I wrote, or the SDL2 CGet recipe Paul Fultz wrote, etc etc.
All of these may 1. use different names to specify with find_package (ie I use find_package(sdl2) but some people prefer screaming case ala find_package(SDL2)) and 2. will bring in completely different variables.
So while find_package may be a standard idiom for “clean” CMake code, the moment we use one of these SDL2 packages we make our CMake script incompatable with any CMake code that uses a a different SDL2 package. We also make it hard to change the SDL2 package we use in the future.
Writing Package-Portable CMake Code
What we want to do is break the dependency between our CMake scripts and the exact package we're using.
In my case, I want to be able to use the SDL2 libraries from two sources.
One is the Download SDL2 GitHub project, which I wrote for consumption with CGet. You can install this with CMake using
git clone https://github.com/TimSimpson/download-sdl2.git
cd download-sdl2
mkdir build && cd build
cmake -H../ -B./
cmake --build . --target install
This project installs the SDL2 libraries in a standard way to /usr/local/include and /usr/local/lib.
If you don't want to pollute /usr/local you can use the excellent Cget to create a seperate C env, or prefix path for your project and install SDL2 there, like so:
cget install TimSimpson/download-sdl2
Either way, your CMake code can from then out on consume the SDL2 libraries by calling find_package(sdl2) which will bring the following variables into scope:
- sdl2::sdl2 - The main SDL2 library.
- sdl2::image - The main SDL2 image library.
- sdl2::ttf - The main SDL2 TTF library.
I love using standard CMake package installs with Cget. I've been using it for about a year and found it to be sturdy, dependable, and intuitive.
But lately, I've been curious about using the Conan C++ package manager, mostly to speed up CI builds by downloading cached binaries. Currently my Travis builds need to build and install the SDL2 libraries before they build whichever project uses them, and this takes a long time. While I think it's good practice to build all the dependencies for any applications you plan on distributing from scratch, for simple CI builds I sometimes envy Conan's ability to function as a binary package store.
There's also a ton of useful packages available for Conan, mostly written by the BinCrafters team.
What I'd like to do is make my CMake projects which use the Download SDL2 GitHub project work with the BinCrafter's SDL2 package as well.
Conan is a strange beast. It advertises itself as build-system agnositc, and in a certain sense that's true. But the flipside is that it's extremely difficult to make your build-system Conan agnostic. It's very difficult to use Conan without tying your build process- and the build process of anyone consuming your work- to Conan.
Conan uses different “generators” to create build system specific glue code. There's actually four different generators for CMake, which lead to different results with find_package.
I've tried all four CMake generators but will only go over the two which are still in my memory:
The CMake paths generator creates a tool chain file. The appeal here is that you invoke CMake and give it this toolchain file, and it makes all the Conan stuff work in your CMake code which can then be free of Conan specific functions or features. In theory this means you could write a CMake file that's compatable with standard CMake but then use Conan to fetch your packages.
However, since Conan generates the CMake glue this means the variables follow a somewhat stange automated format. find_package(SDL2) brings in SDL2::SDL2, while find_package(SDL2_Image) brings in SDL2_Image::SDL2_Image, etc.
The other approach is the plainly named CMake generator. This creates a .cmake file in your build directory that you have to include from your normal CMake file, tying your build to Conan. There's actually two variations on how this works as well:
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()
This approach pulls in all of the stuff brought in by Conan (libraries, include directories, etc) and stores it in the global CMake variables. In other words it becomes unnecessary to use find_package at all, but the downside is the global path variables used for includes and libraries are all populated for every target in your CMake file, whether you wanted to specify a dependency on everything you brought in with Conan or not.
The second approach is this:
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup(TARGETS)
This instead pulls in more modern looking CMake targets. However, these targets are different than the ones created with the CMake paths generator I discussed above.
First, all of the variables from all targets are put into scope right away, without the need to call find_package. Second, everything is put into a CONAN_PKG namespace.
In this case, the SDL2 library is found in CONAN_PKG::sdl2, the SDL2 Image library is found in CONAN_PKG::sdl2_image, etc.
So Conan can't even keep the CMake targets it creates the same between the various generators it creates.
What we need to do is find a way to make a typical “clean” CMake script- the kind that would work without Conan- use the CMake variables Conan's glue script brings up, without changing the “clean” script so it forces us to always use Conan.
The Integrated Build Problem
There's a similar problem at play with integrated builds.
An integrated build is when you have several CMake projects that define packages and can be installed, but rather than constantly installing them to a C env you'd like to generate a single build script where they simply depend on each other.
Put this way: if you have project A that makes library A, and project B that uses library A by calling find_package(A), you sometimes would like to crate a parenty CMake script which calls add_subdirectory for projects A and B and then makes B simply depend on the library create by project A.
The problem is, when project B calls find_package it will look for an installed library A instead of the one being built in the same super project.
You could just run the install target for project A, but this takes extra time and is easy to forget about.
A simpler way is to just ensure that the variables brought into scope by find_package(A) will be the same ones brought into scope by calling add_subdirectory on A's directory.
We can do this by creating a macro for find_package that overwrites CMake's find_package functionality to make it ignore certain packages we don't want it to mess with- in this case A:
It looks like this:
# Assume this CMakeLists.txt file will never be consumed via `add_subdirectory`
# and lives at the root of a directory which have projects A, B, and C
# living in it via git submodules or something similar.
set(subprojects A B C)
# Make a macro for `find_package`. CMake will automatically save the old
# find_package as `_find_package`.
macro(find_package )
if(NOT ${ARGV0} IN_LIST subprojects) # Ignore packages A, B, and C
_find_package(${ARGV}) # run find_package for everything else
endif()
endmacro()
# Assume that the CMake projects for A, B, and C are in the current directory
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/A)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/B)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/C)
Creating a macro for find_package is a dirty trick, but that's ok if the top level CMakeLists.txt file isn't expected to be portable. The CMake code for projects A, B and C are still portable and can be consumed for other use cases.
Achieving package portability
Going back to the issue of consuming the SDL2 libraries from either my own, cget-compatable package or Bincrafter's Conan packages, we can use the same find_package trick to make the Bincrafter packages resemble the ones I created.
This will again require a new top-level CMake project. Since Conan users don't care about reusing CMake files, and only want to consume the binary artifacts, we can just make a top level CMake file that only works for Conan. In my case I'll just make a directory named conan and put it there.
Because this is a top level CMakeLists.txt file that will include “clean” CMakeLists.txt files, we can feel free to resort to some hacks. In particular, I'm going to use Conan's plain cmake generator I mentioned above which requires calling a Conan specific function to introduce global targets. The result looks like this:
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup(TARGETS)
While this does bring a lot of variables into scope, they're all in the namespace CONAN_PKG so it shouldn't interfere with the clean CMake code that will be included later.
Next we have a problem with header only libraries. In typical CMake usage, these libraries are installed to a system or user directory (such as /usr/local/include) where they become available globally. But with Conan, the headers are only available if you specify the modern targets.
A good example is the GSL libraries. If you install them using CMake, they get plopped right into the /usr/local/include or wherever your include directory is on your current prefix path. There's no need to specify anything about them from within CMake since any source file can just include them. But if you use Conan, any library that depends on them must have a dependency on CONAN_PKG::gsl_micrsoft since Conan stores the include files for every package in a unique directory.
In some ways that's cleaner, but it's also not the way the built in CMake support for the GSL operates. To make the Conan GSL package act like normal, we'll just take its include directories and add it to the global include directories list:
function(add_header_library SRC)
get_property(var TARGET ${SRC} PROPERTY INTERFACE_INCLUDE_DIRECTORIES)
include_directories(${var})
endfunction()
# look at the generated file `conanbuildinfo.cmake` to find the names of the
# targets created for each Conan package.
add_header_library(CONAN_PKG::gsl_microsoft)
Next up we have actual binary libraries, like the SDL2.
Conan already gives us imported targets (“CONAN_PKG::sdl2”, “CONAN_PKG:sdl2_image” etc). We just want these to have a slightly different name (“sdl2::sdl2”, “sdl2::image”, etc).
Ideally there'd be a way in CMake to create an “empty” library target which just bundled dependencies. For example, we could create an alias library called “sdl2::sdl2” and then add dependencies to the CONAN_PKG targets.
Unfortunately that doesn't work. The cleanest approach I've found is to create INTERFACE IMPORTED libraries and then extract all of the relevant properties from the CONAN_PKG targets and copy them onto the new targets manually. Here's a function that seems to clone all of the necessary properties:
function(clone_library DST SRC)
add_library(${DST} INTERFACE IMPORTED)
get_property(var TARGET ${SRC} PROPERTY INTERFACE_LINK_LIBRARIES)
set_property(TARGET ${DST} PROPERTY INTERFACE_LINK_LIBRARIES ${var})
get_property(var TARGET ${SRC} PROPERTY INTERFACE_INCLUDE_DIRECTORIES)
set_property(TARGET ${DST} PROPERTY INTERFACE_INCLUDE_DIRECTORIES ${var})
get_property(var TARGET ${SRC} PROPERTY INTERFACE_COMPILE_DEFINITIONS)
set_property(TARGET ${DST} PROPERTY INTERFACE_COMPILE_DEFINITIONS ${var})
get_property(var TARGET ${SRC} PROPERTY INTERFACE_COMPILE_OPTIONS)
set_property(TARGET ${DST} PROPERTY INTERFACE_COMPILE_OPTIONS ${var})
endfunction()
We then create the necessary libraries in our find_package macro when the argument expects a given package:
macro(find_package )
if ("${ARGV0}" STREQUAL "sdl2")
clone_library(sdl2::sdl2 CONAN_PKG::sdl2)
clone_library(sdl2::image CONAN_PKG::sdl2_image)
clone_library(sdl2::ttf CONAN_PKG::sdl2_ttf)
elseif(NOT ${ARGV0} IN_LIST subprojects)
# Ignore subprojects, run _find_package for everything else
_find_package(${ARGV})
endif()
endmacro()
Finally, we include the “clean” CMake file using add_subdirectory. If this is for a typical CMake project where the clean CMakeLists.txt file lives in the root directory, then conan/CMakeLists.txt will need to go up a directory to reference it like so:
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/..
${CMAKE_CURRENT_BINARY_DIR}/output)
Voilà! The Cmake project now works with both package sources.
Here is the final result.
