Conan creates CMake targets inconsistent with everything, including Conan itself
Saturday June 13, 2020 09:32:55

Update: Things in Conan are improving beyond what I document here. I've added an update to the end of this post describing how.

Conan provides multiple “generators” to use for integrating with CMake. Two of them - cmake_paths and cmake_find_package- promise to allow the use of Conan from CMake “unintrusively”, which presumably means if you wanted, you could kick Conan to the curb and still use your CMake scripts with other package managers.

cmake_paths creates a CMake tool chain file which lets your CMake scripts consume the package config files created as part of Conan packaged CMake projects (the thing which the mainstream Conan package sources don't support adding) while “cmake_find_package” creates find modules for every dependency you need.

One major issue: the CMake targets created by using the “find modules” are different from the ones you get from the package config files. This means the generator you use when running Conan will affect the CMake code you have to write.

The BinCrafters SDL 2.0.9 package, used with cmake_find_package, will create a file named Findsdl2.cmake. Your CMake code then needs to call find_package(sdl2) and use the targets sdl2::sdl2. But if you use the cmake_paths generator to get the package files instead you'll need to call find_package(SDL2) and use the targets SDL2::SDL2-static (and, in some cases, SDL2::SDL2).

For other Conan packages, such as BinCrafter's SDL 2 Image, cmake_paths simply doesn't work. There are no CMake package files included with the Conan package, so using find_package simply won't work.

However, it's possible to combine the cmake_paths and cmake_find_package generators like Voltron so you get both the package config files and also the find modules at the same time. If you do this, you can select which version is loaded in CMake by using either the CONFIG (cmake_paths) or MODULE (cmake_find_package) argument when you call find_package.

So, all is well! You get the great, standard CMake targets you crave along with the ability to use a package manager that seems to have near infinite reach and ability to conceptualize a task such as “install Strawberry Perl and then use it to build a single yet critical dependency, but only on Windows”.

Well, not quite.

I created a library called lp3-sdl which used BinCrafter's SDL 2.0.9 and had a CMake script capable of installing a real CMake package config file, but was then turned into a real Conan package as part of CI. I then made a test project in Conan which pulled down lp3-sdl as a dependency and tried to build something simple to make sure that the Conan / CMake packaging process had worked.

What I found was in Windows I got CMake errors in the test project because the CMake target sdl2::sdl2 was not found. After spending hours debugging I finally realized what had happened.

In the test project's CMake file I loaded the lp3-sdl dependency by calling find_package(Lp3_Sdl CONFIG REQUIRED). I specified CONFIG to make sure it would use the CMake package config file I'd hand-crafted and stored inside the Conan package instead of the Conan generated find module files. The package config CMake script then called find_dependency(SDL2) and made the main lp3-sdl library dependent on the target SDL2::SDL2-static / SDL2::SDL2.

However, the test project also had a dependency on BinCrafter's SDL 2 Image package, and because of that _had_ to use the cmake_find_package generator. This meant it got find modules for every dependency Conan knew the test project needed, including transitive dependencies such as SDL2.

Because 1. Findsdl2.cmake was sitting in the build directory, 2. CMake by default uses find modules if it can find them instead of the package config files and 3. Windows has a case insensitive file system, find_dependency(sdl2) in lp3-sdl's package config script caused Conan's cmake_find_package style target, ie sdl2::sdl2, to be introduced instead of SDL2::SDL2-static / SDL2::SDL2, which the package config script still attached as a part of the “INTERFACE_LINK_LIBRARIES” to the main library it exported.

Finally the test project created an executable which depended on lp3-sdl's library which depended on SDL2:: style targets that weren't there. At this point CMake showed me an error stating that it could not find SDL2::SDL2-static when linking the test project executable, which was confusing because by that point I was expecting all of the find_package shannannigans to be finished.

So to summarize: Conan creates CMake targets which are not only inconsistent with CMake's built-in Find Modules (which are supported by at least some C++ package managers) but with alternate runs of Conan itself!

I sort of kind of worked around it with one more hack. In lp3-sdl's CMake file, I now determine which target I need to depend on, and save it as a variable:

if (TARGET SDL2::SDL2)
      set(SDL2_TARGET_NAME "SDL2::SDL2")
      target_link_libraries(lp3_sdl PUBLIC SDL2::SDL2)
elseif(TARGET SDL2::SDL2-static)
    set(SDL2_TARGET_NAME "SDL2::SDL2-static")
    target_link_libraries(lp3_sdl PUBLIC SDL2::SDL2-static)
else()
    message(FATAL_ERROR "Could not find target SDL2::SDL2 or SDL2::SDL2-static!")
endif()

Later, when I'm writing the package config file, I add a ton of gross code after find_dependency(SDL2) to, if necessary, create SDL2::SDL2-static (or SDL2::SDL2) as an alias to sdl2::sdl2, which Conan has created:

file(
    WRITE "${PROJECT_BINARY_DIR}/Lp3_SdlConfig.cmake"
    "
include(CMakeFindDependencyMacro)
if (NOT \"\${CMAKE_SYSTEM_NAME}\" MATCHES \"Emscripten\")
    # On Windows, Conan generated Findsdl2.cmake may get used here instead of
    # the one we want. If that's happened, we can tell because the targets
    # we need will be missing. So make aliases of the Conan targets instead.
    find_dependency(SDL2)
    if (NOT TARGET ${SDL2_TARGET_NAME} AND TARGET \"sdl2::sdl2\")
        add_library(${SDL2_TARGET_NAME} INTERFACE IMPORTED)
        set_target_properties(${SDL2_TARGET_NAME} PROPERTIES INTERFACE_LINK_LIBRARIES \"sdl2::sdl2\")
    elseif (NOT TARGET ${SDL2_TARGET_NAME} AND TARGET \"SDL2::SDL2\")
        add_library(${SDL2_TARGET_NAME} INTERFACE IMPORTED)
        set_target_properties(${SDL2_TARGET_NAME} PROPERTIES INTERFACE_LINK_LIBRARIES \"SDL2::SDL2\")
    endif()
endif()
include(\"\${CMAKE_CURRENT_LIST_DIR}/Lp3_Sdl-targets.cmake\")
")

So, there you have it.

Is this a solution?

Of course it isn't! At this point I have no faith at all that what I'm doing will scale in the future and won't break on some similarly hard to detect issue.

This came out of a three day weekend where I spent hours working around endless problems exactly like this one. Many of the fixes I tried out ended up causing new problems, and often the only way I could ever know for sure if everything was working or if I even had the right conceptual model for what was going on was to spend repeatedly build several projects back to back in one of the slowest feedback loops possible (even after I wrote a series of scripts to automate all the calls to Conan and CMake) or to create entirely new standalone “recreate” projects in Conan and CMake.

So much of this wasted effort is because the Conan community doesn't believe adherring to a standard or at least consistent target naming policy is worth doing, even as they've advertised using Conan with CMake “unintrusively” in the past.

Meanwhile, BinCrafters states very directly that the use of find_package “should be avoided” when consuming their packages. With all due respect to the hard working people behind BinCrafters, I'm going to lay this out there: saying “should be” is nonsense. They need to explicitly state something like “we don't support using find_package. It will not work even if you sometimes think it does. So if you're using our packages don't even think of using CMake like that.”

If find_package is considered a no-no for a player as big as BinCrafters, and Conan Center itself is deleting the package config files created by Boost itself, then who is the cmake_paths generator for? I'm guessing people like Mathieu Ropert are using it in situations where they build everything from scratch themselves.

For others though, be wary: just because the Conan program itself offers flexibility to build software any crazy way you want it doesn't mean the plethora of publicly available packages for Conan are going to support it.

Update: James from Conan.io let me know that Conan has added methods to use the canonical CMake targets when Conan itself is generating CMake scripts, so it's possible to use those names even when the CMake package config files are not being used. He went onto say achieving parity with the canonical CMake target names is a high priority, and work is underway to update all packages in Conan Center to generate the canonical CMake targets when possible.

That's great news. It's still worth keeping in mine though that the issues described above will happen if projects mix CMake package config files with Conan generated find modules while using Conan packages that don't yet utilize these new features.

One last tangent that's worth mentioning: above, you may have noticed I was using SDL 2.0.9. Ironically, the BinCrafter's package for SDL 2.0.12 appears to support the generating the CMake target with the name “SDL2”. I say “appears” to because I hit an issue trying to update my project to use it: it wanted to build about a dozen dependencies, including things I already have on my machine via apt, like pulseaudio, and these would fail to compile for some reason or another, so eventually I gave up / ran out of weekend time. The fact that merely trying to use the SDL2 libraries will sometimes makes me feel like a Gentoo Linux user is a topic for another day.



The trouble with Conan and CMake
Sunday June 7, 2020 09:32:55

Conan claims to work well with CMake and find_package, but if you spend any time trying to get well known dependencies such as Zlib or SDL2 to work with it you'll quickly find calling find_package when Conan packages are available produces different results than calling it when those packages are installed through older or more “standard” means.

Quick recap: using CMake's find_package command causes CMake to load information on an external package and pulls targets (think: namespaced global variables) into scope. Unfortunately the naming schemes that Conan gives these targets wind up being different than the ones CMake chooses.

For example, here's how find_package might look for the Boost libraries and Zlib if both dependencies were installed locally, using a mechanism such as apt-get install for Debian linux:

find_package(boost)
find_package(ZLIB)
...
target_link_libraries(my_lib PUBLIC boost::headers ZLIB::ZLIB)

There is no intuitive way to know what targets find_package pulls in, as they differ for every package. However, CMake has built in support over a hundred packages and you can find details of how they behave by looking at the docs.

Unfortunately Conan - which offers access to hundreds of packages CMake doesn't have native support for - does not play nice with the built in naming schemes of Cmake. For example, if you're using the Conan packages for Zlib, you have to do this:

find_package(zlib)
target_link_libraries(my_lib PUBLIC zlib::zlib)

It should be possible, in theory, for Conan to support the naming schemes CMake gives for every package it has built in support for. The mainainers of each Conan dependency (such as zlib) would just need to create CMake configuration files, which would get installed with the rest of the package's data. In fact, some present this as the best practice to follow if you're creating your own Conan packages (I happen to agree).

However, the thought leaders behind the biggest Conan packaging repositories explicitly choose not to do this. They even go so far as to delete the CMake configuration files created by the authors of the software they are packaging for use by Conan.

I'm positive that the people working full time on Conan packages have probably thought over their reasoning here more than I possibly could have, but still: this really, really sucks. It makes a lie out of the promise that you can use Conan while keeping your CMake scripts exactly the same as they would have been, and hurts people's ability to create and distribute CMake packages that will also work with tools such as cget and vcpkg.



Finding Masks that Don't Suck
Sunday May 17, 2020 12:00:00

Scientists are saying if 80% of people wore a mask when outside the home we could stop the spread of Covid 19, but despite this the majority of Americans aren't doing it.

Most of the negative feedback I've heard people give regarding masks is that wearing them makes them feel funny or embarassed. It reminds me of this amazing Pete Holmes sketch where Professor X fires Cyclops because instead of bolting some kind of visor to his face to keep his laser eyes from murdering everyone in front of him, he instead props up a pair of casual sunglasses to keep from “looking like a dork.”

However, a second, more valid reason I've seen people give masks grief is that they're uncomfortable. If you feel that way I'd recommend spending a bit of money to buy a decent one.

A month or so back when the CDC was like “whoopsies, turns out masks are good everyone please wear one now” I bought a dozen of them from Custom Ink for $30. This was back when people were like “just make one with an old sock and some bubble gum, it's so easy!” so I figured how bad could 12 masks be even if they were $30?

The answer terrible. These masks scrunched up to the height of a pencil when not in use, and to expand we had to roll them back and forth over our face (you know, the area you'renever supposed to touch?). Afterwards the masks clung to our mouths, making breathing through them almost impossible. Additionally because they were basically just ripped up t-shirts with holes sloppily cut in them for ears the fit was never tight, and they'd inevitably slip off my nose (you ever see idiots walking around with masks that aren't covering their noses? That's how that happens).

The masks were so bad as to be unusable, and we actually threw them in the trash after a week.

While I was waiting for the crappy masks to arrive, a buddy of mine commended me on saving money as his wife had bought fancy “expensive” masks that costs $15 each. Ironically after I learned Custom Ink had used the pandemic to sell my family garbage I ended up rushing to buy several of these “boutique” masks.

They're from a place called Malu Organic, so if you're the kind of guy that nervously observes your pecs for days after you accidentally sip a soy latte that might bother you. But they're as comfortable as a mask can be.

Most Saturdays I wear mine for hours while I do lawn work outside. Some people say masks make you look threatening, but I've never had any of my neighbors say anything at all to me when I wear mine:

There's also a pocket so you can put a filter inside. I stick a coffee filter in there ala Matthew McConaughey, which is a fun superstitious thing to do which probably doesn't add any protection at all.

Anyway, I recommend spending a few extra bucks and buying some masks from Malu Organic if what you're dealing with now sucks.



Restoring Launchy to its former Glory in Lubuntu
Sunday December 23, 2018 09:32:55

When Windows 8 came out years back and was panned by critics for taking away the start menu and thus making it too hard to launch apps, all I could think was “there are power users out there not using a keyboard based app launcher?”

For years, I'd been using Launchy, a cross-platform app launcher that lets me open up apps by hitting “Alt+Space” and typing the name of the app you want to open. It's very similar to OSX's SpotLight feature. This is far more efficient than searching the labyrinth that is the start menu or any other GUI based catalog (especially on Windows XP).

It seems in recent years my friend Launchy has lost a step. On Windows I often have better luck finding apps using Microsoft's built in keystroke launcher (which is similar to Launchy but takes longer to appear and vanish).

In Lubuntu the situation was worse. The built-in calculator stopped working a long time ago, but more importantly the majority of my installed apps simply didn't appear in Launchy.

I'm ashamed to admit that for a few years now I've simply dealt with this subpar experience by using a mouse and clicking through the various menus to find stuff (don't judge me. I already hate myself for it!).

It hit me recently that this had become a big problem and was probably altering my behavior by making me avoid opening apps I was less familiar with, so I spent some time today figuring out a fix. To my surprise Googling was almost useless; not many people seem to talk about Launchy much anymore, let alone the intersection of its issues and Lubuntu. Anyway I've written my fix below in hopes it will help some future googler.

Fixing Launchy

Launchy keeps a catalog of apps, which it creates by searching well known directories for .desktop files. These simple text files are what Lubuntu uses to display shortcuts in various places.

Turns out Launchy will silently ignores many of these files if they get too complicated.

Thankfully it's easy to create simplified .desktop files Launchy can deal with.

Create a dedicated directory for Launchy

First, create a dedicated directory for all of the Launchy desktop files you're about to create. For example, I have a simple directory for all the scripts I use from my terminal which I back up to source control. In that directory, I've create a new directory named after each machine which is dedicated to Launchy. This allows me to view all the apps I think I should be able to open with Launchy at a glance.

Open up Launchy, and click on the gear icon to bring up the settings window.

Next, click on the tab labeled “Catalog”.

Here you'll see a long list of directories. Take note of these.

Now click the minus sign to delete all of them. Then click the plus sign to add the one directory you've created just for Launchy.

Next, under “File Types”, make sure “desktop” Add *.desktop to the “File Types” list if it isn't there already by using the plus and minus buttons.

Copy in any .desktop files you may need

Next, you'll want to copy (not symlink) any .desktop files you'd like Launchy to use into the empty directory you created above. It may have already worked with a few of these in the past, while others will need to be modified.

You can figure out where these desktop files live by right clicking their icons in the Lubuntu start menu thing, then clicking “Properties”. This will bring up a window titled “File Properties” which will show the location of the desktop file in “Target file”. These will live in places like /usr/share/applications, /var/lib/snapd/desktop/applications, etc.

Modify the .desktop files to be as simple as possible

Now that you have a directory full of .desktop files you can start testing Launchy to see which ones already work.

For each desktop file, open Launchy, right click on it and select “Rebuild catalog”. Then type the name of the app and see if it starts up.

If it does, great.

If not, edit the file to be simpler.

For an example of what a “simple” desktop file looks like, here is what my desktop entry for Sublime Text looked like originally when it didn't work:

[Desktop Entry]
Version=1.0
Type=Application
Name=Sublime Text
GenericName=Text Editor
Comment=Sophisticated text editor for code, markup and prose
Exec=/opt/sublime_text/sublime_text %F
Terminal=false
MimeType=text/plain;
Icon=sublime-text
Categories=TextEditor;Development;
StartupNotify=true
Actions=Window;Document;

[Desktop Action Window]
Name=New Window
Exec=/opt/sublime_text/sublime_text -n
OnlyShowIn=Unity;

[Desktop Action Document]
Name=New File
Exec=/opt/sublime_text/sublime_text --command new_file
OnlyShowIn=Unity;

I had to remove the sections entitled “Desktop Action Window” and “Desktop Action Document”. That left me with:

[Desktop Entry]
Version=1.0
Type=Application
Name=Sublime Text
GenericName=Text Editor
Comment=Sophisticated text editor for code, markup and prose
Exec=/opt/sublime_text/sublime_text %F
Terminal=false
MimeType=text/plain;
Icon=sublime-text
Categories=TextEditor;Development;
StartupNotify=true
Actions=Window;Document;

which Launchy detected easily.

As another example, Chrome's .desktop file lived in /var/lib/snapd/desktop/applications/chromium_chromium.desktop and was 293 lines long. Many of these lines where text entries defining strings in alternate languages, such as:

Name=Chromium Web Browser
Name[ast]=Restolador web Chromium
Name[bg]=Уеб четец Chromium
Name[bn]=ক্রোমিয়াম ওয়েব ব্
... etc ...

I deleted all of those alternate strings as well as the extra sections that defined other desktop actions.

That made Launchy actually show “Chromium Web Browser” as an option when I typed it in, but it wouldn't start the app (again, a silent failure).

It turned out the culprit was a complicated Exec entry that looked like this:

Exec=env BAMF_DESKTOP_FILE_HINT=/var/lib/snapd/desktop/applications/chromium_chromium.desktop /snap/bin/chromium %U

I simplified this to Exec=/snap/bin/chromium %U and it worked.

The final result was this:

[Desktop Entry]
Version=1.0
Name=Chromium Web Browser
GenericName=Web Browser
Comment=Access the Internet
Exec=/snap/bin/chromium %U
Terminal=false
Type=Application
Icon=/snap/chromium/562/chromium.png
Categories=Network;WebBrowser;
MimeType=text/html;text/xml;application/xhtml_xml;x-scheme-handler/http;x-scheme-handler/https;
StartupNotify=true
Actions=NewWindow;Incognito;TempProfile;

Now I can pretty much avoid using any of Lubuntu's awful, Windows 98 era menus and just focus on the shortcuts to apps I care about, which I can also manage in a simplified form in the same local git repo I use for everything else. This frees me up to enjoy the unrivaled performance and lack of nonsense (no animation when switching workspaces!) that has made Lubuntu my daily driver over the past two years.





<---2020-06-19 09:32:55 history 2018-12-09 09:32:55--->



-All material © 2007 Tim Simpson unless otherwise noted-