Keep Code Awake with Very Sleepy
Wednesday January 1, 2014 18:50:11

I've always appreciated quick programs, especially ones that perform well even on ancient or cheap hardware. I feel that most software running today is far slower than it needs to be, which has led to a global waste of hardware resources as everyone compensates by upgrading their gear. So of course I'd like to aspire to write quick code myself.

Most advice I've read on writing efficient code states that measurements are key and should be taken before taking any action at all. Unfortunately, everything I've read fails to mention what tools should be used for measuring code performance or cite products costing thousands of dollars. They often include a snarky comment about how if you're a serious developers you'll recoup your investment anyway. While I've slowly accrued enough hours of experience writing C++ over the years that I wouldn't laugh at myself too hard if I said I was a serious C++ developer, because the majority of work I do is at home in my spare time I just can't justify dropping several grand on code analysis tools.

So I was very happy the other day to discover Very Sleepy, a totally free code profiler that ended up helping me trim nearly 30% off the running time of Macaroni for C++.

Very Sleepy is noninvasive, meaning you don't have to recompile an application to use it (though the app must be compiled with debug symbols). How it works is by suspending the app once each millisecond and recording the stack trace. It then determines how much time is being spent in each function. Here's how I used it to get major performance gains from Macaroni in less than one hour.

Over the past year or so, I've moved from working on Macaroni for C++ in my spare time to actually using it for a game engine. The experience has made me aware of different, more practical issues in Macaroni itself. One thing I've noticed is while run times with Macaroni aren't that bad, they aren't where I'd like them to be either.

A few months back I added a feature to Macaroni which shows the time it's spent running so far next to every line output to the console. That helped me to get some solid numbers on how long projects were taking to parse and build.

Today, I was running a project which Macaroni spent 5.732 seconds simply analyzing and generating code for before it even invoked bjam. This isn't necessarily trivial work, but since Macaroni is not itself a real C++ compiler it's the kind of task I'd ultimately like to see happen in one second or less.

So, I invoked Macaroni as usual but added the “startPrompt” argument, which makes it sit around and wait for a user to press enter. This allows the program to get attached to a debugger. I then opened up Very Sleepy, and found Macaroni on it's list of running processes. I double clicked it, and then raced back to enter something into Macaroni's dummy prompt, so the time spent waiting at the prompt wouldn't skew the results (note that it's also possible to launch a process from Very Sleepy, but I was being lazy - though hopefully this serves as a proof of sorts that even casual usage can yield tangible benefits).

Since Very Sleepy was busy profiling it, Macaroni took longer to run- about 40 seconds or so- but when it stopped I got the following nice window showing all of the function calls in Macaroni and which ones had taken the most time.

One thing that's interesting is that the “% Inclusive” column shows the time spent not just running code in a function, but running all the code that function itself called. Because of this, the highest “% Inclusive” value is for the main method followed by other methods, such as “Macaroni::Environment::ProjectEnvironment::RunCommand,” which are the jumping off points for almost everything that happens in a typical invocation of Macaroni. However, it's still easy to spot functions which you know aren't called that early in the call graph.

The winning stinker of the bunch was “boost::filesystem::canonical” which is used by Macaroni's own Path class (which has different semantics in order to represent C++ source files easily, and which I wrote before I even knew of boost::filesystem… this project has taken so many years) to return an absolute path. As time has gone by, I've peppered the more recently created project / build system in Macaroni with things that need the absolute path, and it turns out the boost::filesystem function underpinning that is relatively expensive.

The fix was to simply cache the absolute path the first time it was computed, which took no time at all. This simple change cut two seconds off the run time, bringing the time until bjam was invoked from 5.732 to 3.874.

In short, Very Sleepy is an incredibly useful tool which makes spotting performance issues extremely simple. I'm grateful to it's authors for writing something so useful and giving it away for free.

Texts- a WYSIWG editor for lovers of plain text
Sunday December 8, 2013 22:30:09

Yesterday I discovered the regretably named Texts, a WYSIWYG text editor that saves to plain text files in the Markdown formatting syntax, which is also used by innumerable sites including Github and StackOverflow.

I once used Microsoft Word for anything that wasn't program code, but as time went on I found myself on machines without Word installed. Buying Word was never worth it to me, yet alternatives such as OpenOffice felt too clumsy to be enjoyable. At some point I decided I'd be able to live using plain text files alone, particularly after I discovered Restructured Text, the primary format syntax used by Python's standard documentation tools. The cool thing about RST files was that they stayed simple text files which could be editted easily in Sublime Text and could live comfortably next to my code and in Github repositories.

However, something always felt off about using Sublime Text to write scripts or stories, probably thanks to the monospaced fonts used for viewing code. While I spent the first five years of my computer enabled life writing stories in Word Perfect and other old school word processors which had nothing but monospaced fonts, at some point using them for creative writing began to feel unnatural. My theory is that as years went by and variable width fonts became more prevelant, my brain began to recognize monospaced fonts as used for reading and writing code only, which caused them to stifle the writing process. While it's possible to use variable width fonts for Sublime Text I couldn't find an easy way to change it on a per file extension basis, meaning I'd have to change it globally and thus ruin the tool's ability to edit code.

The Texts editor on the other hand allows me to write documents with formatting options such as headers and bold text visible, which makes the experience almost as enjoyable as it was on Word. It has the added advantage of saving the files to a format that's friendly to all the programmer tools I already use, such as Git, DiffMerge or Sublime Text. It even supports saving documents in multiple types of Markdown, such as the flavor used by Github.

The only real disadvantage is it supports Markdown instead of Restructured Text- there's a lot of Python documentation that would've been really nice to edit with Texts. Markdown also is a less standardized format than its chief rival. Additionally, Texts is currently a bit rough around the edges- I can't find figure out how to activate the spell checker or exit from full screen mode. However, as its the only real tool I've ever found to scratch this particular itch I still recommend it to anyone who wants the nicities of a WYSIWYG editor but is loathe to give up the independence offered by plain text files.

Syntax highlighting for Boost Build / Bjam in Sublime Text
Saturday December 7, 2013 20:06:00

I was opening a Bjam file today and found myself thinking about how intimidating the language looks to most newcomers, until the basics of the syntax are understood. I also thought of what a drag it is to open a bjam file and see monochrome.

So, I used some of the knowledge I gained earlier when I added syntax support for Macaroni to Sublime Text and whipped up a fresh tmLanguage file which colorizes Boost Build files. Thanks to Sublime Text's easy and intuitive nature as well as several helpful tutorials it only took a few hours.

Of course, it isn't perfect- it falls apart on truly complex jam files- but it's better than nothing.

Click here to view the Github project.

Add color to Boost Build with BJolt
Wednesday November 27, 2013 15:58:25

I recently added CMake support to Macaroni after growing incredibly frustrated with how long it took to compile some apps with Boost Build on my ancient Acer Aspire One. Because CMake can generate Visual Studio project files, this meant I was able to easily work on Macaroni projects again in Visual Studio.

Unfortunately, Visual Studio 2013 seems to have be more hostile towards Macaroni than its 2008 iteration; syntax is still correctly highlighted, but the IDE adds scribbles to almost everything. Worst of all, it refuses to jump to errors in Macaroni source code when you click on them in the IDE even though it's own compiler is outputting the appropriate lines. I actually used to use VS2008 with Macaroni all the time several years ago and had a good experience, so VS2013's intolerance for mcpp files (possibly arising from the fact it's improved as a C++ IDE) is a bit sad. The end result is I stick to Sublime and the command line when writing and testing code and use VS2013 when I need the debugger.

However, not being able to click on those errors really was a bummer, and perhaps its what led me do something to improve the experience of using Boost Build on the command line. The worst part of using it today is that while Macaroni itself has colorized output, when it runs Boost Build the affair is strictly monochromatic, meaning I've gotten very good at scanning through dozens or even hundreds of lines of text to find the one or two error messages that caused the failure. It looks something like this:

So, in about two mornings worth of time last weekend I created Bjolt, which runs bjam and colorizes its output as well as formatting long lines to make them easier to read in Windows. Now it looks more like this:

So far it finds errors and warning from the Visual C++ compiler and linker, as well as error messages from Boost Test. In the future I'll probably add support for GCC and Clang (ironically Clang has beautiful colorization, but Boost Build unfortunately ruins it).

Download Bjolt here.

<---2015-04-13 16:18:00 history 2013-08-25 03:20:19--->

-All material © 2007 Tim Simpson unless otherwise noted-