Table of contents
- Auto-generating the basic wrapper
- Fixing conversion errors – Part 1
- Dynamic linking
- Fixing conversion errors — Part 2
- Improving the wrapper
- Further links of interest
One of the many attractive things about Nim is its ability to interface with C libraries relatively easily, be they either statically linked or dynamically loaded. As all Nim source code is ultimately transformed to C code during compilation 1, Nim’s C FFI is unsurprisingly quite minimal. We still need to do some work, though, namely creating a Nim wrapper that will define our Nim API to the C library.
Thankfully, we don’t need to do all this by hand; there’s a handy tool aptly called
c2nim that can automatically generate such a wrapper from the C header files. But, as we’ll shortly see, while the tool is a great help to do the bulk of the grunt work, the generated files often need some further manual massaging to become usable.
In this article, we’ll examine the full process of creating a Nim wrapper for the well-known FMOD audio library, more specifically, for the FMOD Low Level API. Fortunately, FMOD provides both a C++ and C API, so we can just use the C headers which usually makes the job a lot easier than dealing with all the C++ nonsense…
First, we need to register at the FMOD website to be able to download the FMOD Studio API. The naming is a bit misleading because it actually contains the header and library files for both the FMOD Studio API and the FMOD Low Level API. There are three separate downloads for Windows, Linux and OS X. The C header files are located in
api/lowlevel/inc and the shared libraries in
api/lowlevel/lib inside the archives.
We’ll also need to install the c2nim tool to convert the C header files into Nim wrappers (it doesn’t come with the standard Nim installation). The project’s GitHub page contains the installation instructions.
Auto-generating the basic wrapper
The main header file is
fmod.h, but if we just tried to convert it using
c2nim, we would get errors. The reasons for this is that
c2nim does not perform C preprocessor expansion—we’ll need some help from
gcc to do that as the first step:
gcc -E fmod.h -o fmod_prep.h
Now we can run
c2nim on the resulting preprocessed header file without errors:
Yikes! Let’s try to compile it:
% nim c fmod_prep.nim Hint: used config file '/Users/johnnovak/.choosenim/toolchains/nim-0.18.0/config/nim.cfg' [Conf] Hint: system [Processing] Hint: fmod_prep [Processing] fmod_prep.nim(219, 45) Error: undeclared identifier: 'FMOD_SYSTEM'
[ CUE SAD TROMBONE… ]
Well, looks like there’s some extra work to be done here!
Fixing conversion errors – Part 1
Okay, first comes the easy part, let’s fix the compilation errors!
Opaqueue C structs
It turns out that the above error was raised because
c2nim just ignores opaque C structs. So we’ll need to manually add the Nim equivalents of all the opaque structs found in
Here’s the corresponding Nim conversion:
Our next compilation attempt is awarded with the following error:
% nim c fmod_prep.nim Hint: used config file '/Users/johnnovak/.choosenim/toolchains/nim-0.18.0/config/nim.cfg' [Conf] Hint: system [Processing] Hint: fmod_prep [Processing] fmod_prep.nim(775, 52) Error: undeclared identifier: 'FMOD_DSP_STATE'
This is caused by circular type definitions in the C header and it’s quite easy to fix—we just need to collapse all individual type definitions into a single
type block (mutually dependent types are only allowed within a single
type block in Nim).
Unsigned integer literals
The C type of the FMOD constants is
unsigned int, which gets mapped to unsigned 32-bit integers by most C compilers by tradition. In Nim, however, integer literals are interpreted as Nim signed
int types, which are mapped to the word-length of the target architecture—signed 64-bit ints, in our case. The current Nim implementation (0.18.0) has a quirk that it will convert such signed 64-bit int literals only if they fit into the signed width range of the target variable (which is signed 32-bit in this case):
So the following definition
will result in the below compilation error:
fmod.nim(2327, 40) Error: type mismatch: got <int64> but expected 'FMOD_MEMORY_TYPE = uint32'
This can get a bit confusing, but the workaround is quite simple: just append the
'u32 suffix to all literals that cannot be represented in the signed version of the target width:
Alright, we can compile our Nim wrapper now, but we’ll need to make a few adjustments to make it work with the FMOD shared libraries.
This is how the generated Nim function signatures look like:
The most flexible way to support shared library loading on multiple platforms is to add a user-defined
fmodImport pragma to all function signatures and of course the
cdecl pragma to use C calling conventions:
The definition of the
fmodImport pragma is the following (note that it’s possible to link against the logging version of FMOD by specifying the
-d:fmodDebugLog compiler option):
Fixing conversion errors — Part 2
So far so good, now we can compile the wrapper, we can load the shared library and access its exported functions from Nim, but there’s still one critical adjustment that needs to be made, otherwise we’d get failures at runtime. Apart from that, some useful constant and helper function definitions got lost in the conversion process, so we’ll need to add them in manually as well.
These problems are usually only spotted when one tries to actually use the generated wrapper, so it’s recommended to always give the wrappers some testing before releasing them to the public and don’t just assume that
c2nim did the right thing.
FMOD callbacks and function pointers
FMOD makes an extensive use of user-defined callback functions in its low-level API. Now, as we’ll implement these callbacks in Nim, we need to tell the compiler to use C calling conventions for them, otherwise we’d get random crashes at runtime2.
This is how a such callback definition looks like as output by
All we need to do is add the
cdecl pragma to all
FMOD_*_CALLBACK type definitions:
FMOD also exposes a large number of its internal C functions through structs containing function pointers (all
FMOD_*_FUNC type definitions); we’ll need to mark these as C functions as well:
I just realised at the end that if you supply the
--cdecl option to
c2nim, it will correctly annotate all function and function pointer declarations with the
cdecl pragma—certainly much more convenient than having to do it manually!
FMOD creates its own threads (at least by default), so these callbacks will be most likely invoked from different threads which would wreak havoc on the Nim garbage collector (meaning we’ll get random crashes). The solution is to compile with thread local storage emulation turned off (
-d:tlsEmulation=off) and invoke
system.setupForeignThreadGc() at the start of every callback proc. For further details see the Nim Backend Integration Manual.
It becomes quickly apparent during actual usage that lots of the
FMOD_* constants defined as
#define macros in the C headers are missing from our wrapper. We can instruct
gcc to include all macro definitions in the preprocessed output, but this will include every single
#define macro, including the internal ones used by the compiler, so it’s best to narrow the results down the ones we’re actually interested in:
gcc -E -dD fmod.h | grep "#define FMOD_" > fmod_constants.h
Now it’s just a matter of simply converting them to Nim constants. The reverb presets deserve a special mention:
Observer how nicer these look in Nim :)
Error handling helpers
Another thing that’s missing is the
FMOD_ErrorString helper function from
fmod_error.h to convert FMOD error codes into human readable messages. It’s trivial to convert the function, we’re just mentioning it here for completeness.
Improving the wrapper
Now that the wrapper is fully functional, we’ll make a little adjustment to make it more Nim-like. Recall how a typical FMOD function looks like:
This is the de-facto standard “object-oriented C” style, where the functions are prefixed with the classname (
FMOD_System in this case) and the first argument is the
this instance pointer. We can remove the prefix to make the API more Nim like:
After performing the above adjustment on all functions (with the help of some Vim macro magic), we can take advantage of Nim’s method call syntax and identifier equality rules to use the API in an object-oriented style (error checking is omitted for brevity):
That’s it folks! It might seem a bit complicated first, but it’s a pretty quick process once you are aware of all the gotchas.
The biggest drawback of this approach, though, is its very manual nature. Every time the API changes, the conversion process must be repeated which is time consuming and error prone. There exists a helper tool called nimgen that aims to automate this process, so if I was to do this again, I would certainly give that tool a go. Still, doing it fully manually at least once is a valuable learning experience to understand what should actually be automated.
Happy Nimming! :)
One way to spot Nim proc pointers is that they occupy twice as much memory than C function pointers. So on 64-bit while a C function pointer is 8-bytes, a Nim proc pointer is 16-bytes (as of Nim 0.18.0). One beneficial side-effect of this is that all C structs containing function pointers will end up being the wrong size if the
cdeclpragma is not added to the callback definitions, and because FMOD is strict about checking struct sizes passed in to its functions, we’d get struct size mismatch errors from FMOD instead of just crashing. In fact, this is how I spotted this problem in the first place. ↩