Creating a Nim wrapper for FMOD
Table of contents
Overview
One of the many attractive things about Nim is its ability to interface with C libraries relatively easily, be they either statically linked or dynamically loaded. As all Nim source code is ultimately transformed to C code during compilation 1, Nim’s C FFI is unsurprisingly quite minimal. We still need to do some work, though, namely creating a Nim wrapper that will define our Nim API to the C library.
Thankfully, we don’t need to do all this by hand; there’s a handy tool aptly
called c2nim
that can automatically generate such a wrapper from the
C header files. But, as we’ll shortly see, while the tool is a great help to
do the bulk of the grunt work, the generated files often need some further
manual massaging to become usable.
In this article, we’ll examine the full process of creating a Nim wrapper for the well-known FMOD audio library, more specifically, for the FMOD Low Level API. Fortunately, FMOD provides both a C++ and C API, so we can just use the C headers which usually makes the job a lot easier than dealing with all the C++ nonsense…
Prerequisites
First, we need to register at the FMOD website to be able
to download the FMOD Studio API. The naming is
a bit misleading because it actually contains the header and library files for
both the FMOD Studio API and the FMOD Low Level API. There are three
separate downloads for Windows, Linux and OS X. The C header files are located
in api/lowlevel/inc
and the shared libraries in api/lowlevel/lib
inside
the archives.
We’ll also need to install the c2nim tool to convert the C header files into Nim wrappers (it doesn’t come with the standard Nim installation). The project’s GitHub page contains the installation instructions.
Auto-generating the basic wrapper
The main header file is fmod.h
, but if we just tried to convert it using
c2nim
, we would get errors. The reasons for this is that c2nim
does not
perform C preprocessor expansion—we’ll need some help from gcc
to do that
as the first step:
gcc -E fmod.h -o fmod_prep.h
Now we can run c2nim
on the resulting preprocessed header file without
errors:
c2nim fmod_prep.h
Yikes! Let’s try to compile it:
% nim c fmod_prep.nim
Hint: used config file '/Users/johnnovak/.choosenim/toolchains/nim-0.18.0/config/nim.cfg' [Conf]
Hint: system [Processing]
Hint: fmod_prep [Processing]
fmod_prep.nim(219, 45) Error: undeclared identifier: 'FMOD_SYSTEM'
[ CUE SAD TROMBONE… ]
Well, looks like there’s some extra work to be done here!
Fixing conversion errors – Part 1
Okay, first comes the easy part, let’s fix the compilation errors!
Opaqueue C structs
It turns out that the above error was raised because c2nim
just ignores
opaque C structs. So we’ll need to manually add the Nim equivalents of all
the opaque structs found in fmod_common.h
:
typedef struct FMOD_SYSTEM FMOD_SYSTEM;
typedef struct FMOD_SOUND FMOD_SOUND;
typedef struct FMOD_CHANNELCONTROL FMOD_CHANNELCONTROL;
typedef struct FMOD_CHANNEL FMOD_CHANNEL;
...
Here’s the corresponding Nim conversion:
type
FMOD_SYSTEM* = object
FMOD_SOUND* = object
FMOD_CHANNELCONTROL* = object
FMOD_CHANNEL* = object
...
Circular types
Our next compilation attempt is awarded with the following error:
% nim c fmod_prep.nim
Hint: used config file '/Users/johnnovak/.choosenim/toolchains/nim-0.18.0/config/nim.cfg' [Conf]
Hint: system [Processing]
Hint: fmod_prep [Processing]
fmod_prep.nim(775, 52) Error: undeclared identifier: 'FMOD_DSP_STATE'
This is caused by circular type definitions in the C header and it’s quite
easy to fix—we just need to collapse all individual type definitions
into a single type
block (mutually dependent types are only allowed within
a single type
block in Nim).
Unsigned integer literals
The C type of the FMOD constants is unsigned int
, which gets mapped to
unsigned 32-bit integers by most C compilers by tradition. In Nim, however,
integer literals are interpreted as Nim signed int
types, which are mapped to
the word-length of the target architecture—signed 64-bit ints, in our case.
The current Nim implementation (0.18.0) has a quirk that it will convert such
signed 64-bit int literals only if they fit into the signed width range of
the target variable (which is signed 32-bit in this case):
So the following definition
type
FMOD_MEMORY_TYPE* = cuint
...
FMOD_MEMORY_ALL*: FMOD_MEMORY_TYPE = 0xFFFFFFFF
will result in the below compilation error:
fmod.nim(2327, 40) Error: type mismatch: got <int64> but expected 'FMOD_MEMORY_TYPE = uint32'
This can get a bit confusing, but the workaround is quite simple: just append
the 'u32
suffix to all literals that cannot be represented in the signed
version of the target width:
FMOD_MEMORY_ALL*: FMOD_MEMORY_TYPE = 0xFFFFFFFF'u32
Dynamic linking
Alright, we can compile our Nim wrapper now, but we’ll need to make a few adjustments to make it work with the FMOD shared libraries.
This is how the generated Nim function signatures look like:
proc FMOD_System_PlaySound*(system: ptr FMOD_SYSTEM; sound: ptr FMOD_SOUND;
channelgroup: ptr FMOD_CHANNELGROUP; paused: FMOD_BOOL;
channel: ptr ptr FMOD_CHANNEL): FMOD_RESULT
The most flexible way to support shared library loading on multiple platforms
is to add a user-defined fmodImport
pragma to all function signatures and
of course the cdecl
pragma to use C calling conventions:
proc FMOD_System_PlaySound*(system: ptr FMOD_SYSTEM; sound: ptr FMOD_SOUND;
channelgroup: ptr FMOD_CHANNELGROUP; paused: FMOD_BOOL;
channel: ptr ptr FMOD_CHANNEL): FMOD_RESULT {.fmodImport, cdecl.}
The definition of the fmodImport
pragma is the following (note that it’s
possible to link against the logging version of FMOD by specifying the
-d:fmodDebugLog
compiler option):
import strformat
when defined(fmodDebugLog):
var L {.compileTime.} = "L"
else:
var L {.compileTime.} = ""
when defined(windows):
when defined(amd64):
const FmodDll = fmt"fmod{L}64.dll"
when defined(i386):
const FmodDll = fmt"fmod{L}.dll"
elif defined(macosx):
const FmodDll = fmt"libfmod{L}.dylib"
else:
const FmodDll = fmt"libfmod{L}.so"
{.pragma: fmodImport, dynlib: FmodDll.}
Fixing conversion errors — Part 2
So far so good, now we can compile the wrapper, we can load the shared library and access its exported functions from Nim, but there’s still one critical adjustment that needs to be made, otherwise we’d get failures at runtime. Apart from that, some useful constant and helper function definitions got lost in the conversion process, so we’ll need to add them in manually as well.
These problems are usually only spotted when one tries to actually use the
generated wrapper, so it’s recommended to always give the wrappers some testing
before releasing them to the public and don’t just assume that c2nim
did the
right thing.
FMOD callbacks and function pointers
FMOD makes an extensive use of user-defined callback functions in its low-level API. Now, as we’ll implement these callbacks in Nim, we need to tell the compiler to use C calling conventions for them, otherwise we’d get random crashes at runtime2.
This is how a such callback definition looks like as output by c2nim
:
FMOD_SOUND_PCMREAD_CALLBACK* = proc (sound: ptr FMOD_SOUND; data: pointer;
datalen: cuint): FMOD_RESULT
All we need to do is add the cdecl
pragma to all FMOD_*_CALLBACK
type
definitions:
FMOD_SOUND_PCMREAD_CALLBACK* = proc (sound: ptr FMOD_SOUND; data: pointer;
datalen: cuint): FMOD_RESULT {.cdecl.}
FMOD also exposes a large number of its internal C functions through structs
containing function pointers (all FMOD_*_FUNC
type definitions); we’ll need
to mark these as C functions as well:
FMOD_DSP_GETUSERDATA_FUNC* = proc (dsp_state: ptr FMOD_DSP_STATE;
userdata: ptr pointer): FMOD_RESULT {.cdecl.}
I just realised at the end that if you supply the --cdecl
option to c2nim
,
it will correctly annotate all function and function pointer declarations with
the cdecl
pragma—certainly much more convenient than having to do it
manually!
FMOD creates its own threads (at least by default), so these callbacks will be
most likely invoked from different threads which would wreak havoc on the Nim
garbage collector (meaning we’ll get random crashes). The solution is to
compile with thread local storage emulation turned off (-d:tlsEmulation=off
)
and invoke system.setupForeignThreadGc()
at the start of every callback proc.
For further details see the Nim Backend Integration Manual.
Missing constants
It becomes quickly apparent during actual usage that lots of the FMOD_*
constants defined as #define
macros in the C headers are missing from our
wrapper. We can instruct gcc
to include all macro definitions in the
preprocessed output, but this will include every single #define
macro,
including the internal ones used by the compiler, so it’s best to narrow the
results down the ones we’re actually interested in:
gcc -E -dD fmod.h | grep "#define FMOD_" > fmod_constants.h
Now it’s just a matter of simply converting them to Nim constants. The reverb presets deserve a special mention:
#define FMOD_PRESET_OFF { 1000, 7, 11, 5000, 100, 100, 100, 250, 0, 20, 96, -80.0f }
Observer how nicer these look in Nim :)
const
FMOD_PRESET_OFF* = FMOD_REVERB_PROPERTIES(
decayTime: 1000,
earlyDelay: 7,
lateDelay: 11,
hfReference: 5000,
hfDecayRatio: 100,
diffusion: 100,
density: 100,
lowShelfFrequency: 250,
lowShelfGain: 0,
highCut: 20,
earlyLateMix: 96,
wetLevel: -80
)
Error handling helpers
Another thing that’s missing is the FMOD_ErrorString
helper function from
fmod_error.h
to convert FMOD error codes into human readable messages. It’s
trivial to convert the function, we’re just mentioning it here for
completeness.
Improving the wrapper
Now that the wrapper is fully functional, we’ll make a little adjustment to make it more Nim-like. Recall how a typical FMOD function looks like:
proc FMOD_System_PlaySound*(system: ptr FMOD_SYSTEM; sound: ptr FMOD_SOUND;
channelgroup: ptr FMOD_CHANNELGROUP; paused: FMOD_BOOL;
channel: ptr ptr FMOD_CHANNEL): FMOD_RESULT {.fmodImport, cdecl.}
This is the de-facto standard “object-oriented C” style, where the functions
are prefixed with the classname (FMOD_System
in this case) and the first
argument is the this
instance pointer. We can remove the prefix to make the
API more Nim like:
proc playSound*(system: ptr FMOD_SYSTEM; sound: ptr FMOD_SOUND;
channelgroup: ptr FMOD_CHANNELGROUP; paused: FMOD_BOOL;
channel: ptr ptr FMOD_CHANNEL): FMOD_RESULT {.fmodImport, cdecl.}
After performing the above adjustment on all functions (with the help of some Vim macro magic), we can take advantage of Nim’s method call syntax and identifier equality rules to use the API in an object-oriented style (error checking is omitted for brevity):
var
res: FmodResult
system: ptr FmodSystem
sound: ptr FmodSound
channel: ptr FmodChannel
discard create(system.addr)
discard system.init(512, FMOD_INIT_NORMAL, nil)
discard system.createSound("media/jaguar.wav", FMOD_DEFAULT, nil, sound.addr)
discard system.playSound(sound, nil, 0, channel.addr)
sound.release()
Conclusion
That’s it folks! It might seem a bit complicated first, but it’s a pretty quick process once you are aware of all the gotchas.
The biggest drawback of this approach, though, is its very manual nature. Every time the API changes, the conversion process must be repeated which is time consuming and error prone. There exists a helper tool called nimgen that aims to automate this process, so if I was to do this again, I would certainly give that tool a go. Still, doing it fully manually at least once is a valuable learning experience to understand what should actually be automated.
The finished version of the nim-fmod
wrapper is available on
GitHub with some examples included
and as a Nimble package.
Happy Nimming! :)
Further links of interest
Of course, this is not true in case of the experimental JavaScript backend. ↩︎
One way to spot Nim proc pointers is that they occupy twice as much memory than C function pointers. So on 64-bit while a C function pointer is 8-bytes, a Nim proc pointer is 16-bytes (as of Nim 0.18.0). One beneficial side-effect of this is that all C structs containing function pointers will end up being the wrong size if the
cdecl
pragma is not added to the callback definitions, and because FMOD is strict about checking struct sizes passed in to its functions, we’d get struct size mismatch errors from FMOD instead of just crashing. In fact, this is how I spotted this problem in the first place. ↩︎
Comments
comments powered by Disqus