808bits

I wrote a ProtectServer FM in Zig and skipped the vendor compiler

2026-11-16 · 5 min · FIPS and HSMs

The compiler Thales documents for building ProtectServer functionality modules is GCC 4.6.1 for one generation of the toolkit and GCC 4.8.2 for the next. The first was released in June 2011 and the second in October 2013. The newer page adds that with a later GCC “warnings might appear during FM compilation”.

A functionality module, FM for short, is custom code that you sign and the HSM runs inside its own boundary, next to the keys. You write one when the PKCS#11 interface cannot do what you need, a signature scheme the firmware lacks for instance. The module is a 32-bit big-endian PowerPC binary, so the work starts with a cross-compiler, and Thales says FMs “can only be cross-compiled on Linux systems”.

I wrote one last December, over five days. It signs with BIP-340 Schnorr and ECDSA on secp256k1, with BLS12-381 and with ECDSA on P-256, and the vendor’s GCC was never part of the build.

Day one, in C

FMs are written in C, so that is where I started: a small module that did Schnorr signatures through libsecp256k1. What I did not want was the old GCC, so the build file was Zig’s from the start. Zig compiles C, it carries its cross-compilation targets in the one download, and asking for PowerPC is a field in the build file.

By the end of that day the C was gone. The Zig module that replaced it was about the same length, so brevity is not the argument. The vendor’s headers went through Zig’s C translation and I called the SDK functions directly, with no bindings written or generated. The same source also built natively, which meant the tests could run on my laptop.

A team with a C FM they trust can take the first half of that day and stop there: keep the code, change the compiler.

The key wipe

On the second day I added BLS through blst, and moved both C libraries from git submodules to Zig’s package manager, pinned to an exact revision. The build compiles each of them twice with the same compiler, for PowerPC and for the host.

That day I also noticed I had been clearing derived keys with a plain memory fill. Nothing reads the buffer afterwards, and that is the kind of write an optimiser is allowed to remove. Zig did not protect me from that any more than C would have. The standard library has a wipe that survives optimisation, and what the language adds is a way to make sure it runs:

fn sign(req: SignRequest, sig: *[64]u8) !void {
    var key: [32]u8 = undefined;
    defer std.crypto.secureZero(u8, &key);

    try deriveKey(req.key_id, &key);
    try schnorrSign(&key, req.msg, sig);
}

fn handle(buf: []const u8, sig: *[64]u8) Status {
    const req = parseSign(buf) catch return .bad_request;
    sign(req, sig) catch |err| return switch (err) {
        error.NoSuchKey => .no_such_key,
        error.DeviceError => .device_error,
    };
    return .ok;
}

This is a sketch written for the article, not the module’s code. defer runs the wipe on every path out of sign, including the two try lines that return early. The switch is checked by the compiler, so if deriveKey learns a new way to fail, the build stops until that error has a status code.

The error handling came later. For two days the module passed C-style return codes around, and on the third I moved it to Zig errors. It now turns about 90 PKCS#11 return codes into errors the compiler tracks.

Stateless keys

The first design stored keys in the HSM and had a command to delete them. On the third day I replaced it: each request derives its key from a master key that cannot be extracted, signs, and wipes it. The code got smaller, and much of what went was tests for key lifecycles that no longer existed.

That change touched the wire protocol, the module, the host client and the tests. They are all in one language and one build, so an old request type had nowhere to survive unnoticed.

Signatures that differed

On the last day I added ECDSA, first with the secp256k1 in Zig’s standard library and then with libsecp256k1 in its place. I wrote tests that give both implementations the same key and message, and I expected identical signatures, since both derive the nonce deterministically.

They differed. libsecp256k1 follows RFC 6979. Zig’s standard library uses the scheme from the “Deterministic ECDSA and EdDSA Signatures with Additional Randomness” draft, which feeds 32 more bytes into the HMAC and fills them with zeros when the caller passes no noise. So the nonces differ and so do the signatures, and both are valid. The test now has Zig’s verifier accept what the C library signed. P-256 in the module is still the standard library’s own code.

The second implementation was one import away, which is the only reason that test exists.

What stays vendor

Zig replaced the compiler and nothing else. The headers, the libraries the module links against and the tool that signs the image are still Thales’s, and the HSM still checks that signature against a certificate before it runs the module. I am leaving out the build file and everything about signing and loading.

Zig is not at 1.0, and it shows. The builtin I used to import C headers has since been removed from the language, and this month I ported the module so that translation happens in the build file. 32-bit PowerPC on Linux is a Tier 2 target, which gets less testing than x86-64.

The objection I would raise myself is that Thales documents one compiler and this is not it, and the code runs next to the keys. The output is an ordinary ELF object that calls the same vendor libraries through the C calling convention, so what can go wrong is calling-convention detail, and that can be tested. Anyone with a support agreement should still ask first.

I have none. My ProtectServer is a decommissioned QA unit that a bank sold off, and it lives at home.