zlib for Linux

v1.3.2Linux

zlib 1.3.2 for native Node.js addons on Linux, precompiled for x64 and arm64, glibc 2.28 or later as @crossbind/port-zlib-linux.

npm install @crossbind/port-zlib-linux@beta

npm ↗Port recipe and licence files ↗

Install

shell

npm install @crossbind/port-zlib-linux@beta
npm install --save-dev crossbind@beta @crossbind/core-embind-napi@beta

crossbind.config.js

import zlibLinux from '@crossbind/port-zlib-linux/crossbind.config.js';
 
export default {
    dependencies: [zlibLinux],
    paths: { config: import.meta.url },
};

shell

npx crossbind build -p linux

The addons and their loader land in dist; the Node.js playbook has the whole flow.

Usage

The examples the WebAssembly page runs, as Linux compiles them: the same headers and the same calls. They are checked on the WebAssembly build.

Compress and decompress a buffer

The two calls most zlib code makes: compress2 at a chosen level, and uncompress with the original size the caller kept.

src/native/zlib_codec.h

#pragma once
 
#include <zlib.h>
 
#include <stdexcept>
#include <string>
 
// One-shot zlib: compress2 and uncompress, in the zlib format (RFC 1950). Bytes cross the binding as
// a byte string: one UTF-16 code unit (0-255) per byte.
class Zlib {
public:
    static std::string version() { return zlibVersion(); }
 
    static std::u16string compress(const std::string& text, int level) {
        uLongf size = compressBound(static_cast<uLong>(text.size()));
        std::string out(size, '\0');
        const int status = compress2(reinterpret_cast<Bytef*>(&out[0]), &size, reinterpret_cast<const Bytef*>(text.data()), static_cast<uLong>(text.size()), level);
        if (status != Z_OK) throw std::runtime_error(status == Z_STREAM_ERROR ? "level must be between -1 and 9" : zError(status));
        std::u16string bytes(size, u'\0');
        for (uLongf i = 0; i < size; ++i) bytes[i] = static_cast<unsigned char>(out[i]);
        return bytes;
    }
 
    // The zlib format does not record the original size, so the caller keeps it next to the data.
    static std::string decompress(const std::u16string& bytes, int originalSize) {
        if (originalSize < 0 || originalSize > (256 << 20)) throw std::invalid_argument("originalSize must be between 0 and 256 MiB");
        std::string in(bytes.size(), '\0');
        for (size_t i = 0; i < bytes.size(); ++i) {
            if (bytes[i] > 0xFF) throw std::invalid_argument("not a byte string");
            in[i] = static_cast<char>(bytes[i]);
        }
        std::string out(static_cast<size_t>(originalSize), '\0');
        uLongf size = static_cast<uLongf>(out.size());
        const int status = uncompress(reinterpret_cast<Bytef*>(&out[0]), &size, reinterpret_cast<const Bytef*>(in.data()), static_cast<uLong>(in.size()));
        if (status == Z_BUF_ERROR) throw std::runtime_error("the data is larger than originalSize");
        if (status != Z_OK) throw std::runtime_error(status == Z_DATA_ERROR ? "corrupt or incomplete zlib data" : zError(status));
        out.resize(size);
        return out;
    }
};

main.js

import { initNative, Zlib } from './native/zlib_codec.h';
 
await initNative();
const text = 'crossbind '.repeat(100);
const packed = await Zlib.compress(text, 9);
const bytes = Uint8Array.from(packed, (c) => c.charCodeAt(0));
console.log(await Zlib.version(), bytes.length, [...bytes.slice(0, 2)].map((b) => b.toString(16)).join(' '));
console.log((await Zlib.decompress(packed, text.length)) === text);

PRINTS

1.3.2 27 78 da
true

Write a .gz that remembers its file name

deflateInit2 with windowBits 15 + 16 writes gzip instead of zlib, and deflateSetHeader fills the header fields that gunzip -N restores. inflateGetHeader reads them back.

src/native/gzip_header.h

#pragma once
 
#include <zlib.h>
 
#include <memory>
#include <stdexcept>
#include <string>
#include <vector>
 
// gzip (RFC 1952) with the header fields gunzip -N restores: the original file name and the
// modification time. Bytes cross the binding as a byte string: one UTF-16 code unit (0-255) per byte.
class Gzip {
public:
    static std::u16string compress(const std::string& data, const std::string& name, double mtime, int level) {
        z_stream stream{};
        // windowBits 15 + 16: a 32 KiB window, and the gzip wrapper instead of the zlib one.
        if (deflateInit2(&stream, level, Z_DEFLATED, 15 + 16, 8, Z_DEFAULT_STRATEGY) != Z_OK) throw std::invalid_argument("level must be between -1 and 9");
        std::unique_ptr<z_stream, int (*)(z_stream*)> end(&stream, deflateEnd);
        gz_header header{};
        header.name = reinterpret_cast<Bytef*>(const_cast<char*>(name.c_str()));
        header.time = static_cast<uLong>(mtime);
        header.os = 3;  // Unix
        deflateSetHeader(&stream, &header);
        std::string out(deflateBound(&stream, static_cast<uLong>(data.size())), '\0');
        stream.next_in = reinterpret_cast<Bytef*>(const_cast<char*>(data.data()));
        stream.avail_in = static_cast<uInt>(data.size());
        stream.next_out = reinterpret_cast<Bytef*>(&out[0]);
        stream.avail_out = static_cast<uInt>(out.size());
        if (deflate(&stream, Z_FINISH) != Z_STREAM_END) throw std::runtime_error("deflate did not finish");
        std::u16string bytes(stream.total_out, u'\0');
        for (size_t i = 0; i < bytes.size(); ++i) bytes[i] = static_cast<unsigned char>(out[i]);
        return bytes;
    }
 
    static std::string fileName(const std::u16string& gz) {
        std::string name;
        uLong mtime = 0;
        readHeader(gz, name, mtime);
        return name;
    }
 
    // Seconds since 1970-01-01T00:00:00Z.
    static double modified(const std::u16string& gz) {
        std::string name;
        uLong mtime = 0;
        readHeader(gz, name, mtime);
        return static_cast<double>(mtime);
    }
 
    static std::string decompress(const std::u16string& gz) {
        std::string in = fromUnits(gz);
        z_stream stream{};
        if (inflateInit2(&stream, 15 + 32) != Z_OK) throw std::runtime_error("inflateInit2 failed");  // + 32: gzip or zlib, from the header
        std::unique_ptr<z_stream, int (*)(z_stream*)> end(&stream, inflateEnd);
        stream.next_in = reinterpret_cast<Bytef*>(&in[0]);
        stream.avail_in = static_cast<uInt>(in.size());
        std::string out;
        std::vector<char> chunk(64 << 10);
        int status = Z_OK;
        while (status != Z_STREAM_END) {
            stream.next_out = reinterpret_cast<Bytef*>(chunk.data());
            stream.avail_out = static_cast<uInt>(chunk.size());
            status = inflate(&stream, Z_NO_FLUSH);
            if (status != Z_OK && status != Z_STREAM_END) throw std::runtime_error(stream.msg ? stream.msg : "truncated or corrupt input");
            out.append(chunk.data(), chunk.size() - stream.avail_out);
            if (out.size() > (256u << 20)) throw std::runtime_error("refusing to decompress more than 256 MiB");
        }
        return out;
    }
 
private:
    static std::string fromUnits(const std::u16string& units) {
        std::string bytes(units.size(), '\0');
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<char>(units[i]);
        }
        return bytes;
    }
 
    // inflateGetHeader fills the fields while inflate reads the header; Z_BLOCK stops right after it.
    static void readHeader(const std::u16string& gz, std::string& name, uLong& mtime) {
        std::string in = fromUnits(gz);
        z_stream stream{};
        if (inflateInit2(&stream, 15 + 16) != Z_OK) throw std::runtime_error("inflateInit2 failed");  // + 16: gzip only
        std::unique_ptr<z_stream, int (*)(z_stream*)> end(&stream, inflateEnd);
        std::vector<Bytef> nameBuffer(1024, 0);
        gz_header header{};
        header.name = nameBuffer.data();
        header.name_max = static_cast<uInt>(nameBuffer.size() - 1);
        inflateGetHeader(&stream, &header);
        Bytef output = 0;
        stream.next_in = reinterpret_cast<Bytef*>(&in[0]);
        stream.avail_in = static_cast<uInt>(in.size());
        stream.next_out = &output;
        stream.avail_out = 1;
        const int status = inflate(&stream, Z_BLOCK);
        if ((status != Z_OK && status != Z_STREAM_END) || header.done != 1) throw std::runtime_error(stream.msg ? stream.msg : "not a complete gzip header");
        name = reinterpret_cast<const char*>(nameBuffer.data());
        mtime = header.time;
    }
};

main.js

import { initNative, Gzip } from './native/gzip_header.h';
 
await initNative();
const rows = Array.from({ length: 200 }, (_, i) => `${i + 1},sensor-${i % 4},${18 + ((i * 7) % 9)}`);
const csv = ['reading,sensor,celsius', ...rows].join('\n') + '\n';
const gz = await Gzip.compress(csv, 'readings.csv', Date.UTC(2026, 0, 1) / 1000, 9);
const bytes = Uint8Array.from(gz, (c) => c.charCodeAt(0));
console.log(`${csv.length} B -> ${bytes.length} B, starts ${bytes[0].toString(16)} ${bytes[1].toString(16)}`);
console.log(await Gzip.fileName(gz), new Date((await Gzip.modified(gz)) * 1000).toISOString());
console.log((await Gzip.decompress(gz)) === csv);

PRINTS

3115 B -> 747 B, starts 1f 8b
readings.csv 2026-01-01T00:00:00.000Z
true

Compress small messages with a preset dictionary

A short message has little to refer back to. Load earlier messages with deflateSetDictionary and inflateSetDictionary, and each new one compresses against them.

src/native/zlib_dictionary.h

#pragma once
 
#include <zlib.h>
 
#include <memory>
#include <stdexcept>
#include <string>
#include <vector>
 
// Raw deflate (RFC 1951, no header or checksum) with a preset dictionary: both sides load the same
// bytes first, so a short message can point back at strings it never contained. An empty dictionary
// gives plain raw deflate.
class ZlibDictionary {
public:
    ZlibDictionary(const std::string& dictionary, int level) : dictionary(dictionary), level(level) {}
 
    std::u16string compress(const std::string& message) const {
        z_stream stream{};
        // windowBits -15: raw deflate with a 32 KiB window.
        if (deflateInit2(&stream, level, Z_DEFLATED, -15, 8, Z_DEFAULT_STRATEGY) != Z_OK) throw std::invalid_argument("level must be between -1 and 9");
        std::unique_ptr<z_stream, int (*)(z_stream*)> end(&stream, deflateEnd);
        if (!dictionary.empty()) deflateSetDictionary(&stream, reinterpret_cast<const Bytef*>(dictionary.data()), static_cast<uInt>(dictionary.size()));
        std::string out(deflateBound(&stream, static_cast<uLong>(message.size())), '\0');
        stream.next_in = reinterpret_cast<Bytef*>(const_cast<char*>(message.data()));
        stream.avail_in = static_cast<uInt>(message.size());
        stream.next_out = reinterpret_cast<Bytef*>(&out[0]);
        stream.avail_out = static_cast<uInt>(out.size());
        if (deflate(&stream, Z_FINISH) != Z_STREAM_END) throw std::runtime_error("deflate did not finish");
        std::u16string bytes(stream.total_out, u'\0');
        for (size_t i = 0; i < bytes.size(); ++i) bytes[i] = static_cast<unsigned char>(out[i]);
        return bytes;
    }
 
    std::string decompress(const std::u16string& bytes) const {
        std::string in(bytes.size(), '\0');
        for (size_t i = 0; i < bytes.size(); ++i) {
            if (bytes[i] > 0xFF) throw std::invalid_argument("not a byte string");
            in[i] = static_cast<char>(bytes[i]);
        }
        z_stream stream{};
        if (inflateInit2(&stream, -15) != Z_OK) throw std::runtime_error("inflateInit2 failed");
        std::unique_ptr<z_stream, int (*)(z_stream*)> end(&stream, inflateEnd);
        // Raw deflate carries no dictionary ID, so the dictionary goes in before the first inflate().
        if (!dictionary.empty()) inflateSetDictionary(&stream, reinterpret_cast<const Bytef*>(dictionary.data()), static_cast<uInt>(dictionary.size()));
        stream.next_in = reinterpret_cast<Bytef*>(&in[0]);
        stream.avail_in = static_cast<uInt>(in.size());
        std::string out;
        std::vector<char> chunk(16 << 10);
        int status = Z_OK;
        while (status != Z_STREAM_END) {
            stream.next_out = reinterpret_cast<Bytef*>(chunk.data());
            stream.avail_out = static_cast<uInt>(chunk.size());
            status = inflate(&stream, Z_NO_FLUSH);
            if (status != Z_OK && status != Z_STREAM_END) throw std::runtime_error(stream.msg ? stream.msg : "truncated or corrupt input");
            out.append(chunk.data(), chunk.size() - stream.avail_out);
            if (out.size() > (1u << 20)) throw std::runtime_error("a message above 1 MiB is not a small message");
        }
        return out;
    }
 
private:
    std::string dictionary;
    int level;
};

main.js

import { initNative, ZlibDictionary } from './native/zlib_dictionary.h';
 
await initNative();
const event = (i) => `{"event":"click","user":${1000 + ((i * 37) % 900)},"page":"/products/${i % 12}","ms":${(i * 7919) % 400}}`;
const dictionary = Array.from({ length: 20 }, (_, i) => event(i)).join('\n');
const plain = await new ZlibDictionary('', 9);
const primed = await new ZlibDictionary(dictionary, 9);
 
const message = event(4321);
const alone = await plain.compress(message);
const packed = await primed.compress(message);
console.log(`dictionary: ${dictionary.length} B`);
console.log(`${message.length} B message: ${alone.length} B alone, ${packed.length} B with the dictionary`);
console.log((await primed.decompress(packed)) === message);

PRINTS

dictionary: 1195 B
59 B message: 59 B alone, 11 B with the dictionary
true

Checksum data in pieces

crc32 and adler32 continue a running value across pieces, and crc32_combine and adler32_combine join the checksums of pieces computed separately.

src/native/zlib_checksum.h

#pragma once
 
#include <zlib.h>
 
#include <string>
 
// CRC-32 (what gzip, ZIP and PNG store) and Adler-32 (what the zlib format stores). A running value
// continues across pieces, and two values combine into the checksum of the joined data, so pieces can
// be checksummed separately, in any order.
class Checksum {
public:
    static unsigned int crc32(const std::string& data) { return crc32Update(0, data); }
    static unsigned int adler32(const std::string& data) { return adler32Update(1, data); }
 
    static unsigned int crc32Update(unsigned int crc, const std::string& data) {
        return static_cast<unsigned int>(::crc32(crc, reinterpret_cast<const Bytef*>(data.data()), static_cast<uInt>(data.size())));
    }
 
    static unsigned int adler32Update(unsigned int adler, const std::string& data) {
        return static_cast<unsigned int>(::adler32(adler, reinterpret_cast<const Bytef*>(data.data()), static_cast<uInt>(data.size())));
    }
 
    // The checksum of first + second, from the two checksums and the length of second in bytes.
    static unsigned int crc32Combine(unsigned int first, unsigned int second, double secondLength) {
        return static_cast<unsigned int>(::crc32_combine(first, second, static_cast<z_off_t>(secondLength)));
    }
 
    static unsigned int adler32Combine(unsigned int first, unsigned int second, double secondLength) {
        return static_cast<unsigned int>(::adler32_combine(first, second, static_cast<z_off_t>(secondLength)));
    }
};

main.js

import { initNative, Checksum } from './native/zlib_checksum.h';
 
await initNative();
const hex = (value) => value.toString(16).padStart(8, '0');
const text = 'The quick brown fox jumps over the lazy dog';
console.log(`crc32 ${hex(await Checksum.crc32(text))}, adler32 ${hex(await Checksum.adler32(text))}`);
 
const [head, tail] = ['The quick brown fox ', 'jumps over the lazy dog'];
const running = await Checksum.crc32Update(await Checksum.crc32(head), tail);
const combined = await Checksum.crc32Combine(await Checksum.crc32(head), await Checksum.crc32(tail), tail.length);
const adler = await Checksum.adler32Combine(await Checksum.adler32(head), await Checksum.adler32(tail), tail.length);
console.log(`running crc32 ${hex(running)}, combined crc32 ${hex(combined)}, combined adler32 ${hex(adler)}`);

PRINTS

crc32 414fa339, adler32 5bdc0fda
running crc32 414fa339, combined crc32 414fa339, combined adler32 5bdc0fda

"Stream a file through gzip" writes its input with m.FS, which Linux does not have; the C++ takes paths, so it works unchanged on files in the app's storage. It runs on the WebAssembly page.

What is different on Linux

crossbind build -p linux links one .node addon per architecture into dist, next to a loader, dist/<name>.native.cjs, that require and import both load. A plain crossbind build skips it.
await initNative() once, then call the classes: calls are synchronous, and no Worker is involved.
The library and the C++ runtime are linked into the addon statically. It runs on glibc 2.28 or later (RHEL 8, Debian 10, Ubuntu 20.04 and newer), not on musl distributions such as Alpine.
There is no m.FS: the C++ reads real paths, and data such as GDAL_DATA or proj.db is copied to dist/data.
The build runs in Docker on any host, a Mac included. worker_threads are not supported yet.

Other platforms

zlib overview: the apps, every platform's setup and the packages.
zlib for WebAssembly: browsers, Node.js and edge runtimes.
zlib for Android: React Native apps on Android.
zlib for iOS: React Native apps on iOS.
zlib for macOS: native Node.js addons and Electron on macOS.
zlib for Windows: native Node.js addons on Windows.
zlib for WASI: command-line programs under wasmtime.

Facts on this page come from the port manifests in the repository and from what npm served on beta when the site was built. See the Libraries guide for the full consumer flow.

MORE LIBRARIES

cURL Expat GDAL GEOS GeoTIFF iconv LERC libjpeg-turbo libTIFF OpenSSL PROJ SpatiaLite SQLite WebP Zstandard