LERC for Linux

v4.2.0Linux

LERC 4.2.0 for native Node.js addons on Linux, precompiled for x64 and arm64, glibc 2.28 or later as @crossbind/port-lerc-linux.

npm install @crossbind/port-lerc-linux@beta

npm ↗Port recipe and licence files ↗

Install

shell

npm install @crossbind/port-lerc-linux@beta
npm install --save-dev crossbind@beta @crossbind/core-embind-napi@beta

crossbind.config.js

import lercLinux from '@crossbind/port-lerc-linux/crossbind.config.js';
 
export default {
    dependencies: [lercLinux],
    paths: { config: import.meta.url },
};

shell

npx crossbind build -p linux

The addons and their loader land in dist; the Node.js playbook has the whole flow.

Usage

The examples the WebAssembly page runs, as Linux compiles them: the same headers and the same calls. They are checked on the WebAssembly build.

Compress heights to within 1 cm

LERC's main job: lerc_encode stores a float raster so that no value moves by more than the error you allow, and lerc_decode reads it back. The example checks the bound on every one of the 65,536 heights.

src/native/lerc_codec.h

#pragma once
 
#include <Lerc_c_api.h>
 
#include <algorithm>
#include <cmath>
#include <stdexcept>
#include <string>
#include <vector>
 
// One band of float32 heights, row by row from the top left, in and out of LERC. Bytes cross the
// binding as a byte string: one UTF-16 code unit (0-255) per byte.
class LercCodec {
public:
    static std::string version() {
        return std::to_string(LERC_VERSION_MAJOR) + "." + std::to_string(LERC_VERSION_MINOR) + "." + std::to_string(LERC_VERSION_PATCH);
    }
 
    // Every decoded height stays within maxError of the original; 0 keeps every bit.
    static std::u16string encode(const std::u16string& heights, int width, int height, double maxError) {
        const std::vector<float> values = toFloats(heights, width, height);
        const double bound = boundFor(values, maxError);
        unsigned int size = 0;
        check(lerc_computeCompressedSize(values.data(), kFloat, 1, width, height, 1, 0, nullptr, bound, &size), "sizing");
        std::vector<unsigned char> blob(size);
        unsigned int written = 0;
        check(lerc_encode(values.data(), kFloat, 1, width, height, 1, 0, nullptr, bound, blob.data(), size, &written), "encoding");
        return toUnits(blob.data(), written);
    }
 
    static std::u16string decode(const std::u16string& blob) {
        const std::vector<unsigned char> bytes = fromUnits(blob);
        const auto size = static_cast<unsigned int>(bytes.size());
        unsigned int info[11] = {};
        double range[3] = {};
        check(lerc_getBlobInfo(bytes.data(), size, info, range, 11, 3), "reading the header");
        const int width = static_cast<int>(info[3]);
        const int height = static_cast<int>(info[4]);
        if (info[1] != kFloat || info[2] != 1 || info[5] != 1) throw std::invalid_argument("this codec reads one band of float32");
        if (info[6] != info[3] * info[4]) throw std::invalid_argument("this blob has missing pixels: decode it with its mask");
        std::vector<float> values(static_cast<size_t>(width) * height);
        check(lerc_decode(bytes.data(), size, 0, nullptr, 1, width, height, 1, kFloat, values.data()), "decoding");
        return toUnits(reinterpret_cast<const unsigned char*>(values.data()), values.size() * sizeof(float));
    }
 
private:
    static constexpr unsigned int kFloat = 6;  // dt_float in Lerc_types.h
 
    // LERC quantizes in double precision and rounds back to float32, which can overshoot maxError by
    // half a float32 step (31 µm at 1,000 m). Like Esri's own sample, ask for a little less: one
    // float32 step at the largest height.
    static double boundFor(const std::vector<float>& values, double maxError) {
        if (maxError <= 0) return 0;
        float largest = 0;
        for (const float value : values) largest = std::max(largest, std::fabs(value));
        const double step = static_cast<double>(std::nextafter(largest, INFINITY)) - largest;
        return maxError > step ? maxError - step : 0;
    }
 
    static void check(lerc_status status, const char* step) {
        static const char* const names[] = {"ok", "failed", "wrong parameter", "buffer too small", "NaN", "uses noData", "dimensions too large"};
        if (status != 0) throw std::runtime_error(std::string("LERC failed ") + step + ": " + (status < 7 ? names[status] : "unknown error"));
    }
 
    static std::vector<float> toFloats(const std::u16string& units, int width, int height) {
        if (width <= 0 || height <= 0 || units.size() != static_cast<size_t>(width) * height * sizeof(float)) {
            throw std::invalid_argument("expected width * height float32 values");
        }
        std::vector<float> values(static_cast<size_t>(width) * height);
        auto* bytes = reinterpret_cast<unsigned char*>(values.data());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return values;
    }
 
    static std::vector<unsigned char> fromUnits(const std::u16string& units) {
        std::vector<unsigned char> bytes(units.size());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return bytes;
    }
 
    static std::u16string toUnits(const unsigned char* bytes, size_t size) {
        std::u16string units(size, u'\0');
        for (size_t i = 0; i < size; ++i) units[i] = bytes[i];
        return units;
    }
};

main.js

import { initNative, LercCodec } from './native/lerc_codec.h';
 
await initNative();
let seed = 42;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const heights = new Float32Array(width * height); // metres, row by row from the top left
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) {
        const dx = x - 128;
        const dy = y - 128;
        heights[y * width + x] = 1500 - (dx * dx + dy * dy) / 70 + random(1000) / 1000; // a hill, rough to 1 m
    }
}
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const toBytes = (text) => Uint8Array.from(text, (unit) => unit.charCodeAt(0));
 
const blob = await LercCodec.encode(toText(new Uint8Array(heights.buffer)), width, height, 0.01);
const decoded = new Float32Array(toBytes(await LercCodec.decode(blob)).buffer);
let worst = 0;
for (let i = 0; i < heights.length; i += 1) worst = Math.max(worst, Math.abs(decoded[i] - heights[i]));
console.log(`LERC ${await LercCodec.version()}: ${heights.byteLength} B of float32 heights -> ${blob.length} B`);
console.log(`largest error ${worst.toFixed(4)} m, within 1 cm: ${worst <= 0.01}`);

PRINTS

LERC 4.2.0: 262144 B of float32 heights -> 94775 B
largest error 0.0099 m, within 1 cm: true

Read a blob's header without decoding it

lerc_getBlobInfo answers from the header alone: size, data type, bands, valid pixels, the value range and the largest error the encoder allowed. A tile viewer uses it to size its buffers, or to skip an empty tile, before decoding. The blob comes from the codec in the first example, which asks for one float32 step less than 10 cm.

src/native/lerc_blob_info.h

#pragma once
 
#include <Lerc_c_api.h>
 
#include <cstdio>
#include <stdexcept>
#include <string>
 
// What a LERC blob holds, read from its header without decoding a pixel. Bytes cross the binding as
// a byte string: one UTF-16 code unit (0-255) per byte.
class LercBlobInfo {
public:
    // {"codec","width","height","depth","bands","type","validPixels","masks","blobSize","zMin","zMax","maxZErrorUsed"}
    static std::string read(const std::u16string& blob) {
        std::string bytes(blob.size(), '\0');
        for (size_t i = 0; i < blob.size(); ++i) {
            if (blob[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<char>(blob[i]);
        }
        unsigned int info[11] = {};  // version, type, depth, width, height, bands, valid pixels, blob size, masks, depth, noData bands
        double range[3] = {};        // zMin, zMax, and the largest error the encoder allowed
        const lerc_status status = lerc_getBlobInfo(reinterpret_cast<const unsigned char*>(bytes.data()), static_cast<unsigned int>(bytes.size()), info, range, 11, 3);
        if (status != 0) throw std::runtime_error("not a LERC blob (status " + std::to_string(status) + ")");
        static const char* const types[] = {"int8", "uint8", "int16", "uint16", "int32", "uint32", "float32", "float64"};
        const std::string codec = info[0] == 0 ? "Lerc1" : "Lerc2 v" + std::to_string(info[0]);
        return "{\"codec\":\"" + codec + "\",\"width\":" + std::to_string(info[3]) + ",\"height\":" + std::to_string(info[4]) + ",\"depth\":" + std::to_string(info[2]) +
               ",\"bands\":" + std::to_string(info[5]) + ",\"type\":\"" + (info[1] < 8 ? types[info[1]] : "unknown") + "\",\"validPixels\":" + std::to_string(info[6]) +
               ",\"masks\":" + std::to_string(info[8]) + ",\"blobSize\":" + std::to_string(info[7]) + ",\"zMin\":" + number(range[0]) + ",\"zMax\":" + number(range[1]) +
               ",\"maxZErrorUsed\":" + number(range[2]) + "}";
    }
 
private:
    // 17 significant digits read back as the same double.
    static std::string number(double value) {
        char text[40];
        std::snprintf(text, sizeof text, "%.17g", value);
        return text;
    }
};

main.js

import { initNative, LercBlobInfo } from './native/lerc_blob_info.h';
import { LercCodec } from './native/lerc_codec.h';
 
await initNative();
let seed = 42;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const heights = new Float32Array(width * height);
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) {
        const dx = x - 128;
        const dy = y - 128;
        heights[y * width + x] = 1500 - (dx * dx + dy * dy) / 70 + random(1000) / 1000;
    }
}
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const blob = await LercCodec.encode(toText(new Uint8Array(heights.buffer)), width, height, 0.1);
 
const info = JSON.parse(await LercBlobInfo.read(blob));
console.log(`${info.codec}: ${info.width}x${info.height}, ${info.bands} band of ${info.type}, ${info.validPixels} valid pixels, ${info.blobSize} B`);
console.log(`heights ${info.zMin.toFixed(3)} to ${info.zMax.toFixed(3)} m, stored within ${info.maxZErrorUsed.toFixed(6)} m`);

PRINTS

Lerc2 v6: 256x256, 1 band of float32, 65536 valid pixels, 67591 B
heights 1032.268 to 1500.945 m, stored within 0.099878 m

Keep integer data exact

Class maps and sensor counts must not change at all. With a maxZError of 0, LERC stores integers exactly (it raises 0 to 0.5, which rounds back to the same whole number) and float32 bit for bit. Here, a 12-bit sensor band in 16-bit pixels.

src/native/lerc_lossless.h

#pragma once
 
#include <Lerc_c_api.h>
 
#include <stdexcept>
#include <string>
#include <vector>
 
// Lossless LERC for any of its eight data types, bands one after another, each row by row. A
// maxZError of 0 keeps every value: LERC raises it to 0.5 for integers, which rounds back to the
// same whole number, and keeps float32 and float64 bit for bit. Bytes cross the binding as a byte
// string: one UTF-16 code unit (0-255) per byte.
class LercLossless {
public:
    // type: "int8", "uint8", "int16", "uint16", "int32", "uint32", "float32" or "float64".
    static std::u16string encode(const std::u16string& values, const std::string& type, int width, int height, int bands) {
        const unsigned int dataType = typeCode(type);
        if (width <= 0 || height <= 0 || bands <= 0 || values.size() != static_cast<size_t>(width) * height * bands * kSizes[dataType]) {
            throw std::invalid_argument("expected width * height * bands values of " + type);
        }
        const std::vector<unsigned char> data = fromUnits(values);
        unsigned int size = 0;
        check(lerc_computeCompressedSize(data.data(), dataType, 1, width, height, bands, 0, nullptr, 0.0, &size), "sizing");
        std::vector<unsigned char> blob(size);
        unsigned int written = 0;
        check(lerc_encode(data.data(), dataType, 1, width, height, bands, 0, nullptr, 0.0, blob.data(), size, &written), "encoding");
        return toUnits(blob.data(), written);
    }
 
    // The values in the blob's own type, band after band.
    static std::u16string decode(const std::u16string& blob) {
        const std::vector<unsigned char> bytes = fromUnits(blob);
        const auto size = static_cast<unsigned int>(bytes.size());
        unsigned int info[11] = {};
        double range[3] = {};
        check(lerc_getBlobInfo(bytes.data(), size, info, range, 11, 3), "reading the header");
        const unsigned int dataType = info[1];
        const int depth = static_cast<int>(info[2]);
        const int width = static_cast<int>(info[3]);
        const int height = static_cast<int>(info[4]);
        const int bands = static_cast<int>(info[5]);
        if (dataType > 7) throw std::invalid_argument("unknown LERC data type");
        if (info[6] != info[3] * info[4]) throw std::invalid_argument("this blob has missing pixels: decode it with its mask");
        std::vector<unsigned char> values(static_cast<size_t>(depth) * width * height * bands * kSizes[dataType]);
        check(lerc_decode(bytes.data(), size, 0, nullptr, depth, width, height, bands, dataType, values.data()), "decoding");
        return toUnits(values.data(), values.size());
    }
 
private:
    static constexpr size_t kSizes[] = {1, 1, 2, 2, 4, 4, 4, 8};  // bytes per value, in Lerc_types.h order
 
    static unsigned int typeCode(const std::string& type) {
        static const char* const names[] = {"int8", "uint8", "int16", "uint16", "int32", "uint32", "float32", "float64"};
        for (unsigned int code = 0; code < 8; ++code) {
            if (type == names[code]) return code;
        }
        throw std::invalid_argument("unknown type " + type);
    }
 
    static void check(lerc_status status, const char* step) {
        static const char* const names[] = {"ok", "failed", "wrong parameter", "buffer too small", "NaN", "uses noData", "dimensions too large"};
        if (status != 0) throw std::runtime_error(std::string("LERC failed ") + step + ": " + (status < 7 ? names[status] : "unknown error"));
    }
 
    static std::vector<unsigned char> fromUnits(const std::u16string& units) {
        std::vector<unsigned char> bytes(units.size());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return bytes;
    }
 
    static std::u16string toUnits(const unsigned char* bytes, size_t size) {
        std::u16string units(size, u'\0');
        for (size_t i = 0; i < size; ++i) units[i] = bytes[i];
        return units;
    }
};

main.js

import { initNative, LercLossless } from './native/lerc_lossless.h';
 
await initNative();
let seed = 7;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const band = new Uint16Array(width * height); // a gradient plus sensor noise, 12-bit values
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) band[y * width + x] = 1800 + ((x + y) >> 1) + random(64);
}
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const toBytes = (text) => Uint8Array.from(text, (unit) => unit.charCodeAt(0));
 
const blob = await LercLossless.encode(toText(new Uint8Array(band.buffer)), 'uint16', width, height, 1);
const back = new Uint16Array(toBytes(await LercLossless.decode(blob)).buffer);
console.log(`uint16, ${width}x${height}: ${band.byteLength} B -> ${blob.length} B`);
console.log(`identical: ${back.length === band.length && back.every((value, i) => value === band[i])}`);

PRINTS

uint16, 256x256: 131072 B -> 59091 B
identical: true

Leave out pixels that have no data

Rasters mark gaps with a NoData value such as -9999. Stored as a height, it stretches the value range of every block it lands in. Passed as the validity mask (pValidBytes in lerc_encode and lerc_decode), it costs at most a bit per pixel, and decoding hands the mask back.

src/native/lerc_nodata.h

#pragma once
 
#include <Lerc_c_api.h>
 
#include <algorithm>
#include <cmath>
#include <stdexcept>
#include <string>
#include <vector>
 
// One band of float32 heights with gaps. Pixels equal to noData (NaN too, when noData is NaN) go into
// LERC's validity mask, a bit per pixel, instead of into the values; decoding puts noData back. Bytes
// cross the binding as a byte string: one UTF-16 code unit (0-255) per byte.
class LercNoData {
public:
    static std::u16string encode(const std::u16string& heights, int width, int height, double noData, double maxError) {
        const std::vector<float> values = toFloats(heights, width, height);
        std::vector<unsigned char> valid(values.size());
        float largest = 0;
        for (size_t i = 0; i < values.size(); ++i) {
            valid[i] = missing(values[i], noData) ? 0 : 1;
            if (valid[i]) largest = std::max(largest, std::fabs(values[i]));
        }
        // LERC rounds back to float32, which can overshoot maxError by half a float32 step: ask for one step less.
        const double step = static_cast<double>(std::nextafter(largest, INFINITY)) - largest;
        const double bound = maxError > step ? maxError - step : 0;
        unsigned int size = 0;
        check(lerc_computeCompressedSize(values.data(), kFloat, 1, width, height, 1, 1, valid.data(), bound, &size), "sizing");
        std::vector<unsigned char> blob(size);
        unsigned int written = 0;
        check(lerc_encode(values.data(), kFloat, 1, width, height, 1, 1, valid.data(), bound, blob.data(), size, &written), "encoding");
        return toUnits(blob.data(), written);
    }
 
    static std::u16string decode(const std::u16string& blob, double noData) {
        const std::vector<unsigned char> bytes = fromUnits(blob);
        const auto size = static_cast<unsigned int>(bytes.size());
        unsigned int info[11] = {};
        double range[3] = {};
        check(lerc_getBlobInfo(bytes.data(), size, info, range, 11, 3), "reading the header");
        const int width = static_cast<int>(info[3]);
        const int height = static_cast<int>(info[4]);
        if (info[1] != kFloat || info[2] != 1 || info[5] != 1) throw std::invalid_argument("this reads one band of float32");
        std::vector<float> values(static_cast<size_t>(width) * height);
        std::vector<unsigned char> valid(values.size());
        check(lerc_decode(bytes.data(), size, 1, valid.data(), 1, width, height, 1, kFloat, values.data()), "decoding");
        for (size_t i = 0; i < values.size(); ++i) {
            if (!valid[i]) values[i] = static_cast<float>(noData);
        }
        return toUnits(reinterpret_cast<const unsigned char*>(values.data()), values.size() * sizeof(float));
    }
 
private:
    static constexpr unsigned int kFloat = 6;  // dt_float in Lerc_types.h
 
    static bool missing(float value, double noData) { return std::isnan(noData) ? std::isnan(value) : value == static_cast<float>(noData); }
 
    static void check(lerc_status status, const char* step) {
        static const char* const names[] = {"ok", "failed", "wrong parameter", "buffer too small", "NaN", "uses noData", "dimensions too large"};
        if (status != 0) throw std::runtime_error(std::string("LERC failed ") + step + ": " + (status < 7 ? names[status] : "unknown error"));
    }
 
    static std::vector<float> toFloats(const std::u16string& units, int width, int height) {
        if (width <= 0 || height <= 0 || units.size() != static_cast<size_t>(width) * height * sizeof(float)) {
            throw std::invalid_argument("expected width * height float32 values");
        }
        std::vector<float> values(static_cast<size_t>(width) * height);
        auto* bytes = reinterpret_cast<unsigned char*>(values.data());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return values;
    }
 
    static std::vector<unsigned char> fromUnits(const std::u16string& units) {
        std::vector<unsigned char> bytes(units.size());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return bytes;
    }
 
    static std::u16string toUnits(const unsigned char* bytes, size_t size) {
        std::u16string units(size, u'\0');
        for (size_t i = 0; i < size; ++i) units[i] = bytes[i];
        return units;
    }
};

main.js

import { initNative, LercNoData } from './native/lerc_nodata.h';
import { LercCodec } from './native/lerc_codec.h';
 
await initNative();
let seed = 42;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const heights = new Float32Array(width * height);
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) {
        const dx = x - 128;
        const dy = y - 128;
        heights[y * width + x] = 1500 - (dx * dx + dy * dy) / 70 + random(1000) / 1000;
    }
}
for (let i = 0; i < heights.length; i += 1) if (random(50) === 0) heights[i] = -9999; // the survey missed 2% of its points
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const toBytes = (text) => Uint8Array.from(text, (unit) => unit.charCodeAt(0));
const text = toText(new Uint8Array(heights.buffer));
 
const asHeight = await LercCodec.encode(text, width, height, 0.01); // the codec from the first example
const blob = await LercNoData.encode(text, width, height, -9999, 0.01);
const back = new Float32Array(toBytes(await LercNoData.decode(blob, -9999)).buffer);
let gaps = 0;
let worst = 0;
for (let i = 0; i < heights.length; i += 1) {
    if (heights[i] === -9999) gaps += back[i] === -9999 ? 1 : 0;
    else worst = Math.max(worst, Math.abs(back[i] - heights[i]));
}
console.log(`-9999 stored as a height: ${asHeight.length} B`);
console.log(`-9999 as missing pixels: ${blob.length} B`);
console.log(`${gaps} gaps come back as -9999, largest error elsewhere ${worst.toFixed(4)} m`);

PRINTS

-9999 stored as a height: 147759 B
-9999 as missing pixels: 98308 B
1273 gaps come back as -9999, largest error elsewhere 0.0099 m

What is different on Linux

crossbind build -p linux links one .node addon per architecture into dist, next to a loader, dist/<name>.native.cjs, that require and import both load. A plain crossbind build skips it.
await initNative() once, then call the classes: calls are synchronous, and no Worker is involved.
The library and the C++ runtime are linked into the addon statically. It runs on glibc 2.28 or later (RHEL 8, Debian 10, Ubuntu 20.04 and newer), not on musl distributions such as Alpine.
There is no m.FS: the C++ reads real paths, and data such as GDAL_DATA or proj.db is copied to dist/data.
The build runs in Docker on any host, a Mac included. worker_threads are not supported yet.

Other platforms

LERC overview: the apps, every platform's setup and the packages.
LERC for WebAssembly: browsers, Node.js and edge runtimes.
LERC for Android: React Native apps on Android.
LERC for iOS: React Native apps on iOS.
LERC for macOS: native Node.js addons and Electron on macOS.
LERC for Windows: native Node.js addons on Windows.
LERC for WASI: command-line programs under wasmtime.

Facts on this page come from the port manifests in the repository and from what npm served on beta when the site was built. See the Libraries guide for the full consumer flow.

MORE LIBRARIES

cURL Expat GDAL GEOS GeoTIFF iconv libjpeg-turbo libTIFF OpenSSL PROJ SpatiaLite SQLite WebP zlib Zstandard