Meta open‑sources OpenZL
Meta published OpenZL, a format‑aware compression framework that aims to compress data without loss while improving developer efficiency across pipelines. The open‑source announcement was shared on social, positioning OpenZL as a tooling improvement for teams handling diverse data formats. (x.com)
Data compression usually treats a file as one long stream of bytes. Meta said OpenZL tries a different approach: describe the file’s structure first, then compress it without losing any data. (engineering.fb.com) Meta published OpenZL on October 6, 2025 through its Engineering blog and a public GitHub repository under the `facebook/openzl` project. The repository showed about 3,000 stars and 140 forks when accessed, and Meta said the software was already used in production inside the company. (engineering.fb.com, github.com) The basic idea is to tell the compressor what kind of data it is looking at, the way a shipping label tells a sorter what is inside a box. Meta said OpenZL takes a format description or preset, runs an offline trainer to build a compression configuration, and stores a decode recipe inside each frame so one universal decompressor can read the result. (engineering.fb.com, openzl.org) That matters for structured binary data such as databases, machine learning datasets, network formats, and other files with repeated fields, fixed ranges, or typed columns. Meta said general-purpose compressors leave gains on the table in those cases because they have to guess at patterns that a schema can spell out directly. (engineering.fb.com, openzl.org) OpenZL also ships with a format-description tool called Simple Data Description Language, or SDDL. Its documentation says engineers can mark parts of a file as types such as bytes, integers, or floating-point numbers, so OpenZL can split similar values into streams and compress them more effectively. (openzl.org, openzl.org) Meta framed the project as a follow-on to lessons from Zstandard, the compression format it introduced in 2016. In the OpenZL launch post, Meta said further gains inside a general-purpose format were getting harder, while custom compressors created maintenance, auditing, and patching burdens for every separate decoder. (engineering.fb.com) The tradeoff is that OpenZL is not pitched as a drop-in replacement for every compressed file. The GitHub README says the project is under active development, and that the application programming interface, compressed format, and included codecs and graphs are all subject to change as the software matures. (github.com) Meta still promised some compatibility guardrails for tagged releases. The README says files compressed with a release-tagged version should remain decompressible by newer releases for at least the next several years, while new releases should also be able to generate frames compatible with at least the previous release. (github.com) On OpenZL’s public site, Meta compared one sample from the Silesia Compression Corpus against Zstandard and XZ, reporting a 2.06 compression ratio for OpenZL versus 1.31 for Zstandard at level 3 and 1.64 for XZ at level 9, with OpenZL compressing at 203 megabytes per second in that example. Those are vendor-published numbers, not an independent benchmark across workloads. (openzl.org) For teams that move large, repetitive datasets through training, storage, and analytics pipelines, Meta’s pitch is straightforward: keep one decompressor, add format knowledge, and spend less time building a separate compressor for every file type. (engineering.fb.com, github.com)
