ALP: Adaptive Lossless Floating-Point Encoding in Apache Parquet
Categories:
Apache Parquet has added the Adaptive Lossless floating-Point (ALP) Encoding – a new lightweight floating-point encoding with compression ratios similar to zstd, much faster decompression, random-access support, and GPU- and SIMD-friendly decoding.
ALP works best for decimal values stored as floating-point types (32-bit FLOAT and 64-bit DOUBLE), such as
- Monetary values (exchange rates, public funds, stocks, prices, etc.) – e.g.,
1.2345or22.03 - Geographic coordinates (longitude/latitude) – e.g.,
42.3584,-71.0598 - Scientific measurements (temperature, pressure, speed, degrees, etc.) – e.g.,
-273.15,9.81,3.14159
ALP is not suitable for data that uses a wide range of exponents or a large
number of significant digits, such as vector embeddings, which typically span
the full floating-point range. Such data can continue to use existing Parquet
features such as PLAIN or BYTE_STREAM_SPLIT encoding followed by
general-purpose compression like ZSTD.
Decimal values can be stored with Parquet’s DECIMAL logical type, but that
type requires the precision and scale to be known and declared up front and
cannot store values outside that range. For this reason, systems commonly
store decimal values as FLOAT or DOUBLE when the exact shape of their data
is not known beforehand. For example,
JavaScript’s only* number type is DOUBLE, common data science tools such as
pandas infer float64 for decimal-looking values, and NumPy has no decimal dtype at all.
* JavaScript also has BigInt, but it can only represent integers.
Why ALP?
Encoding floating-point data is a complicated engineering problem due to the nature of floating-point values. They do not exactly represent most real values. This leads to rounding errors that prevent using existing lightweight encodings like Delta and Frame of Reference (FOR).
Prior to ALP, BYTE_STREAM_SPLIT was the only non-dictionary alternative to
PLAIN for FLOAT/DOUBLE values in Parquet. It does not reduce the size of the
data but can improve the compression ratio and speed when a heavyweight
compressor is used afterwards.
Heavyweight compression effectively decreases the data size, but at the cost of:
- Decode speed – decompression speed is often the bottleneck in data access.
- Random access – reading one value requires decoding an entire data page containing potentially thousands of other values.
- Data dependence – variable-length compression means that decoding a value requires decoding previous values, making it hard to parallelize with modern hardware such as SIMD instructions and GPUs.
ALP is designed to solve all three of these problems for common data patterns, while achieving a similar compression ratio to heavyweight compression.
Parquet applies an encoding first, then an optional compression codec as a
separate step. The charts below compare the PLAIN and BYTE_STREAM_SPLIT
encodings followed by ZSTD compression with the ALP encoding and no
additional compression. Users can expect ALP to decode 10x faster and
retrieve individual values thousands of times faster, with a slightly lower
compression ratio and slightly faster compression.1
PLAIN+ZSTD and BYTE_STREAM_SPLIT+ZSTD (each encoding followed by per-page ZSTD compression), and ALP (no compression codec), across 30 datasets on three machines. Higher is better.
Random-access speed is measured by decoding 100 deterministic, uniformly distributed rows from city_temperature_f.Note that these numbers are for the pre-release Rust implementation of ALP, and
we expect performance to improve as implementations are optimized and tuned.
Even so, ALP is already faster than zstd in many cases, despite years of
optimization work on zstd implementations. We also measured similar
improvements for the C++ implementation.
Technical Overview
ALP takes advantage of a common pattern: many values stored as FLOAT or
DOUBLE originated as decimal numbers with relatively few digits, such as
prices or measurements. This section explains the intuition behind ALP and
then covers the encoding and decoding pipelines in more detail.
ALP encodes floating-point values in batches called “vectors” of between 8
and 32K values (e.g., 1024). Each value in a vector is encoded as an
integer, and the vector as a whole stores two more integers shared by all its
values: an “exponent” (e) and a “factor” (f). Each vector can use a
different exponent and factor, and how they are chosen is explained below. The
original value is recovered by computing
value = encoded × 10f × 10-e
This calculation uses floating-point arithmetic, which rounds to the nearest
representable value and thus may not reproduce the original value exactly. When
that happens, ALP stores the original full-precision value separately as an
“exception”, keeping the encoding lossless. Special values such as NaN,
±Infinity, and -0.0 are also stored as exceptions.
Within each vector, the encoded values are stored by subtracting the lowest value (frame of reference) and then bit-packing to a fixed width. Exceptions are stored directly after the encoded array. The layout of each ALP vector is shown below.
Since each value is stored as a bit-packed integer of a fixed width, locating an arbitrary row requires only computing the offset of its encoded bits. Applying the frame of reference, exponent, and factor to that integer recovers the original floating-point value. Finally, the exception indices are checked for the target row, and if an exception is present, its value is returned instead.
Example
Consider encoding the value 8.0605, which cannot be exactly
represented in IEEE 754. It is stored as
the 32-bit floating-point number 8.06050014495849609375. It can also be
encoded as 80605 with exponent e = 8 and factor f = 4. Applying
the recovery formula with 32-bit floating-point arithmetic, which rounds after
each multiplication, yields
80605 × 104 × 10-8 → 8.06050014495849609375 (FLOAT)
This is the nearest representable FLOAT to 8.0605 and matches the original
stored floating-point value exactly. However, if the original value had been
8.0605123 (stored as the 32-bit value 8.060512542724609375), the encoded
value would still be 80605 and the decoded value still
8.06050014495849609375, which differs from the original. That value would
therefore be stored as an exception.
Picking the exponent and factor well is key to ALP’s performance. Each Parquet writer is free to choose them for each vector using any algorithm. The Parquet specification provides an example sampling-based algorithm that aims to minimize the encoded size. Typically, the exponent is chosen to capture most decimal digits in the vector while minimizing exceptions, and the factor is chosen to remove as many trailing zeros as possible.
Example
Assuming some value in the vector requires e = 8, it is valid to encode
0.0123, 0.0245, and 0.0201 with multiple factor choices:
e = 8, f = 0:1230000,2450000,2010000e = 8, f = 4:123,245,201
The second choice is better: it yields smaller encoded values, which require fewer bits to store.
Finally, ALP subtracts the minimum encoded value (the frame of reference) from every encoded value before bit-packing, further reducing the bits required.
Example
The values above require only 7 bits each after subtracting the frame of reference:
- Input values:
123,245, and201(8bits per value) - Minimum value (frame of reference):
123 - Final bit-packed values:
0,122, and78(7bits per value)
The encoding pipeline is straightforward, as shown in the following example of encoding a vector:
1024 64-bit floating-point values using ALP.To encode this vector, the parameters e = 4 and f =
3 are chosen first. Then the values are transformed to integers using the
formula encoded = round(value × 104 × 10-3). Each integer is
checked by reversing the transformation with decoded = encoded × 103 × 10-4.
Values that do not round-trip, such as 8.0605123 (which decodes to 8.1), are
stored in the exception array. The minimum value across the vector, 3335,
becomes the frame of reference and is subtracted from each integer, and the
resulting deltas are bit-packed using 15 bits.
In this example, ALP uses 1920 bytes for the bit-packed deltas, plus a
13-byte vector header and space for exceptions. PLAIN uses 8192 bytes for
the same 1024 values. This comparison excludes page-level metadata for both
encodings. See the ALP Encoding specification for more details on how the
parameters are chosen and how rounding and exception handling work.
Decoding a vector requires similar steps, but in reverse, as shown below.
1024 values back to floating-point values using ALP.First, the bit-packed deltas are unpacked, and the original values are computed
by original = (3335 + delta) × 103 ×
10-4. Then any exceptions are “patched” by overwriting the
output array at the exception positions with the exception values.
Acknowledgements
ALP was first published in a SIGMOD 2024 paper by Azim Afroozeh, Leonardo Kuffó, and Peter Boncz from the Database Architectures Group at CWI. The Vortex and Lance formats adopted ALP early, demonstrating its benefits in industrial applications. In late 2025, the community began the standardization process. Along with the authors of this blog, many community members contributed, including Divjot Arora, Arnav Balyan, Devan Benz, Ryan Blue, Alkis Evlogimenos, Vinoo Ganesh, Adrian Garcia Badaracco, Curt Hagenlocher, Amogh Jahagirdar, Micah Kornfield, Robert Kruszewski, Julien Le Dem, Kevin Liu, Steve Loughran, Ismaël Mejía, mwish, Antoine Pitrou, Adam Reeve, Ed Seidl, Russell Spitzer, Matt Topol, Jeffrey Vo, Daniel Weeks, Gang Wu, and Zehua Zou.
Ecosystem Adoption
The encoding was released as part of parquet-format 2.14.0 in September 2026.
ALP is already supported in at least one major open source implementation (the
parquet 60.0.0 Rust crate), and we expect other Parquet
implementations to add support in the coming months. Please check the
Implementation Status page for the current state of support.
You can also try it today on your own datasets using the tool in the ALP benchmark repository.
Conclusion
ALP brings fast, parallelizable decoding and practical random access to floating-point data in a standard form that any Parquet implementation can read once it adds support for the encoding. Its addition is one more example of Apache Parquet evolving to meet the needs of modern data systems.
As with all additions to Parquet, this was a community endeavor, with many individuals and vendors working together to agree on a common standard and produce a well-documented specification and multiple reference implementations. We expect ALP to be widely adopted in the Parquet ecosystem over the coming years.
Resources
The code and instructions to reproduce these results and try ALP on your own Parquet datasets are in the alp_benchmark repository, which uses the Rust Parquet implementation. ↩︎