diff options
author | Yann Collet <cyan@fb.com> | 2019-05-30 16:45:21 (GMT) |
---|---|---|
committer | Yann Collet <cyan@fb.com> | 2019-05-30 16:45:21 (GMT) |
commit | 22adbb176afa46ebe1b799f7758381da8461bfe4 (patch) | |
tree | dd8502824a9d3704289b5b307fbea2f97b4098b7 /lib | |
parent | b2ba857a4f295af79f94207cab1b9febec14a380 (diff) | |
download | lz4-22adbb176afa46ebe1b799f7758381da8461bfe4.zip lz4-22adbb176afa46ebe1b799f7758381da8461bfe4.tar.gz lz4-22adbb176afa46ebe1b799f7758381da8461bfe4.tar.bz2 |
add more doc on in-place (de)compression
Diffstat (limited to 'lib')
-rw-r--r-- | lib/lz4.h | 30 |
1 files changed, 25 insertions, 5 deletions
@@ -473,21 +473,35 @@ LZ4LIB_STATIC_API void LZ4_attach_dictionary(LZ4_stream_t* workingStream, const * It's possible to have input and output sharing the same buffer, * for highly contrained memory environments. * In both cases, it requires input to lay at the end of the buffer, - * and buffer must have some margin, hence be larger than final size. + * and decompression to start at beginning of the buffer. + * Buffer size must feature some margin, hence be larger than final size. + * + * |<------------------------buffer----------------------------------->| + * |<------------compressed data---------->| + * |<-----------decompressed size------------------>| + * |<-----margin----->| * * This technique is more useful for decompression, * since decompressed size is typically larger, * and margin is mostly required to avoid stripe overflow, so it's short. * - * For compression though, margin must be able to cope with both + * In-place decompression will work inside any buffer + * which size is >= LZ4_DECOMPRESS_INPLACE_BUFFER_SIZE(decompressedSize). + * This presumes that decompressedSize > compressedSize. + * Otherwise, it means compression actually expanded data, + * which can happen when data is not compressible (already compressed, or encrypted), + * and it would be more efficient to store such data with a flag indicating it's not compressed. + * + * For compression, margin is larger, as it must be able to cope with both * history preservation, requiring input data to remain unmodified up to LZ4_DISTANCE_MAX, * and data expansion, which can happen when input is not compressible. - * As a consequence, buffer size requirements are much higher than average compressed size, - * hence memory savings are limited. + * As a consequence, buffer size requirements are much higher, + * and memory savings offered by in-place compression are more limited. * * There are ways to limit this cost for compression : * - Reduce history size, by modifying LZ4_DISTANCE_MAX. - * Lower values will also reduce compression ratio, except when input_size < LZ4_DISTANCE_MAX, + * Note that it is a compile-time constant, so all future compression will apply this parameter. + * Lower values will reduce compression ratio, except when input_size < LZ4_DISTANCE_MAX, * so it's a reasonable trick when inputs are known to be small. * - Require the compressor to deliver a "maximum compressed size". * When this size is < LZ4_COMPRESSBOUND(inputSize), then compression can fail, @@ -496,6 +510,12 @@ LZ4LIB_STATIC_API void LZ4_attach_dictionary(LZ4_stream_t* workingStream, const * and typically design a backup scheme to send data uncompressed. * The combination of both techniques can significantly reduce * the amount of margin required for in-place compression. + * + * In-place compression can work in any buffer + * which size is >= LZ4_COMPRESS_INPLACE_BUFFER_SIZE(maxCompressedSize) + * with maxCompressedSize == LZ4_COMPRESSBOUND(srcSize) for guaranteed compression success. + * This macro depends on both maxCompressedSize and LZ4_DISTANCE_MAX, + * so there are ways to reduce memory requirements by playing with them. */ #define LZ4_DECOMPRESS_INPLACE_MARGIN 32 |