summaryrefslogtreecommitdiffstats
path: root/doc
diff options
context:
space:
mode:
authorYann Collet <cyan@fb.com>2019-05-30 16:45:21 (GMT)
committerYann Collet <cyan@fb.com>2019-05-30 16:45:21 (GMT)
commit22adbb176afa46ebe1b799f7758381da8461bfe4 (patch)
treedd8502824a9d3704289b5b307fbea2f97b4098b7 /doc
parentb2ba857a4f295af79f94207cab1b9febec14a380 (diff)
downloadlz4-22adbb176afa46ebe1b799f7758381da8461bfe4.zip
lz4-22adbb176afa46ebe1b799f7758381da8461bfe4.tar.gz
lz4-22adbb176afa46ebe1b799f7758381da8461bfe4.tar.bz2
add more doc on in-place (de)compression
Diffstat (limited to 'doc')
-rw-r--r--doc/lz4_manual.html30
1 files changed, 25 insertions, 5 deletions
diff --git a/doc/lz4_manual.html b/doc/lz4_manual.html
index 0530966..091f537 100644
--- a/doc/lz4_manual.html
+++ b/doc/lz4_manual.html
@@ -361,21 +361,35 @@ int LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream);
It's possible to have input and output sharing the same buffer,
for highly contrained memory environments.
In both cases, it requires input to lay at the end of the buffer,
- and buffer must have some margin, hence be larger than final size.
+ and decompression to start at beginning of the buffer.
+ Buffer size must feature some margin, hence be larger than final size.
+
+ |<------------------------buffer----------------------------------->|
+ |<------------compressed data---------->|
+ |<-----------decompressed size------------------>|
+ |<-----margin----->|
This technique is more useful for decompression,
since decompressed size is typically larger,
and margin is mostly required to avoid stripe overflow, so it's short.
- For compression though, margin must be able to cope with both
+ In-place decompression will work inside any buffer
+ which size is >= LZ4_DECOMPRESS_INPLACE_BUFFER_SIZE(decompressedSize).
+ This presumes that decompressedSize > compressedSize.
+ Otherwise, it means compression actually expanded data,
+ which can happen when data is not compressible (already compressed, or encrypted),
+ and it would be more efficient to store such data with a flag indicating it's not compressed.
+
+ For compression, margin is larger, as it must be able to cope with both
history preservation, requiring input data to remain unmodified up to LZ4_DISTANCE_MAX,
and data expansion, which can happen when input is not compressible.
- As a consequence, buffer size requirements are much higher than average compressed size,
- hence memory savings are limited.
+ As a consequence, buffer size requirements are much higher,
+ and memory savings offered by in-place compression are more limited.
There are ways to limit this cost for compression :
- Reduce history size, by modifying LZ4_DISTANCE_MAX.
- Lower values will also reduce compression ratio, except when input_size < LZ4_DISTANCE_MAX,
+ Note that it is a compile-time constant, so all future compression will apply this parameter.
+ Lower values will reduce compression ratio, except when input_size < LZ4_DISTANCE_MAX,
so it's a reasonable trick when inputs are known to be small.
- Require the compressor to deliver a "maximum compressed size".
When this size is < LZ4_COMPRESSBOUND(inputSize), then compression can fail,
@@ -384,6 +398,12 @@ int LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream);
and typically design a backup scheme to send data uncompressed.
The combination of both techniques can significantly reduce
the amount of margin required for in-place compression.
+
+ In-place compression can work in any buffer
+ which size is >= LZ4_COMPRESS_INPLACE_BUFFER_SIZE(maxCompressedSize)
+ with maxCompressedSize == LZ4_COMPRESSBOUND(srcSize) for guaranteed compression success.
+ This macro depends on both maxCompressedSize and LZ4_DISTANCE_MAX,
+ so there are ways to reduce memory requirements by playing with them.
</p></pre><BR>