| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
Now it's only about LZ4F_compressFrame(),
and it's only about the Compression Context
(not the hash tables, which used to be integrated).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
as noted by @yixiutt, the temporary buffer inside lz4frame compression
is being invalidated at end of compression,
forcing it to be re-allocated at next compression job.
This shouldn't be necessary.
This change was introduced in #236, as a way to fix #232,
but neither the issue is explained, nor why the patch fixes it.
This patch revert to previous behavior,
where temporary buffer is kept between compression calls.
This results in a net reduction of allocation workload.
Additionally, the temporary buffer should only need malloc(),
not calloc(), thus saving initialization.
This diff implements both changes.
Long fuzzer tests will be run to ensure that no sanitizer warning get triggered.
Additionally :
fixed a minor ubsan warning in LZ4F_decompress().
|
|
|
|
|
|
|
|
|
|
|
| |
lz4frame favors the faster prefix mode
when decompressing a frame with linked blocks.
This significantly improved CLI decompression on files compressed with -BD4 setting.
On my laptop, decompressing `enwik9` went from 0.89s to 0.52s.
This improvement is only for linked blocks.
It's more visible for small block sizes.
|
|
|
|
|
| |
on command line, using existing long command --no-frame-crc.
Note : it's effectively more than that, since _all_ checksums are disabled.
|
| |
|
|\
| |
| | |
Support for Custom Memory managers
|
| | |
|
| | |
|
| |\ |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
for compression context only for the time being,
using LZ4F_createCompressionContext_advanced().
Added basic test in frametest.c
|
| |/
|/|
| |
| |
| |
| |
| | |
fix #977
fseek() doesn't work for pipe,
switch to "read and forget" mode in such case.
|
| | |
|
|/
|
|
|
| |
and proper documentation.
Also : updated manual
|
|
|
|
| |
one less argument
|
|
|
|
|
|
|
|
|
| |
* replace assert with test for LZ4F_uncompressedUpdate
* update documentation to incldue correct docstring
* remove unecessary entry point
* remove compress_linked_block_mode from fuzzing test
Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
|
|
|
|
| |
Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
|
|
|
|
|
|
|
|
|
|
|
| |
when the block mode changes a flush is executed, to prevent
mixing compressed and uncompressed data.
Prior to this commit dstStart, dstPtr, dstCapacity
where not updated to include the offset from bytesWritten.
For inputs > blockSize this meant the flushed data was
overwritten.
Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
|
|
|
|
| |
add static dependency to examples
|
|
|
|
| |
Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
|
|
|
|
|
|
| |
add a fuzzing test for uncompressed frame api
Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
|
|
|
|
|
|
| |
This commit fixes the review findings
Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
|
|
|
|
|
|
|
|
| |
new method `uncompressed_update` allows to insert blocks without
compression into the lz4 stream.
The usage is documented in the frameCompress example
Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
|
|
|
|
|
|
| |
and removed prefix from RETURN_ERROR(_IF)
to improve brevity
as it's just a local macro (no bleeding in user's namespace).
|
| |
|
| |
|
|
|
|
| |
created a new error code in the process : LZ4F_ERROR_compressionState_uninitialized
|
| |
|
|
|
|
|
|
|
| |
ensure that `frametest` works fine with these values,
notably with low LZ4_MEMORY_USAGE (dict test notably)
following suggestions from @t-mat at #1016
|
| |
|
|
|
|
|
|
|
| |
Ensure that the memory block we're trying
to reuse is large enough for the new state.
Fixes #974
|
|
|
|
|
| |
ensure `dictBase` is only used
when there is an actual dictionary content.
|
|
|
|
|
|
| |
now line 912
by ensuring pointer arithmetic is only performed
if there is a reason for an internal buffer to be used.
|
| |
|
|
|
|
|
| |
This is now explicitly documented and asserted.
fix #927
|
|
|
|
| |
fix one (rare & complex) issue discovered by this test
|
| |
|
|
|
|
| |
memcpy() on NULL is UB, even if length is 0.
|
|
|
|
|
| |
detected by scan-build and cppcheck
fix #786
|
|
|
|
| |
properly track history
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
EndMark, the 4-bytes value indicating the end of frame,
must be `0x00000000`.
Previously, it was just mentioned as a `0-size` block.
But such definition could encompass uncompressed blocks of size 0,
with a header of value `0x80000000`.
But the intention was to also support uncompressed empty blocks.
They could be used as a keep-alive signal.
Note that compressed empty blocks are already supported,
it's just that they have a size 1 instead of 0 (for the `0` token).
Unfortunately, the decoder implementation was also wrong,
and would also interpret a `0x80000000` block header as an endMark.
This issue evaded detection so far simply because
this situation never happens, as LZ4Frame always issues
a clean 0x00000000 value as a endMark.
It also does not flush empty blocks.
This is fixed in this PR.
The decoder can now deal with empty uncompressed blocks,
and do not confuse them with EndMark.
The specification is also clarified.
Finally, FrameTest is updated to randomly insert empty blocks during fuzzing.
|
| |
|
| |
|
|
|
|
|
| |
When `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is defined we skip
magic and checksum checks. This makes it easier to fuzz decompression.
|
| |
|
| |
|
| |
|
|
|
|
| |
fix #679, reported by @degski
|
|
|
|
| |
as suggested by @sloutsky in #671
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| | |
since Visual 2017,
worries about potential overflow, which are actually impossible.
Replaced (c * a) by (c ? a : 0).
Will likely replaced a * by a cmov.
Probably harmless for performance.
|