summaryrefslogtreecommitdiffstats
path: root/lib/lz4frame.c
Commit message (Collapse)AuthorAgeFilesLines
* uncompressed-blocks: Allow uncompressed blocks for all modesAlexander Mohr2022-10-101-1/+0
| | | | | | | | | this commit changes that uncompressed blocks are only available for independent blocks. Building and updating the dictionary from an uncompressed block has no adverse side effects. Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
* added notes about LZ4_compressFrame() and stack/heap memory usageYann Collet2022-09-201-1/+1
|
* Clarifiy documentation for LZ4F_HEAPMODEYann Collet2022-09-131-2/+2
| | | | | | Now it's only about LZ4F_compressFrame(), and it's only about the Compression Context (not the hash tables, which used to be integrated).
* allocation optimization for lz4frame compressionYann Collet2022-09-081-7/+6
| | | | | | | | | | | | | | | | | | | | | | | as noted by @yixiutt, the temporary buffer inside lz4frame compression is being invalidated at end of compression, forcing it to be re-allocated at next compression job. This shouldn't be necessary. This change was introduced in #236, as a way to fix #232, but neither the issue is explained, nor why the patch fixes it. This patch revert to previous behavior, where temporary buffer is kept between compression calls. This results in a net reduction of allocation workload. Additionally, the temporary buffer should only need malloc(), not calloc(), thus saving initialization. This diff implements both changes. Long fuzzer tests will be run to ensure that no sanitizer warning get triggered. Additionally : fixed a minor ubsan warning in LZ4F_decompress().
* faster CLI decompression speed for frames with -BD4 settingYann Collet2022-08-121-16/+22
| | | | | | | | | | | lz4frame favors the faster prefix mode when decompressing a frame with linked blocks. This significantly improved CLI decompression on files compressed with -BD4 setting. On my laptop, decompressing `enwik9` went from 0.89s to 0.52s. This improvement is only for linked blocks. It's more visible for small block sizes.
* can select validation of CRC during benchmarkYann Collet2022-07-291-4/+4
| | | | | on command line, using existing long command --no-frame-crc. Note : it's effectively more than that, since _all_ checksums are disabled.
* added ability to skip checksum calculation when decoding LZ4 FramesYann Collet2022-07-291-9/+15
|
* Merge pull request #1115 from lz4/lz4f_customMemYann Collet2022-07-161-77/+155
|\ | | | | Support for Custom Memory managers
| * implemented LZ4F_createCDict_advanced()Yann Collet2022-07-131-16/+29
| |
| * implemented LZ4F_createDecompressionContext_advanced()Yann Collet2022-07-131-14/+25
| |
| * Merge branch 'dev' into lz4f_customMemYann Collet2022-07-131-5/+5
| |\
| * | implemented first custom memory manager interfaceYann Collet2022-07-131-59/+113
| | | | | | | | | | | | | | | | | | | | | for compression context only for the time being, using LZ4F_createCompressionContext_advanced(). Added basic test in frametest.c
* | | support skippable frames within pipeYann Collet2022-07-151-2/+0
| |/ |/| | | | | | | | | | | fix #977 fseek() doesn't work for pipe, switch to "read and forget" mode in such case.
* | fix stricter enum type requirements for C++Yann Collet2022-07-131-4/+4
| |
* | minor : proper interface for LZ4F_getBlockSize()Yann Collet2022-07-131-1/+1
|/ | | | | and proper documentation. Also : updated manual
* minor refactor : simplify LZ4F_makeBlockYann Collet2022-07-111-27/+20
| | | | one less argument
* review: fix findingsAlexander Mohr2022-07-051-1/+1
| | | | | | | | | * replace assert with test for LZ4F_uncompressedUpdate * update documentation to incldue correct docstring * remove unecessary entry point * remove compress_linked_block_mode from fuzzing test Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
* uncompressed-api: allow uncompressed_update only for independent blocksAlexander Mohr2022-07-051-99/+73
| | | | Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
* lz4frame: correct start and size after flushAlexander Mohr2022-07-041-1/+4
| | | | | | | | | | | when the block mode changes a flush is executed, to prevent mixing compressed and uncompressed data. Prior to this commit dstStart, dstPtr, dstCapacity where not updated to include the offset from bytesWritten. For inputs > blockSize this meant the flushed data was overwritten. Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
* meson: fix meson buildAlexander Mohr2022-06-111-1/+1
| | | | add static dependency to examples
* lz4frame: fix different linkage errorAlexander Mohr2022-06-111-1/+1
| | | | Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
* fuzz-test: add fuzz test for uncompressed apiAlexander Mohr2022-06-111-2/+1
| | | | | | add a fuzzing test for uncompressed frame api Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
* review: Fix review findingsAlexander Mohr2022-06-101-2/+5
| | | | | | This commit fixes the review findings Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
* frame-api: add method to insert uncomressed dataAlexander Mohr2022-06-091-62/+166
| | | | | | | | new method `uncompressed_update` allows to insert blocks without compression into the lz4 stream. The usage is documented in the frameCompress example Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
* introduced FORWARD_IF_ERROR()Yann Collet2022-05-101-48/+49
| | | | | | and removed prefix from RETURN_ERROR(_IF) to improve brevity as it's just a local macro (no bleeding in user's namespace).
* introduced macro LZ4F_RETURN_ERROR_IFYann Collet2022-05-071-32/+23
|
* introduced macro LZ4F_RETURN_ERRORYann Collet2022-05-071-37/+39
|
* removed ERROR_GENERIC from lz4frame.hYann Collet2022-05-071-3/+3
| | | | created a new error code in the process : LZ4F_ERROR_compressionState_uninitialized
* updated one error codeYann Collet2022-04-061-1/+1
|
* Introduce MIN and MAX bounds to LZ4_MEMORY_USAGEYann Collet2022-01-311-2/+2
| | | | | | | ensure that `frametest` works fine with these values, notably with low LZ4_MEMORY_USAGE (dict test notably) following suggestions from @t-mat at #1016
* Fix typos found by codespellDimitri Papadopoulos2021-11-251-1/+1
|
* Don't reuse state memory that's too small.Eddy Jansson2021-07-311-1/+13
| | | | | | | Ensure that the memory block we're trying to reuse is large enough for the new state. Fixes #974
* fix UB lz4:988 and lz4:1178Yann Collet2021-05-281-0/+2
| | | | | ensure `dictBase` is only used when there is an actual dictionary content.
* fix UB of lz4frame:907Yann Collet2021-05-281-56/+63
| | | | | | now line 912 by ensuring pointer arithmetic is only performed if there is a reason for an internal buffer to be used.
* minor : lz4frame include stdlib only if neededYann Collet2020-11-101-1/+1
|
* LZ4F_decompress requires a valid dctx stateYann Collet2020-11-071-0/+1
| | | | | This is now explicitly documented and asserted. fix #927
* add LZ4F_decompress() tests with (NULL,0) input and outputYann Collet2020-10-021-32/+52
| | | | fix one (rare & complex) issue discovered by this test
* make scan-build accept assert()Yann Collet2020-10-011-6/+9
|
* added memcpy() related SA warning fixesYann Collet2020-10-011-8/+14
| | | | memcpy() on NULL is UB, even if length is 0.
* fix minor static analyzer warningsYann Collet2020-09-301-10/+11
| | | | | detected by scan-build and cppcheck fix #786
* fixed lz4frame with blocks of size 1Yann Collet2020-09-171-1/+4
| | | | properly track history
* Clarifies and fix EndMarkYann Collet2020-08-131-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | EndMark, the 4-bytes value indicating the end of frame, must be `0x00000000`. Previously, it was just mentioned as a `0-size` block. But such definition could encompass uncompressed blocks of size 0, with a header of value `0x80000000`. But the intention was to also support uncompressed empty blocks. They could be used as a keep-alive signal. Note that compressed empty blocks are already supported, it's just that they have a size 1 instead of 0 (for the `0` token). Unfortunately, the decoder implementation was also wrong, and would also interpret a `0x80000000` block header as an endMark. This issue evaded detection so far simply because this situation never happens, as LZ4Frame always issues a clean 0x00000000 value as a endMark. It also does not flush empty blocks. This is fixed in this PR. The decoder can now deal with empty uncompressed blocks, and do not confuse them with EndMark. The specification is also clarified. Finally, FrameTest is updated to randomly insert empty blocks during fuzzing.
* fix LZ4F_compressBound when automatic flushing is enabledChristopher Harvie2020-05-141-0/+3
|
* [lz4frame] Fix unused variable warnings in fuzzing modeNick Terrell2019-07-191-0/+9
|
* [lz4frame] Skip magic and checksums in fuzzing modeNick Terrell2019-07-191-0/+12
| | | | | When `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is defined we skip magic and checksum checks. This makes it easier to fuzz decompression.
* fix temporary buffer use when input size hint is respectedNiko Dzhus2019-05-241-1/+1
|
* Add multiframe report to --list commandgstedman2019-05-151-2/+2
|
* Include block checksum in worst case scenario calculation of dstCapacityBrenden Eng2019-04-261-0/+1
|
* lz4frame: initializers compatibility with C++Yann Collet2019-04-191-3/+5
| | | | fix #679, reported by @degski
* ensure consistent definition and usage of FREEMEMYann Collet2019-04-161-4/+4
| | | | as suggested by @sloutsky in #671
generated by cgit v0.12 at 2024-06-29 03:19:10 (GMT)