summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
| * Created a data producer API and used in decompress_fuzzerbimbashrestha2019-08-162-6/+26
| |
| * Adding fuzz data producer for uint32 and using in decompress_fuzzerbimbashrestha2019-08-162-3/+29
|/ | | | | | | | | | | | | | Summary: Consuming bytes from the end of data instead of from the front to prevent "all-in-one" decisions. Test Plan: Reviewers: Subscribers: Tasks: Tags:
* fixed test errorv1.9.2Yann Collet2019-08-151-1/+1
| | | | | could trigger %0 on exceptional circumstances due to wrong buffer size parameter.
* Merge pull request #777 from terrelln/off-by-oneYann Collet2019-08-101-1/+1
|\ | | | | [LZ4_compress_destSize] Fix off-by-one error
| * [LZ4_compress_destSize] Fix off-by-one errorNick Terrell2019-08-091-1/+1
|/ | | | | | | | | | PR#756 fixed the data corruption bug, but didn't clear `ip`. PR#760 fixed that off-by-one error, but missed the case where `ip == filledIp`, which is harder for the fuzzers to find (it took 20 days not 1 day). Verified this fixed the issue reported by OSS-Fuzz. Credit to OSS-Fuzz.
* Merge pull request #773 from felixhandte/attach-empty-dict-behavior-conformanceYann Collet2019-08-071-10/+21
|\ | | | | Make Attaching an Empty Dict Behave the Same as Using it Directly
| * Only Bump Offset When Attaching Non-Null DictionaryW. Felix Handte2019-08-061-15/+21
| | | | | | | | | | We do want to bump, even if the dictionary is empty, but we **don't** want to bump if the dictionary is null.
| * Add Attach Dict Debug LogW. Felix Handte2019-08-061-0/+4
| |
| * Make Attaching an Empty Dict Behave the Same as Using it DirectlyW. Felix Handte2019-08-061-14/+15
|/ | | | | | | | | | | | | | | | | | | When using an empty dictionary, we bail out of loading or attaching it in ways that leave the working context in potentially slightly different states. In particular, in some paths, we will cause the currentOffset to be non-zero, while in others we would allow it to remain 0. This difference in behavior is perfectly harmless, but in some situations, it can produce slight differences in the compressed output. For sanity's sake, we currently try to maintain a strict correspondence between the behavior of the dict attachment and the dict loading paths. This patch restores them to behaving identically. This shouldn't have any negative side-effects, as far as I can tell. When writing the dict attachment code, I tried to preserve zeroed currentOffsets when possible, since they benchmarked as very slightly faster. However, the case of attaching an empty dictionary is probably rare enought that it's acceptable to minisculely degrade performance in that corner case.
* Merge pull request #772 from lz4/offset0Yann Collet2019-08-062-64/+66
|\ | | | | silence msan warning when offset==0
| * silence msan warning when offset==0Yann Collet2019-08-062-64/+66
| |
* | Merge pull request #771 from terrelln/rep-ext-fixYann Collet2019-08-011-26/+65
|\ \ | |/ |/| [lz4hc] Further improve pattern detection and chain swapping
| * [lz4hc] Chain swap with accelerationNick Terrell2019-07-311-4/+10
| |
| * [lz4hc] Only allow chain swapping forwardsNick Terrell2019-07-311-1/+1
| | | | | | | | | | | | When the match is very long and found quickly, we can do matchLength * nbCompares iterations through the chain swapping, which can really slow down compression.
| * [lz4hc] Fix pattern detection end of dictionaryNick Terrell2019-07-311-20/+45
| | | | | | | | | | | | The pattern detection in extDict mode could put `matchIndex` within the last 3 bytes of the dictionary. This would cause a read out of bounds.
| * [lz4hc] Fix minor pessimization in extDict pattern matchingNick Terrell2019-07-311-1/+1
| | | | | | | | | | | | We should be comparing `matchPtr` not `ip`. This bug just means that this branch was not taken, so we might miss some of the forward length.
| * [lz4hc] Improve pattern detection in ext dictNick Terrell2019-07-311-5/+13
|/ | | | | | | | | | | | It is important to continue to look backwards if the current pattern reaches `lowPrefixPtr`. If the pattern detection doesn't go all the way to the beginning of the pattern, or the end of the pattern it slows down the search instead of speeding it up. The slow unit in `round_trip_stream_fuzzer` used to take 12 seconds to run with -O3, now it takes 0.2 seconds. Credit to OSS-Fuzz
* Merge pull request #768 from terrelln/rep-extYann Collet2019-07-241-27/+47
|\ | | | | [LZ4HC] Speed up pattern compression with external dictionary
| * [LZ4HC] Speed up pattern compression with external dictionaryNick Terrell2019-07-241-27/+47
|/ | | | Fixes #761.
* Merge pull request #766 from Low-power/cli-option---bestYann Collet2019-07-233-0/+11
|\ | | | | Add option '--best' to lz4(1)
| * lz4cli: add option '--best' as an alias of '-12'WHR2019-07-233-0/+11
|/
* Merge pull request #763 from terrelln/unusedYann Collet2019-07-191-0/+9
|\ | | | | [lz4frame] Fix unused variable warnings in fuzzing mode
| * [lz4frame] Fix unused variable warnings in fuzzing modeNick Terrell2019-07-191-0/+9
| |
* | Merge pull request #760 from terrelln/destSizeYann Collet2019-07-191-1/+1
|\ \ | |/ |/| [LZ4_compress_destSize] Fix off-by-one error in fix
| * [LZ4_compress_destSize] Fix off-by-one error in fixNick Terrell2019-07-181-1/+1
| | | | | | | | | | | | | | The next match is looking at the current ip, not the next ip, so it needs to be cleared as well. Credit to OSS-Fuzz
* | Merge pull request #758 from dooxe/developYann Collet2019-07-191-0/+1
|\ \ | | | | | | Added `BUNDLE DESTINATION`
| * | Added BUNDLE DESTINATION in CMakeLists.txt so that it works with newer ↵dooxe2019-07-181-0/+1
| | | | | | | | | | | | versions of cmake
| * | Merge pull request #692 from lz4/devv1.9.1Yann Collet2019-04-2329-464/+825
| |\ \ | | | | | | | | v1.9.1
* | \ \ Merge pull request #762 from terrelln/frame-fuzzYann Collet2019-07-1915-11/+652
|\ \ \ \ | |_|_|/ |/| | | [fuzz] Add LZ4 frame fuzzers
| * | | [fuzz] Add LZ4 frame fuzzersNick Terrell2019-07-196-3/+219
| | | | | | | | | | | | | | | | | | | | | | | | * Round trip fuzzer * Compress fuzzer * Decompress fuzzer
| * | | [lz4frame] Skip magic and checksums in fuzzing modeNick Terrell2019-07-191-0/+12
| | | | | | | | | | | | | | | | | | | | When `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is defined we skip magic and checksum checks. This makes it easier to fuzz decompression.
| * | | Fix LZ4_attach_dictionary with empty dictionaryNick Terrell2019-07-181-1/+1
| | | |
| * | | [fuzz] Add HC fuzzers for round trip, compress, and streamingNick Terrell2019-07-185-14/+202
| | | |
| * | | [fuzzer] Update scripts for new fuzzersNick Terrell2019-07-183-6/+11
| | | |
| * | | [fuzz] Add a streaming round trip fuzzerNick Terrell2019-07-181-0/+217
| | | |
| * | | Unconditionally Clear `dictCtx`W. Felix Handte2019-07-181-5/+3
| | | |
| * | | Fix Data Corruption Bug when Streaming with an Attached Dict in HC ModeW. Felix Handte2019-07-181-0/+5
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This diff fixes an issue in which we failed to clear the `dictCtx` in HC compression. The `dictCtx` is not supposed to be used when an `extDict` is present: matches found in the `dictCtx` do not account for the presence of an `extDict` segment, and their offsets are therefore miscalculated when one is present. This can lead to data corruption. This diff clears the `dictCtx` whenever setting an `extDict`. This issue was uncovered by @terrelln's fuzzing work.
* | | Merge pull request #756 from terrelln/destSizeYann Collet2019-07-171-2/+32
|\ \ \ | | | | | | | | [LZ4_compress_destSize + multi-blocks streaming] Fix rare data corruption bug
| * | | [LZ4_compress_destSize] Fix overflow conditionNick Terrell2019-07-171-1/+1
| | | |
| * | | [LZ4_compress_destSize] Fix rare data corruption bugNick Terrell2019-07-171-0/+30
| | | |
| * | | [LZ4_compress_destSize] Allow 2 more bytes of match lengthNick Terrell2019-07-171-1/+1
|/ / /
* | | Merge pull request #752 from terrelln/fuzzersYann Collet2019-07-169-52/+314
|\ \ \ | | | | | | | | [ossfuzz] Improve the fuzzers
| * | | [ossfuzz] Improve the fuzzersNick Terrell2019-07-158-44/+273
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Run more decompression variants * Round trip the compression fuzzer and do partial decompression as well * Add a compression fuzzer that compresses into a smaller output buffer and test the destSize variant These fuzzers caught 2 bugs that were fixed in the previous commit. * Input buffer over-read in partial decompress * Partial decompress fails if output size is 0
| * | | [lz4] Fix bugs in partial decodingNick Terrell2019-07-151-8/+41
| | | | | | | | | | | | | | | | | | | | * Partial decoding could read a few bytes beyond the end of the input * Partial decoding returned an error with an empty output buffer
* | | | Merge pull request #755 from lz4/custom_distanceYann Collet2019-07-155-8/+18
|\ \ \ \ | | | | | | | | | | ensure conformance with custom LZ4_DISTANCE_MAX
| * | | | ensure conformance with custom LZ4_DISTANCE_MAXYann Collet2019-07-155-8/+18
|/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's now possible to select a custom LZ4_DISTANCE_MAX at compile time, provided it's <= 65535. However, in some cases (when compressing in byU16 mode), the new distance wasn't respected, as it used to implied that it was necessarily within range. Added a distance check for this case. Also : added a new TravisCI test which ensures that custom LZ4_DISTANCE_MAX compiles correctly and compresses correctly (relying on `assert()` to find outsized offsets).
* | | | Merge pull request #753 from Hitatm/fix_LZ4_DISTANCE_MAXYann Collet2019-07-151-1/+1
|\ \ \ \ | |/ / / |/| | | bugfix: correctly control the offset < LZ4_DISTANCE_MAX,when change t…
| * | | bugfix: correctly control the offset < LZ4_DISTANCE_MAX,when change the ↵Hitatm2019-07-151-1/+1
|/ / / | | | | | | | | | value of LZ4_DISTANCE_MAX,
* | | keep the "lorem ipsum" topic of the example stringYann Collet2019-07-121-1/+1
| | | | | | | | | | | | but make it compressible
* | | Merge pull request #751 from hamidzr/simple-buffer-example-inputYann Collet2019-07-121-2/+3
|\ \ \ | | | | | | | | simple buffer example minor input update. fixes #750
generated by cgit v0.12 at 2024-07-18 10:45:33 (GMT)