summaryrefslogtreecommitdiffstats
path: root/lib
Commit message (Collapse)AuthorAgeFilesLines
...
* | fix malloc handlingYan Pashkovsky2020-02-031-1/+4
| |
* | fix: allocate LZ4HC_optimal_t opt on heap each time (#837)Yan Pashkovsky2020-01-311-3/+16
| |
* | Force inline small functions used by LZ4_compress_generic.Bartosz Taudul2020-01-161-6/+6
| |
* | fixed lz4hc assert errorYann Collet2019-12-032-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | when src ptr is in very low memory area (< 64K), the virtual reference to data in dictionary might end up in a very high memory address. Since it's not a "real" memory address, just a virtual one, to calculate distance, it doesn't matter : only distance matters. The assert was to restrictive. Fixed.
* | faster decoding speed with VisualYann Collet2019-12-031-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | by enabling the fast decoder path. Visual requires a different set of macro constants to detect x86 / x64. On my laptop, decoding speed on x64 went up from 3.12 to 3.45 GB/s. 32-bit is less impressive, though still favorable, with speed increasing from 2.55 to 2.60 GB/s. So both cases are now enabled. Suggested by Bartosz Taudul (@wolfpld).
* | Have read_variable_length use fixed size typesNigel Tao2019-09-211-2/+6
|/ | | | | | | | | | Otherwise, the output from decoding LZ4-compressed input could be platform dependent. Also add a compile-time check to confirm the existing code's assumptions that, if <stdint.h> isn't used, then sizeof(int) == 4. Updates #792
* [LZ4_compress_destSize] Fix off-by-one errorNick Terrell2019-08-091-1/+1
| | | | | | | | | | PR#756 fixed the data corruption bug, but didn't clear `ip`. PR#760 fixed that off-by-one error, but missed the case where `ip == filledIp`, which is harder for the fuzzers to find (it took 20 days not 1 day). Verified this fixed the issue reported by OSS-Fuzz. Credit to OSS-Fuzz.
* Only Bump Offset When Attaching Non-Null DictionaryW. Felix Handte2019-08-061-15/+21
| | | | | We do want to bump, even if the dictionary is empty, but we **don't** want to bump if the dictionary is null.
* Add Attach Dict Debug LogW. Felix Handte2019-08-061-0/+4
|
* Make Attaching an Empty Dict Behave the Same as Using it DirectlyW. Felix Handte2019-08-061-14/+15
| | | | | | | | | | | | | | | | | | | When using an empty dictionary, we bail out of loading or attaching it in ways that leave the working context in potentially slightly different states. In particular, in some paths, we will cause the currentOffset to be non-zero, while in others we would allow it to remain 0. This difference in behavior is perfectly harmless, but in some situations, it can produce slight differences in the compressed output. For sanity's sake, we currently try to maintain a strict correspondence between the behavior of the dict attachment and the dict loading paths. This patch restores them to behaving identically. This shouldn't have any negative side-effects, as far as I can tell. When writing the dict attachment code, I tried to preserve zeroed currentOffsets when possible, since they benchmarked as very slightly faster. However, the case of attaching an empty dictionary is probably rare enought that it's acceptable to minisculely degrade performance in that corner case.
* Merge pull request #772 from lz4/offset0Yann Collet2019-08-061-64/+65
|\ | | | | silence msan warning when offset==0
| * silence msan warning when offset==0Yann Collet2019-08-061-64/+65
| |
* | [lz4hc] Chain swap with accelerationNick Terrell2019-07-311-4/+10
| |
* | [lz4hc] Only allow chain swapping forwardsNick Terrell2019-07-311-1/+1
| | | | | | | | | | | | When the match is very long and found quickly, we can do matchLength * nbCompares iterations through the chain swapping, which can really slow down compression.
* | [lz4hc] Fix pattern detection end of dictionaryNick Terrell2019-07-311-20/+45
| | | | | | | | | | | | The pattern detection in extDict mode could put `matchIndex` within the last 3 bytes of the dictionary. This would cause a read out of bounds.
* | [lz4hc] Fix minor pessimization in extDict pattern matchingNick Terrell2019-07-311-1/+1
| | | | | | | | | | | | We should be comparing `matchPtr` not `ip`. This bug just means that this branch was not taken, so we might miss some of the forward length.
* | [lz4hc] Improve pattern detection in ext dictNick Terrell2019-07-311-5/+13
|/ | | | | | | | | | | | It is important to continue to look backwards if the current pattern reaches `lowPrefixPtr`. If the pattern detection doesn't go all the way to the beginning of the pattern, or the end of the pattern it slows down the search instead of speeding it up. The slow unit in `round_trip_stream_fuzzer` used to take 12 seconds to run with -O3, now it takes 0.2 seconds. Credit to OSS-Fuzz
* [LZ4HC] Speed up pattern compression with external dictionaryNick Terrell2019-07-241-27/+47
| | | | Fixes #761.
* Merge pull request #763 from terrelln/unusedYann Collet2019-07-191-0/+9
|\ | | | | [lz4frame] Fix unused variable warnings in fuzzing mode
| * [lz4frame] Fix unused variable warnings in fuzzing modeNick Terrell2019-07-191-0/+9
| |
* | Merge pull request #760 from terrelln/destSizeYann Collet2019-07-191-1/+1
|\ \ | |/ |/| [LZ4_compress_destSize] Fix off-by-one error in fix
| * [LZ4_compress_destSize] Fix off-by-one error in fixNick Terrell2019-07-181-1/+1
| | | | | | | | | | | | | | The next match is looking at the current ip, not the next ip, so it needs to be cleared as well. Credit to OSS-Fuzz
* | [lz4frame] Skip magic and checksums in fuzzing modeNick Terrell2019-07-191-0/+12
| | | | | | | | | | When `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is defined we skip magic and checksum checks. This makes it easier to fuzz decompression.
* | Fix LZ4_attach_dictionary with empty dictionaryNick Terrell2019-07-181-1/+1
| |
* | Unconditionally Clear `dictCtx`W. Felix Handte2019-07-181-5/+3
| |
* | Fix Data Corruption Bug when Streaming with an Attached Dict in HC ModeW. Felix Handte2019-07-181-0/+5
|/ | | | | | | | | | | | This diff fixes an issue in which we failed to clear the `dictCtx` in HC compression. The `dictCtx` is not supposed to be used when an `extDict` is present: matches found in the `dictCtx` do not account for the presence of an `extDict` segment, and their offsets are therefore miscalculated when one is present. This can lead to data corruption. This diff clears the `dictCtx` whenever setting an `extDict`. This issue was uncovered by @terrelln's fuzzing work.
* [LZ4_compress_destSize] Fix overflow conditionNick Terrell2019-07-171-1/+1
|
* [LZ4_compress_destSize] Fix rare data corruption bugNick Terrell2019-07-171-0/+30
|
* [LZ4_compress_destSize] Allow 2 more bytes of match lengthNick Terrell2019-07-171-1/+1
|
* Merge pull request #752 from terrelln/fuzzersYann Collet2019-07-161-8/+41
|\ | | | | [ossfuzz] Improve the fuzzers
| * [lz4] Fix bugs in partial decodingNick Terrell2019-07-151-8/+41
| | | | | | | | | | * Partial decoding could read a few bytes beyond the end of the input * Partial decoding returned an error with an empty output buffer
* | ensure conformance with custom LZ4_DISTANCE_MAXYann Collet2019-07-152-6/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | It's now possible to select a custom LZ4_DISTANCE_MAX at compile time, provided it's <= 65535. However, in some cases (when compressing in byU16 mode), the new distance wasn't respected, as it used to implied that it was necessarily within range. Added a distance check for this case. Also : added a new TravisCI test which ensures that custom LZ4_DISTANCE_MAX compiles correctly and compresses correctly (relying on `assert()` to find outsized offsets).
* | bugfix: correctly control the offset < LZ4_DISTANCE_MAX,when change the ↵Hitatm2019-07-151-1/+1
|/ | | | value of LZ4_DISTANCE_MAX,
* Remove an useless declarationSylvestre Ledru2019-07-041-1/+1
|
* bumped version number to v1.9.2Yann Collet2019-07-011-1/+1
| | | | to reduce risks that future bug reports in `dev` branch report `v1.9.1` as the failing version.
* Fix out-of-bounds read of up to 64 KB in the pastNick Terrell2019-06-281-2/+8
|
* precise again that LZ4 decoder needs metadataYann Collet2019-06-061-22/+31
| | | | and that such metadata must be provided / sent / saved by the application.
* restored FORCE_INLINEYann Collet2019-06-041-1/+0
|
* Merge pull request #717 from lz4/inplaceYann Collet2019-05-313-23/+99
|\ | | | | Added documentation and macro to support in-place compression and decompression
| * added more details for in-place documentationYann Collet2019-05-311-14/+15
| |
| * decompress: changed final memcpy() into memmove()Yann Collet2019-05-311-10/+13
| | | | | | | | for compatibility with in-place decompression scenarios.
| * updated LZ4_DECOMPRESS_INPLACE_MARGINYann Collet2019-05-301-2/+2
| | | | | | | | | | to pass worst case scenario. Now adds margin proportional to input size to counter local expansion.
| * add more doc on in-place (de)compressionYann Collet2019-05-301-5/+25
| |
| * some more minor conversion warnings fixesYann Collet2019-05-291-4/+4
| |
| * ensure lz4.h can be included with or without LZ4_STATIC_LINKING_ONLY in any ↵Yann Collet2019-05-293-4/+16
| | | | | | | | | | | | order ensure correct propagation of LZ4_DISTANCE_MAX
| * added comments and macros for in-place (de)compressionYann Collet2019-05-292-6/+46
| |
* | FAST_DEC_LOOP: only did offset check in specific condition.Chenxi Mao2019-05-311-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When I did FAST_DEC_LOOP performance test, I found the offset check is much more than v1.8.3 You will see the condition check difference via lzbench with dickens test case. v1.8.3 34959 v.1.9.x 1055885 After investigate the code, we could see the difference. v.1.8.3 SKIP the condition check if if condition is true in: https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1463 AND below condition is true https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1478\ The offset check should be invoked. v1.9.3 The offset check code will be invoked in every loop which lead to downgrade. So the fix would be move this check to specific condition to avoid useless condition check. After this change, the call number is same as v1.8.3
* | fix temporary buffer use when input size hint is respectedNiko Dzhus2019-05-241-1/+1
|/
* Merge pull request #708 from gabrielstedman/listYann Collet2019-05-162-2/+11
|\ | | | | Add multiframe report to --list command
| * Add multiframe report to --list commandgstedman2019-05-152-2/+11
| |
generated by cgit v0.12 at 2024-07-16 21:16:40 (GMT)