| Commit message (Collapse) | Author | Age | Files | Lines |
... | |
|/
|
|
|
| |
Nothing internally sets dirty anymore. The only way to get that is if you use
an uninitialized context, in which case your warranty is void anyways.
|
|
|
|
|
|
|
|
|
| |
`LZ4_memcpy()` uses `__builtin_memcpy()` to ensure that clang/gcc
can inline the `memcpy()` calls in freestanding mode.
This is necessary for decompressing the Linux Kernel with LZ4.
Without an analogous patch decompression ran at 77 MB/s, and with
the patch it ran at 884 MB/s.
|
|\
| |
| | |
Avoid old-style function definitions
|
| |
| |
| |
| |
| |
| |
| | |
Define 0-argument functions like foo(void) instead of foo(), in order
to avoid a warning with -Wold-style-definition. This makes it easier
to embed lz4.c in projects that compile with -Werror
-Wold-style-definition.
|
|\ \
| | |
| | | |
Reducing stack usage in _t_alignment checks
|
| | | |
|
|/ /
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Similar work in the kernel:
https://patchwork.kernel.org/patch/11351499/
UBsan (+clang-10) complains about doing pointer arithmetic (adding 0)
to a nullpointer.
This patch is tested with clang-10+ubsan
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
when src ptr is in very low memory area (< 64K),
the virtual reference to data in dictionary
might end up in a very high memory address.
Since it's not a "real" memory address,
just a virtual one, to calculate distance,
it doesn't matter : only distance matters.
The assert was to restrictive.
Fixed.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
by enabling the fast decoder path.
Visual requires a different set of macro constants to detect x86 / x64.
On my laptop, decoding speed on x64 went up from 3.12 to 3.45 GB/s.
32-bit is less impressive, though still favorable,
with speed increasing from 2.55 to 2.60 GB/s.
So both cases are now enabled.
Suggested by Bartosz Taudul (@wolfpld).
|
|/
|
|
|
|
|
|
|
|
| |
Otherwise, the output from decoding LZ4-compressed input could be
platform dependent.
Also add a compile-time check to confirm the existing code's assumptions
that, if <stdint.h> isn't used, then sizeof(int) == 4.
Updates #792
|
|
|
|
|
|
|
|
|
|
| |
PR#756 fixed the data corruption bug, but didn't clear `ip`. PR#760
fixed that off-by-one error, but missed the case where `ip == filledIp`,
which is harder for the fuzzers to find (it took 20 days not 1 day).
Verified this fixed the issue reported by OSS-Fuzz.
Credit to OSS-Fuzz.
|
|
|
|
|
| |
We do want to bump, even if the dictionary is empty, but we **don't** want to
bump if the dictionary is null.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When using an empty dictionary, we bail out of loading or attaching it in
ways that leave the working context in potentially slightly different states.
In particular, in some paths, we will cause the currentOffset to be non-zero,
while in others we would allow it to remain 0.
This difference in behavior is perfectly harmless, but in some situations, it
can produce slight differences in the compressed output. For sanity's sake,
we currently try to maintain a strict correspondence between the behavior of
the dict attachment and the dict loading paths. This patch restores them to
behaving identically.
This shouldn't have any negative side-effects, as far as I can tell. When
writing the dict attachment code, I tried to preserve zeroed currentOffsets
when possible, since they benchmarked as very slightly faster. However, the
case of attaching an empty dictionary is probably rare enought that it's
acceptable to minisculely degrade performance in that corner case.
|
| |
|
|\
| |
| | |
[LZ4_compress_destSize] Fix off-by-one error in fix
|
| |
| |
| |
| |
| |
| |
| | |
The next match is looking at the current ip, not the next ip,
so it needs to be cleared as well.
Credit to OSS-Fuzz
|
|/ |
|
| |
|
| |
|
| |
|
|\
| |
| | |
[ossfuzz] Improve the fuzzers
|
| |
| |
| |
| |
| | |
* Partial decoding could read a few bytes beyond the end of the input
* Partial decoding returned an error with an empty output buffer
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
| |
It's now possible to select a custom LZ4_DISTANCE_MAX at compile time,
provided it's <= 65535.
However, in some cases (when compressing in byU16 mode),
the new distance wasn't respected,
as it used to implied that it was necessarily within range.
Added a distance check for this case.
Also : added a new TravisCI test which ensures that
custom LZ4_DISTANCE_MAX compiles correctly
and compresses correctly (relying on `assert()` to find outsized offsets).
|
| |
|
| |
|
| |
|
|\
| |
| | |
Added documentation and macro to support in-place compression and decompression
|
| |
| |
| |
| | |
for compatibility with in-place decompression scenarios.
|
| | |
|
| |
| |
| |
| |
| |
| | |
order
ensure correct propagation of LZ4_DISTANCE_MAX
|
| | |
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When I did FAST_DEC_LOOP performance test, I found the
offset check is much more than v1.8.3
You will see the condition check difference via lzbench with dickens test case.
v1.8.3 34959
v.1.9.x 1055885
After investigate the code, we could see the difference.
v.1.8.3 SKIP the condition check if
if condition is true in:
https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1463
AND below condition is true
https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1478\
The offset check should be invoked.
v1.9.3
The offset check code will be invoked in every loop which lead to downgrade.
So the fix would be move this check to specific condition
to avoid useless condition check.
After this change, the call number is same as v1.8.3
|
| |
|
| |
|
| |
|
|
|
|
| |
was disabled for tests
|
|
|
|
|
|
|
|
|
|
|
| |
and deprecate LZ4_decompress_fast(),
with deprecation warnings enabled by default.
Note that, as a consequence of the fix,
LZ4_decompress_fast is now __slower__ than LZ4_decompress_safe().
That's because, since it doesn't know the input buffer size,
it must progress more cautiously into the input buffer
to ensure to out-of-bound read.
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
between lz4.c and lz4hc.c .
was left in a strange state after the "amalgamation" patch.
Now only 3 directives remain,
same name across both implementations,
single definition place.
Might allow some light simplification due to reduced nb of states possible.
|
|
|
|
|
|
|
| |
when one block was not compressible,
it would tag the context as `dirty`,
resulting in compression automatically bailing out of all future blocks,
making the rest of the frame uncompressible.
|
|
|
|
|
|
|
|
| |
since Visual 2017,
worries about potential overflow, which are actually impossible.
Replaced (c * a) by (c ? a : 0).
Will likely replaced a * by a cmov.
Probably harmless for performance.
|
|
|
|
|
|
|
|
|
|
| |
make it possible to generate LZ4-compressed block
with a controlled maximum offset (necessarily <= 65535).
This could be useful for compatibility with decoders
using a very limited memory budget (<64 KB).
Answer #154
|
|
|
|
|
| |
it is now a pure initializer, for statically allocated states.
It can initialize any memory area, and because of this, requires size.
|
|
|
|
| |
ensure it's not NULL.
|
|
|
|
|
|
|
| |
it fails on x86 32-bit mode :
Visual reports an alignment of 8-bytes (even with alignof())
but actually only align LZ4_stream_t on 4 bytes.
The alignment check then fails, resulting in missed initialization.
|
|
|
|
| |
updated associated documentation
|