| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
| |
between lz4.c and lz4hc.c .
was left in a strange state after the "amalgamation" patch.
Now only 3 directives remain,
same name across both implementations,
single definition place.
Might allow some light simplification due to reduced nb of states possible.
|
|
|
|
|
|
|
| |
when one block was not compressible,
it would tag the context as `dirty`,
resulting in compression automatically bailing out of all future blocks,
making the rest of the frame uncompressible.
|
|
|
|
|
|
|
|
| |
since Visual 2017,
worries about potential overflow, which are actually impossible.
Replaced (c * a) by (c ? a : 0).
Will likely replaced a * by a cmov.
Probably harmless for performance.
|
|
|
|
|
|
|
|
|
|
| |
make it possible to generate LZ4-compressed block
with a controlled maximum offset (necessarily <= 65535).
This could be useful for compatibility with decoders
using a very limited memory budget (<64 KB).
Answer #154
|
|
|
|
|
| |
it is now a pure initializer, for statically allocated states.
It can initialize any memory area, and because of this, requires size.
|
|
|
|
| |
ensure it's not NULL.
|
|
|
|
|
|
|
| |
it fails on x86 32-bit mode :
Visual reports an alignment of 8-bytes (even with alignof())
but actually only align LZ4_stream_t on 4 bytes.
The alignment check then fails, resulting in missed initialization.
|
|
|
|
| |
updated associated documentation
|
|
|
|
|
|
|
|
|
|
| |
- promoted LZ4_resetStream_fast() to stable
- moved LZ4_resetStream() into deprecate, but without triggering a compiler warning
- update all sources to no longer rely on LZ4_resetStream()
note : LZ4_initStream() proposal is slightly different :
it's able to initialize any buffer, provided that it's large enough.
To this end, it accepts a void*, and returns an LZ4_stream_t*.
|
| |
|
| |
|
| |
|
| |
|
|\
| |
| | |
Optimize decompress generic
|
| |
| |
| |
| |
| | |
New fastpath currently shows a regression on qualcomm
arm chips. Restrict it to x86 for now
|
| |
| |
| |
| |
| |
| |
| |
| | |
For small offsets of size 1, 2, 4 and 8, we can set a single uint64_t,
and then use it to do a memset() variation. In particular, this makes
the somewhat-common RLE (offset 1) about 2-4x faster than the previous
implementation - we avoid not only the load blocked by store, but also
avoid the loads entirely.
|
| |
| |
| |
| |
| |
| |
| |
| | |
Generally we want our wildcopy loops to look like the
memcpy loops from our libc, but without the final byte copy checks.
We can unroll a bit to make long copies even faster.
The only catch is that this affects the value of FASTLOOP_SAFE_DISTANCE.
|
| |
| |
| |
| |
| | |
This store is also causing load-blocked-by-store issues, remove it.
The msan warning will have to be fixed another way if it is still an issue.
|
| |
| |
| |
| |
| | |
This is the remaineder of the original 'shortcut'. If true, we can avoid
the loop in LZ4_wildCopy, and directly copy instead.
|
| |
| |
| |
| |
| |
| | |
We've already checked that we are more than FASTLOOP_SAFE_DISTANCE
away from the end, so this branch can never be true, we will have
already jumped to the second decode loop.
|
| |
| |
| |
| |
| |
| |
| |
| | |
Use LZ4_wildCopy16 for variable-length literals. For literal counts that
fit in the flag byte, copy directly. We can also omit oend checks for
roughly the same reason as the previous shortcut: We check once that both
match length and literal length fit in FASTLOOP_SAFE_DISTANCE, including
wildcopy distance.
|
| |
| |
| |
| |
| |
| | |
Add an LZ4_wildCopy16, that will wildcopy, potentially smashing up
to 16 bytes, and use it for match copy. On x64, this avoids many
blocked loads due to store forwarding, similar to issue #411.
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Copy the main loop, and change checks such that op is always less
than oend-SAFE_DISTANCE. Currently these are added for the literal
copy length check, and for the match copy length check.
Otherwise the first loop is exactly the same as the second. Follow on
diffs will optimize the first copy loop based on this new requirement.
I also tried instead making a separate inlineable function for the copy
loop (similar to existing partialDecode flags, etc), but I think the
changes might be significant enough to warrent doubling the code, instead
pulling out common functionality to separate functions.
This is the basic transformation that will allow several following optimisations.
|
| |
| |
| |
| |
| | |
Make a helper function to read variable lengths for literals and
match length.
|
|/ |
|
|
|
|
|
|
|
| |
Dictionaries don't need to be > 4 bytes, they need to be >= 4 bytes. This test
was overly conservative.
Also removes the test in `LZ4_attach_dictionary()`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes a mismatch in behavior between loading into the context (via
`LZ4_loadDict()`) a very small (<= 4 bytes) non-contiguous dictionary, versus
attaching it with `LZ4_attach_dictionary()`.
Before this patch, this divergence could be reproduced by running
```
make -C tests fuzzer MOREFLAGS="-m32"
tests/fuzzer -v -s1239 -t3146
```
Making sure these two paths behave exactly identically is an easy way to test
the correctness of the attach path, so it's desirable that this remain an
unpolluted, high signal test.
|
| |
|
| |
|
|
|
|
| |
check if full reset is needed.
|
|\
| |
| | |
Support for 128bit pointers like AS400
|
| | |
|
| |
| |
| |
| | |
including 128-bit, like IBM AS-400
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
so "funny" thing with cppcheck
is that no 2 versions give the same list of warnings.
On Mac, I'm using v1.81, which had all warnings fixed.
On Travis CI, it's v1.61, and it complains about a dozen more/different things.
On Linux, it's v1.72, and it finds a completely different list of a half dozen warnings.
Some of these seems to be bugs/limitations in cppcheck itself.
The TravisCI version v1.61 seems unable to understand %zu correctly, and seems to assume it means %u.
|
|/ |
|
|
|
|
|
|
|
| |
and separate it from MATCH_SAFEGUARD_DISTANCE.
While both constants have same value,
they do not seve same purpose, hence should not be confused.
|
|
|
|
| |
added a few more comments and assert()
|
|
|
|
| |
to counter possible (offset==0)
|
| |
|
|
|
|
| |
and removed one blind copy, since there is no more guarantee that at least 4 bytes are still available in output buffer
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
| |
don't fix dictionaries of size 0.
setting dictEnd == source triggers prefix mode,
thus removing possibility to use CDict.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The error can be reproduced using following command :
./frametest -v -i100000000 -s1659 -t31096808
It's actually a bug in the stream LZ4 API,
when starting a new stream
and providing a first chunk to complete with size < MINMATCH.
In which case, the chunk becomes a dictionary.
No hash was generated and stored,
but the chunk is accessible as default position 0 points to dictStart,
and position 0 is still within MAX_DISTANCE.
Then, next attempt to read 32-bits from position 0 fails.
The issue would have been mitigated by starting from index 64 KB,
effectively eliminating position 0 as too far away.
The proper fix is to eliminate such "dictionary" as too small.
Which is what this patch does.
|
|
|
|
| |
as requested by @terrelln
|
|
|
|
| |
fuzzer : fix and robustify ring buffer tests
|
| |
|