summaryrefslogtreecommitdiffstats
path: root/lib
Commit message (Collapse)AuthorAgeFilesLines
* lz4.c: fixed the LZ4_decompress_safe_continue caseAlexey Tourbin2018-04-261-17/+36
| | | | | | | | | | | | | | | | | | | | | | | | The previous change broke decoding with a ring buffer. That's because I didn't realize that the "double dictionary mode" was possible, i.e. that the decoding routine can look both at the first part of the dictionary passed as prefix and the second part passed via dictStart+dictSize. So this change introduces the LZ4_decompress_safe_doubleDict helper, which handles this "double dictionary" situation. (This is a bit of a misnomer, there is only one dictionary, but I can't think of a better name, and perhaps the designation is not all too bad.) The helper is used only once, in LZ4_decompress_safe_continue, it should be inlined with LZ4_FORCE_O2_GCC_PPC64LE attached to LZ4_decompress_safe_continue. (Also, in the helper functions, I change the dictStart parameter type to "const void*", to avoid a cast when calling helpers. In the helpers, the upcast to "BYTE*" is still required, for compatibility with C++.) So this fixes the case of LZ4_decompress_safe_continue, and I'm surprised by the fact that the fuzzer is now happy and does not detect a similar problem with LZ4_decompress_fast_continue. So before fixing LZ4_decompress_fast_continue, the next logical step is to enhance the fuzzer.
* lz4.c: refactor the decoding routinesAlexey Tourbin2018-04-251-53/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I noticed that LZ4_decompress_generic is sometimes instantiated with identical set of parameters, or (what's worse) with a subtly different sets of parameters. For example, LZ4_decompress_fast_withPrefix64k is instantiated as follows: return LZ4_decompress_generic(source, dest, 0, originalSize, endOnOutputSize, full, 0, withPrefix64k, (BYTE*)dest - 64 KB, NULL, 64 KB); while the equivalent withPrefix64k call in LZ4_decompress_usingDict_generic passes 0 for the last argument instead of 64 KB. It turns out that there is no difference in this case: if you change 64 KB to 0 KB in LZ4_decompress_fast_withPrefix64k, you get the same binary code. Moreover, because it's been clarified that LZ4_decompress_fast doesn't check match offsets, it is now obvious that both of these fast/withPrefix64k instantiations are simply redundant. Exactly because LZ4_decompress_fast doesn't check offsets, it serves well with any prefixed dictionary. There's a difference, though, with LZ4_decompress_safe_withPrefix64k. It also passes 64 KB as the last argument, and if you change that to 0, as in LZ4_decompress_usingDict_generic, you get a completely different binary code. It seems that passing 0 enables offset checking: const int checkOffset = ((safeDecode) && (dictSize < (int)(64 KB))); However, the resulting code seems to run a bit faster. How come enabling extra checks can make the code run faster? Curiouser and curiouser! This needs extra study. Currently I take the view that the dictSize should be set to non-zero when nothing else will do, i.e. when passing the external dictionary via dictStart. Otherwise, lowPrefix betrays just enough information about the dictionary. * * * Anyway, with this change, I instantiate all the necessary cases as functions with distinctive names, which also take fewer arguments and are therefore less error-prone. I also make the functions non-inline. (The compiler won't inline the functions because they are used more than once. Hence I attach LZ4_FORCE_O2_GCC_PPC64LE to the instances while removing from the callers.) The number of instances is now is reduced from 18 (safe+fast+partial+4*continue+4*prefix+4*dict+2*prefix64+forceExtDict) down to 7 (safe+fast+partial+2*prefix+2*dict). The size of the code is not the only issue here. Separate helper function are much more amenable to profile-guided optimization: it is enough to profile only a few basic functions, while the other less-often used functions, such as LZ4_decompress_*_continue, will benefit automatically. This is the list of LZ4_decompress* functions in liblz4.so, sorted by size. Exported functions are marked with a capital T. $ nm -S lib/liblz4.so |grep -wi T |grep LZ4_decompress |sort -k2 0000000000016260 0000000000000005 T LZ4_decompress_fast_withPrefix64k 0000000000016dc0 0000000000000025 T LZ4_decompress_fast_usingDict 0000000000016d80 0000000000000040 T LZ4_decompress_safe_usingDict 0000000000016d10 000000000000006b T LZ4_decompress_fast_continue 0000000000016c70 000000000000009f T LZ4_decompress_safe_continue 00000000000156c0 000000000000059c T LZ4_decompress_fast 0000000000014a90 00000000000005fa T LZ4_decompress_safe 0000000000015c60 00000000000005fa T LZ4_decompress_safe_withPrefix64k 0000000000002280 00000000000005fa t LZ4_decompress_safe_withSmallPrefix 0000000000015090 000000000000062f T LZ4_decompress_safe_partial 0000000000002880 00000000000008ea t LZ4_decompress_fast_extDict 0000000000016270 0000000000000993 t LZ4_decompress_safe_forceExtDict
* Merge pull request #503 from lz4/l120Yann Collet2018-04-193-45/+103
|\ | | | | minor length reduction of several large lines
| * modified indentation for consistencyYann Collet2018-04-191-17/+33
| |
| * minor length reduction of several large linesYann Collet2018-04-183-44/+86
| |
* | Merge pull request #502 from lhacc1/devYann Collet2018-04-191-0/+4
|\ \ | |/ |/| Wrap likely/unlikely macroses with #ifndef
| * Wrap likely/unlikely macroses with #ifndefDmitrii Rodionov2018-04-181-0/+4
| | | | | | | | | | It prevent redefine error when project using lz4 has its own likely/unlikely macroses.
* | fixed LZ4_compress_fast_extState_fastReset() in 32-bit modeYann Collet2018-04-171-2/+2
| |
* | fix dictDelta setting errorYann Collet2018-04-171-1/+1
| | | | | | | | wrong test
* | fix matchIndex overflowYann Collet2018-04-171-12/+4
| | | | | | | | can happen with dictCtx
* | Merge branch 'dev' into lowAddrYann Collet2018-04-171-2/+9
|\ \ | |/
| * Always Bump Offset by 64 KB in LZ4_loadDict()W. Felix Handte2018-04-171-2/+9
| | | | | | | | | | This actually ensures the guarantee referred to in the comment in LZ4_compress_fast_continue().
* | fixed dictCtx compressionYann Collet2018-04-171-7/+12
| |
* | edited a few traces for debuggingYann Collet2018-04-171-7/+7
| |
* | fixed minor format warningsYann Collet2018-04-161-3/+3
| |
* | fixed gcc performance regressionYann Collet2018-04-161-2/+4
| |
* | fixed minor unused variable warningYann Collet2018-04-131-3/+0
| |
* | added comment on variables required after _next_matchYann Collet2018-04-131-0/+8
| |
* | fixed potential ptrdiff_t overflow (32-bits mode)Yann Collet2018-04-131-14/+11
| | | | | | | | Also removed pointer comparison, which should solve #485
* | compatibility with gcc-4.4 string.h versionCyan49732018-04-131-3/+3
| | | | | | | | | | | | | | Someone found it would be a great idea to define there a global variable under the very generic name "index". Cause problem with shadow warnings, so no variable can be named "index" now ... Also : automatically update API manual
* | fixed : counting matches which overlap extDict and prefixtest49732018-04-121-10/+17
| |
* | modified a few traces for debugtest49732018-04-122-6/+6
| |
* | fixed LZ4_compress_fast_extState_fastReset()test49732018-04-111-8/+7
| |
* | Merge branch 'dev' into lowAddrtest49732018-04-113-67/+121
|\ \ | |/
| * Fix Silly Warning (const-ness in declaration has no effect on value types!)W. Felix Handte2018-04-111-1/+1
| |
| * Minor FixesW. Felix Handte2018-04-112-11/+13
| |
| * Add a LZ4_STATIC_LINKING_ONLY Macro to Guard Experimental APIsW. Felix Handte2018-04-113-0/+4
| |
| * Expose dictCtx Functionality in LZ4W. Felix Handte2018-04-113-2/+33
| |
| * Rename _extState_noReset -> _extState_fastReset and Edit CommentsW. Felix Handte2018-04-113-27/+41
| |
| * Remove Extraneous Assignment (clearedTable == 0)W. Felix Handte2018-04-111-1/+0
| |
| * Expose a Faster Stream Reset FunctionW. Felix Handte2018-04-103-28/+37
| |
| * Avoid Calling LZ4_prepareTable() in LZ4_compress_fast_continue()W. Felix Handte2018-04-091-20/+14
| |
* | fixed minor conversion warningtest49732018-04-101-1/+2
| | | | | | | | ptr diff -> U32
* | Merge branch 'dev' into lowAddrtest49732018-04-091-21/+17
|\ \ | |/
| * Return to Allowing Early Returns in LZ4_compress_generic()W. Felix Handte2018-04-061-21/+17
| | | | | | | | | | | | Or: `goto` Considered Harmful Or: https://xkcd.com/292/
* | noticed a bug when re-using hash tabletest49732018-04-061-3/+4
| | | | | | | | ./fuzzer -vv -s4217 -t7518
* | fixed byPtr modetest49732018-04-061-22/+25
| | | | | | | | | | | | switch to byU32 when src address is < 64K note : byPtr is still useful in 32-bits, as it's about ~10% faster
* | fixed byPtr match searchtest49732018-04-061-1/+2
| |
* | fixed immediate match searchtest49732018-04-061-7/+5
| |
* | changed LZ4_compress_generic() logictest49732018-04-051-70/+100
| | | | | | | | | | | | to use indexes (U32) instead of Ptr. byPtr is still present.
* | fixed lz4 compression starting at small addresstest49732018-04-051-3/+67
| | | | | | | | when using byU32 and byU16 modes
* | Merge branch 'dev' into lowAddrtest49732018-04-049-612/+851
|\ \ | |/
| * Merge pull request #487 from felixhandte/better-obsoletion-commentYann Collet2018-03-211-6/+10
| |\ | | | | | | Better Describe Functionality of Obsolete Streaming Functions
| | * Also Fix a CommentW. Felix Handte2018-03-211-1/+1
| | |
| | * Better Describe Functionality of Obsolete Streaming FunctionsW. Felix Handte2018-03-211-6/+10
| | |
| * | fix comment styleYann Collet2018-03-212-3/+3
| |/
| * Move LZ4_compress_fast_extState_noReset Declaration to Unstable SectionW. Felix Handte2018-03-141-8/+23
| |
| * Restore the Other Old Streaming Functions in a Degraded FashionW. Felix Handte2018-03-142-2/+28
| |
| * Switch ALLOC() to ALLOC_AND_ZERO() to Paper Over Existing Uninitialized ReadW. Felix Handte2018-03-131-1/+1
| |
| * Split lz4CtxLevel into Two FieldsW. Felix Handte2018-03-131-17/+10
| |
generated by cgit v0.12 at 2024-07-22 00:38:25 (GMT)