| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|\
| |
| | |
conditional pattern analysis
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Pattern analysis (currently limited to long ranges of identical bytes)
is actually detrimental to performance
when `nbSearches` is low.
Reason is : `nbSearches` provides a built-in protection for these cases.
The problem with patterns is that they dramatically increase the number of candidates to visit.
But with a low nbSearches, the match finder just aborts early.
In such cases, pattern analysis adds some complexity without reducing total nb of candidates.
It actually increases compression ratio a little bit, by filtering only "good" candidates,
but at a measurable speed cost, so it's not a good trade-off.
This patch makes pattern analysis optional.
It's enabled for levels 8+ only.
|
|/
|
|
| |
no longer limited to level 9
|
|
|
|
|
|
|
|
|
|
|
|
| |
lz4opt is only competitive vs lz4hc level 10.
Below that level, it doesn't match the speed / compression effectiveness of regular hc parser.
This patch propose to extend lz4opt to levels 10-12.
The new level 10 tend to compress a bit better and a bit faster than previous one (mileage vary depending on file)
The only downside is that `limitedDestSize` mode is now limited to max level 9 (vs 10),
since it's only compatible with regular HC parser.
(Note : I suspect it's possible to convert lz4opt to support it too, but haven't spent time into it).
|
| |
|
| |
|
|
|
|
|
| |
for multi-bytes patterns
(which is not useful for the time being)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The first byte used to be skipped
to avoid a infinite self-comparison.
This is no longer necessary, since init() ensures that index starts at 64K.
The first byte is also useless to search when each block is independent,
but it's no longer the case when blocks are linked.
Removing the first-byte-skip saves
about 10 bytes / MB on files compressed with -BD4 (linked blocks 64Kb),
which feels correct as each MB has 16 blocks of 64KB.
|
| |
|
| |
|
|
|
|
| |
as reported by @terrelln
|
|
|
|
|
| |
works for any repetitive pattern of length 1, 2 or 4 (but not 3!)
works for any endianess
|
|
|
|
|
|
|
| |
- works with byte values other than `0`
- works for any repetitive pattern of length 1, 2 or 4 (but not 3!)
- works for little and big endian systems
- preserve speed of previous implementation
|
|
|
|
| |
dead assignment
|
|
|
|
|
| |
LZ4_setCompressionLevel() can be users accross the whole range of HC levels
No more transition issue between Optimal and HC modes
|
|
|
|
| |
responsibility better handled one layer above (LZ4HC_compress_generic())
|
|
|
|
|
| |
nbSearches now transmitted directly as function parameter
easier to track and debug
|
|
|
|
| |
levels 11+
|
|
|
|
| |
reduced size of LZ4HC state
|
| |
|
|
|
|
| |
improving speed
|
| |
|
|
|
|
| |
tag interesting places
|
|
|
|
|
| |
even gained 2 bytes on calgary.tar...
added conditional traces `g_debuglog_enable`
|
|
|
|
|
|
|
| |
previous strategy would leave a few "bad choices"
on the ground they would be fixed later,
but that requires passing through each position to make the fix
and cannot give the end position of the last useful match.
|
|
|
|
| |
not a success yet
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
| |
compilation when used with other libraries that define FORCE_INLINE
|
|
|
|
| |
Not obvious : copying the state was copying cdict's compression level
|
|
|
|
|
| |
note : only compression API is implemented and tested
still to do : decompression API
|
|
|
|
|
| |
For the default Wimplicit-fallthrough=3 level,
the comment should start with "fall*"
|
|
|
|
|
|
|
| |
at the expense of a little bit of compression ratio.
Now speed is intermediate on calgary corpus :
25 - 12 - 8 - 3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I noticed that, while 'lz4 -12' works fine, 'lz4 -13' does not compress
at all.
$ cat </etc/passwd |wc -c
2565
$ lz4 -12 </etc/passwd |wc -c
1456
$ lz4 -13 </etc/passwd |wc -c
2584
Perhaps the best way to fix this is to clamp the compression level
in the LZ4HC_compress_generic routine.
|
| |
|
| |
|
|
|
|
|
| |
It's incompatible with btopt though,
so cLevel >= 11 feature much reduced performance (degraded mode)
|
|
|
|
|
| |
Limits compression level to 10,
to remain compatible with Hash Chain.
|
|
|
|
| |
where are exposed new prototypes *_destSize()
|
| |
|
| |
|
| |
|
| |
|