summaryrefslogtreecommitdiffstats
path: root/examples/blockStreaming_doubleBuffer.md
diff options
context:
space:
mode:
authorTakayuki MATSUOKA <takayuki.matsuoka@gmail.com>2015-03-24 22:52:17 (GMT)
committerTakayuki MATSUOKA <takayuki.matsuoka@gmail.com>2015-03-24 22:52:17 (GMT)
commit438fee9169d2267f46ddd54f770b5f22a61ebb86 (patch)
tree48f870c63a986f0a0e830d8ba8ee3463157fd8dd /examples/blockStreaming_doubleBuffer.md
parenta38166bd5261ac8ff93dad8fa87eb6ee3deeea2d (diff)
downloadlz4-438fee9169d2267f46ddd54f770b5f22a61ebb86.zip
lz4-438fee9169d2267f46ddd54f770b5f22a61ebb86.tar.gz
lz4-438fee9169d2267f46ddd54f770b5f22a61ebb86.tar.bz2
Add document for "Double Buffer" example
Diffstat (limited to 'examples/blockStreaming_doubleBuffer.md')
-rw-r--r--examples/blockStreaming_doubleBuffer.md99
1 files changed, 99 insertions, 0 deletions
diff --git a/examples/blockStreaming_doubleBuffer.md b/examples/blockStreaming_doubleBuffer.md
new file mode 100644
index 0000000..cd295df
--- /dev/null
+++ b/examples/blockStreaming_doubleBuffer.md
@@ -0,0 +1,99 @@
+# LZ4 Streaming API Example : Double Buffer
+
+`blockStreaming_doubleBuffer.c` is LZ4 Straming API example which implements double buffer (de)compression.
+
+Please note :
+
+ - Firstly, read "LZ4 Streaming API Basics".
+ - This is relatively advanced application example.
+ - Output file is not compatible with lz4frame and platform dependent.
+
+
+## What's the point of this example ?
+
+ - Handle huge file in small amount of memory
+ - Always better compression ratio than Block API
+ - Uniform block size
+
+
+## How the compression works
+
+First of all, allocate "Double Buffer" for input and LZ4 compressed data buffer for output.
+Double buffer has two pages, "first" page (Page#1) and "second" page (Page#2).
+
+```
+ Double Buffer
+
+ Page#1 Page#2
+ +---------+---------+
+ | Block#1 | |
+ +----+----+---------+
+ |
+ v
+ {Out#1}
+
+
+ Prefix Dependency
+ +---------+
+ | |
+ v |
+ +---------+----+----+
+ | Block#1 | Block#2 |
+ +---------+----+----+
+ |
+ v
+ {Out#2}
+
+
+ External Dictionary Mode
+ +---------+
+ | |
+ | v
+ +----+----+---------+
+ | Block#3 | Block#2 |
+ +----+----+---------+
+ |
+ v
+ {Out#3}
+
+
+ Prefix Dependency
+ +---------+
+ | |
+ v |
+ +---------+----+----+
+ | Block#3 | Block#4 |
+ +---------+----+----+
+ |
+ v
+ {Out#4}
+```
+
+Next, read first block to double buffer's first page. And compress it by `LZ4_compress_continue()`.
+For the first time, LZ4 doesn't know any previous dependencies,
+so it just compress the line without dependencies and generates compressed block {Out#1} to LZ4 compressed data buffer.
+After that, write {Out#1} to the file.
+
+Next, read second block to double buffer's second page. And compress it.
+In this time, LZ4 can use dependency to Block#1 to improve compression ratio.
+This dependency is called "Prefix mode".
+
+Next, read third block to double buffer's *first* page. And compress it.
+Also this time, LZ4 can use dependency to Block#2.
+This dependency is called "External Dictonaly mode".
+
+Continue these procedure to the end of the file.
+
+
+## How the decompression works
+
+Decompression will do reverse order.
+
+ - Read first compressed block.
+ - Decompress it to the first page and write that page to the file.
+ - Read second compressed block.
+ - Decompress it to the second page and write that page to the file.
+ - Read third compressed block.
+ - Decompress it to the *first* page and write that page to the file.
+
+Continue these procedure to the end of the compressed file.