zune-inflate-0.2.54/.cargo_vcs_info.json 0000644 00000000152 00000000001 0013523 0 ustar {
"git": {
"sha1": "69502ce83fdfecdd0beefd677e2abb3781b29d98"
},
"path_in_vcs": "zune-inflate"
} zune-inflate-0.2.54/CHANGELOG.md 0000644 0000000 0000000 00000001503 10461020230 0014125 0 ustar 0000000 0000000 ## Version 0.2.54
- Add simple encoder
- Fix no_std compilation
## Version 0.2.52
- Add small fix for refilling where the decoder lacked bits
## Version 0.2.51
- Correctly check for limits in the inner loop
## Version 0.2.0
- Initial release
## Version 0.2.1
- Fix bug where raw deflate outputs would cause errors.
## Version 0.2.2
- Fix bug in which some paths would cause the stream not to refill
## Version 0.2.3
- Small performance improvements, especially on files with a lot of RLE redundant data
## Version 0.2.4
- Fix bug with some gzip that would cause errors during decoding
- Small performance improvement
## Version 0.2.41
- Improve documentation of exposed values
## Version 0.2.42
- Remove broken links in README.
## Version 0.2.50
- Mark library as `#[no_std]`
- Impl `std::error::Error` for library zune-inflate-0.2.54/Cargo.toml 0000644 00000002051 00000000001 0011521 0 ustar # THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
#
# When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g., crates.io) dependencies.
#
# If you are reading this file be aware that the original Cargo.toml
# will likely look very different (and much more reasonable).
# See Cargo.toml.orig for the original contents.
[package]
edition = "2021"
name = "zune-inflate"
version = "0.2.54"
exclude = ["tests/"]
description = "A heavily optimized deflate decompressor in Pure Rust"
homepage = "https://github.com/etemesi254/zune-image/tree/main/zune-inflate"
readme = "README.md"
keywords = [
"compression",
"inflate",
"deflate",
]
categories = ["compression"]
license = "MIT OR Apache-2.0 OR Zlib"
[dependencies.simd-adler32]
version = "0.3.4"
optional = true
default-features = false
[features]
default = [
"zlib",
"gzip",
"std",
]
gzip = []
std = ["simd-adler32/std"]
zlib = ["simd-adler32"]
zune-inflate-0.2.54/Cargo.toml.orig 0000644 0000000 0000000 00000001216 10461020230 0015204 0 ustar 0000000 0000000 [package]
name = "zune-inflate"
version = "0.2.54"
edition = "2021"
description = "A heavily optimized deflate decompressor in Pure Rust"
exclude = ["tests/"]
homepage = "https://github.com/etemesi254/zune-image/tree/main/zune-inflate"
keywords = ["compression", "inflate", "deflate"]
categories = ["compression"]
license = "MIT OR Apache-2.0 OR Zlib"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[features]
zlib = ["simd-adler32"]
gzip = []
std = ["simd-adler32/std"]
default = ["zlib", "gzip", "std"]
[dependencies]
simd-adler32 = { version = "0.3.4", optional = true, default-features = false }
zune-inflate-0.2.54/README.md 0000644 0000000 0000000 00000006546 10461020230 0013607 0 ustar 0000000 0000000 # zune-inflate
This crate features an optimized inflate algorithm supporting
whole buffer decompression.
Supported formats are
- raw deflate
- zlib (deflate with a zlib wrapper on)
- gzip
The implementation is heavily based on Eric Biggers [libdeflate] and hence
has similar characteristics.
Specifically, we do not support streaming decompression but prefer whole buffer decompression.
## Installation
To use in your crate, simply add the following in your
Cargo.toml
```toml
[dependencies]
#other amazing crates from other amazing people
zune-inflate = "0.2.0"
```
## Features
One can enable or disable a specific format using
cargo features.
Specifically, the following can be enabled
- `gzip`: Enable decompressing of gzip encoded data
- `zlib`: Enable decompressing of zlib encoded data
To enable one feature, modify `Cargo.toml` entry to be
```toml
[dependencies]
zune-inflate = { version = "0.2", default-features = false, features = ["#ADD_SPECIFIC_FEATURE"] }
```
## Usage.
The library exposes a simple API for decompressing
data, and depending on what type of data you have, you typically choose
one of the `decode[_suffix]` function to decode your data
The decompressor expects the whole buffer handed upfront
### Decoding raw deflate
To decode raw deflate data, the following code should get you
started.
```rust
use zune_inflate::DeflateDecoder;
let totally_valid_data = [0; 23];
let mut decoder = DeflateDecoder::new( & totally_valid_data);
// panic on errors, because that's the cool way to go
let decompressed_data = decoder.decode_deflate().unwrap();
```
### Decoding zlib
To decode deflate data wrapped in zlib, the following code should get you
started.
```rust
use zune_inflate::DeflateDecoder;
let totally_valid_data = [0; 23];
let mut decoder = DeflateDecoder::new( & totally_valid_data);
// panic on errors, because that's the cool way to go
let decompressed_data = decoder.decode_zlib().unwrap();
```
### Advanced usage
There are advanced options specified by `DeflateOptions` which can change
decompression settings.
## Comparisions.
I'll compare this with `flate2` with `miniz-oxide` backend.
| feature | `zune-inflate` | `flate2` |
|-------------------------|----------------|-------------------|
| zlib decompression | yes | yes |
| delfate decompression | yes | yes |
| gzip | yes | yes |
| compression | soon | yes |
| streaming decompression | no | yes |
| **unsafe** | no | yes[1] |
[1] Flate writes to an uninitialized buffer
As you can see, there are some concrete features we currently lack when compared to
flate2/miniz-oxide.
There's actually nothing riding in for us, except...it's wickedly fast...
### Benchmarks
Up-to date benchmarks are done using criterion and hosted online at [zune-benchmarks] site,
benchmarks for this library have the `inflate: ` prefix.
## Fuzzing
The decoder is currently fuzzed for correctness by both `miniz-oxide` and `zlib-ng`, see the fuzz/src directory
[libdeflater]: https://github.com/adamkewley/libdeflater
[libdeflate]:https://github.com/ebiggers/libdeflate
[criterion]:https://github.com/bheisler/criterion.rs
[zune-benchmarks]:https://etemesi254.github.io/posts/Zune-Benchmarks/ zune-inflate-0.2.54/src/bitstream.rs 0000644 0000000 0000000 00000012411 10461020230 0015443 0 ustar 0000000 0000000 //! `BitStreamReader` API
//!
//! This module provides an interface to read and write bits (and bytes) for
//! huffman
pub struct BitStreamReader<'src>
{
// buffer from which we are pulling in bits from
// used in decompression.
pub src: &'src [u8],
// position in our buffer,
pub position: usize,
pub bits_left: u8,
pub buffer: u64,
pub over_read: usize
}
impl<'src> BitStreamReader<'src>
{
/// Create a new `BitStreamReader` instance
///
/// # Expectations
/// The buffer must be padded with fill bytes in the end,
/// if not, this becomes UB in the refill phase.
pub fn new(in_buffer: &'src [u8]) -> BitStreamReader<'src>
{
BitStreamReader {
bits_left: 0,
buffer: 0,
src: in_buffer,
position: 0,
over_read: 0
}
}
/// Refill the bitstream ensuring the buffer has bits between
/// 56 and 63.
///
#[inline(always)]
pub fn refill(&mut self)
{
/*
* The refill always guarantees refills between 56-63
*
* Bits stored will never go above 63 and if bits are in the range 56-63 no refills occur.
*/
let mut buf = [0; 8];
match self.src.get(self.position..self.position + 8)
{
Some(bytes) =>
{
buf.copy_from_slice(bytes);
// create a u64 from an array of u8's
let new_buffer = u64::from_le_bytes(buf);
// num indicates how many bytes we actually consumed.
let num = 63 ^ self.bits_left;
// offset position
self.position += (num >> 3) as usize;
// shift number of bits
self.buffer |= new_buffer << self.bits_left;
// update bits left
// bits left are now between 56-63
self.bits_left |= 56;
}
None => self.refill_slow()
}
}
#[inline(always)]
pub fn refill_inner_loop(&mut self)
{
/*
* The refill always guarantees refills between 56-63
*
* Bits stored will never go above 63 and if bits are in the range 56-63 no refills occur.
*/
let mut buf = [0; 8];
if let Some(bytes) = self.src.get(self.position..self.position + 8)
{
{
buf.copy_from_slice(bytes);
// create a u64 from an array of u8's
let new_buffer = u64::from_le_bytes(buf);
// num indicates how many bytes we actually consumed.
let num = 63 ^ self.bits_left;
// offset position
self.position += (num >> 3) as usize;
// shift number of bits
self.buffer |= new_buffer << self.bits_left;
// update bits left
// bits left are now between 56-63
self.bits_left |= 56;
}
}
}
#[inline(never)]
fn refill_slow(&mut self)
{
let bytes = &self.src[self.position..];
for byte in bytes
{
if self.bits_left >= 56
{
break;
}
self.buffer |= u64::from(*byte) << self.bits_left;
self.bits_left += 8;
self.position += 1;
}
while self.bits_left < 56
{
self.bits_left += 8;
self.over_read += 1;
}
}
#[inline(always)]
pub fn peek_bits(&self) -> usize
{
debug_assert!(self.bits_left >= LOOKAHEAD as u8);
(self.buffer & ((1 << LOOKAHEAD) - 1)) as usize
}
#[inline(always)]
pub fn peek_var_bits(&self, lookahead: usize) -> usize
{
debug_assert!(self.bits_left >= lookahead as u8);
(self.buffer & ((1 << lookahead) - 1)) as usize
}
#[inline(always)]
pub fn get_bits(&mut self, num_bits: u8) -> u64
{
debug_assert!(self.bits_left >= num_bits);
let mask = (1_u64 << num_bits) - 1;
let value = self.buffer & mask;
self.buffer >>= num_bits;
self.bits_left -= num_bits;
value
}
/// Get number of bits left in the bit buffer.
pub const fn get_bits_left(&self) -> u8
{
self.bits_left
}
/// Get position the stream is in this buffer
/// Or alternatively, number of bits read.
pub fn get_position(&self) -> usize
{
self.position
.saturating_sub(usize::from(self.bits_left >> 3))
}
/// Reset buffer and bits left to zero.
pub fn reset(&mut self)
{
self.buffer = 0;
self.bits_left = 0;
}
/// Return true if the bit buffer can satisfy
/// `bits` read without refilling,
pub const fn has(&self, bits: u8) -> bool
{
self.bits_left >= bits
}
#[inline(always)]
pub fn drop_bits(&mut self, bits: u8)
{
debug_assert!(self.bits_left >= bits);
self.bits_left -= bits;
self.buffer >>= bits;
}
/// Return the remaining bytes in this stream.
///
/// This does not consider bits in the bit-buffer hence
/// may not be accurate
pub const fn remaining_bytes(&self) -> usize
{
self.src.len().saturating_sub(self.position)
}
}
zune-inflate-0.2.54/src/constants.rs 0000644 0000000 0000000 00000012131 10461020230 0015464 0 ustar 0000000 0000000 use crate::utils::const_min_usize;
/// Number of symbols in each Huffman code. Note: for the literal/length
/// and offset codes, these are actually the maximum values; a given block
/// might use fewer symbols.
pub const DEFLATE_NUM_PRECODE_SYMS: usize = 19;
pub const DEFLATE_NUM_LITLEN_SYMS: usize = 288;
pub const DEFLATE_NUM_OFFSET_SYMS: usize = 32;
/// Maximum possible overrun when decoding codeword lengths
pub const DELFATE_MAX_LENS_OVERRUN: usize = 137;
/// Order which precode lengths are stored
pub static DEFLATE_PRECODE_LENS_PERMUTATION: [u8; DEFLATE_NUM_PRECODE_SYMS] = [
16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15
];
pub const PRECODE_ENOUGH: usize = 128;
/// Maximum codeword length across all codes.
pub const DEFLATE_MAX_CODEWORD_LENGTH: usize = 15;
pub const DEFLATE_MAX_OFFSET_CODEWORD_LENGTH: usize = 15;
pub const DEFLATE_MAX_LITLEN_CODEWORD_LENGTH: usize = 15;
pub const PRECODE_TABLE_BITS: usize = 7;
pub const LITLEN_TABLE_BITS: usize = 11;
pub const LITLEN_ENOUGH: usize = 2342;
/// Maximum bits found in the lookup table for offsets
/// offsets larger than this require a lookup into a sub-table
pub const OFFSET_TABLEBITS: usize = 8;
/// Note, default libdeflate value is 402, but with 512,
/// we can remove a branch check by simply doing & 511, and I'll take that.
pub const OFFSET_ENOUGH: usize = 512;
/// Maximum number of symbols across all codes
pub const DEFLATE_MAX_NUM_SYMS: usize = 288;
///Maximum codeword length in bits for each precode
pub const DEFLATE_MAX_PRE_CODEWORD_LEN: u8 = 7;
/// Format for precode decode table entries, Bits not explicitly contain zeroes
///
/// 20-16: presym
/// 10-8 Codeword length(not used)
/// Bit 2-0 Codeword length
///
/// It never has sub-tables since we use PRECODE_TABLEBITS == MAX_PRECODEWORD_LENGTH
///
/// PRECODE_DECODE_RESULTS contains static parts of the entry for each symbol,
/// make_decode_table_entry produces the final results
pub static PRECODE_DECODE_RESULTS: [u32; 19] = make_precode_static_table();
const fn make_precode_static_table() -> [u32; 19]
{
let mut table: [u32; 19] = [0; 19];
let mut i = 0;
while i < 19
{
table[i] = (i as u32) << 16;
i += 1;
}
table
}
/// Presence of a literal entry
pub const HUFFDEC_LITERAL: u32 = 0x80000000;
/// Presence of HUFFDEC_SUITABLE_POINTER or HUFFDEC_END_OF_BLOCK
pub const HUFFDEC_EXCEPTIONAL: u32 = 0x00008000;
/// Pointer entry in the litlen or offset decode table
pub const HUFFDEC_SUITABLE_POINTER: u32 = 0x00004000;
/// End of block entry in litlen decode table
pub const HUFFDEC_END_OF_BLOCK: u32 = 0x00002000;
#[rustfmt::skip]
#[allow(clippy::zero_prefixed_literal)]
const fn construct_litlen_decode_table() -> [u32; 288]
{
let mut results: [u32; 288] = [0; 288];
let mut i = 0;
while i < 256
{
results[i] = ((i as u32) << 16) | HUFFDEC_LITERAL;
i += 1;
}
results[i] = HUFFDEC_EXCEPTIONAL | HUFFDEC_END_OF_BLOCK;
i += 1;
let base_and_bits_tables = [
(003, 0), (004, 0), (005, 0), (006, 0),
(007, 0), (008, 0), (009, 0), (010, 0),
(011, 1), (013, 1), (015, 1), (017, 1),
(019, 2), (023, 2), (027, 2), (031, 2),
(035, 3), (043, 3), (051, 3), (059, 3),
(067, 4), (083, 4), (099, 4), (115, 4),
(131, 5), (163, 5), (195, 5), (227, 5),
(258, 0), (258, 0), (258, 0),
];
let mut j = 0;
while i < 288
{
let (length_base, extra_bits) = base_and_bits_tables[j];
results[i] = (length_base << 16) | extra_bits;
i += 1;
j += 1;
}
results
}
const fn entry(base: u32, extra: u32) -> u32
{
base << 16 | extra
}
#[rustfmt::skip]
#[allow(clippy::zero_prefixed_literal)] // the things we do for alignment
pub static OFFSET_DECODE_RESULTS: [u32; 32] = [
entry(00001, 00), entry(00002, 00), entry(00003, 00), entry(00004, 00),
entry(00005, 01), entry(00007, 01), entry(00009, 02), entry(00013, 02),
entry(00017, 03), entry(00025, 03), entry(00033, 04), entry(00049, 04),
entry(00065, 05), entry(00097, 05), entry(00129, 06), entry(00193, 06),
entry(00257, 07), entry(00385, 07), entry(00513, 08), entry(00769, 08),
entry(01025, 09), entry(01537, 09), entry(02049, 10), entry(03073, 10),
entry(04097, 11), entry(06145, 11), entry(08193, 12), entry(12289, 12),
entry(16385, 13), entry(24577, 13), entry(24577, 13), entry(24577, 13),
];
pub static LITLEN_DECODE_RESULTS: [u32; 288] = construct_litlen_decode_table();
pub const DEFLATE_BLOCKTYPE_DYNAMIC_HUFFMAN: u64 = 2;
pub const DEFLATE_BLOCKTYPE_UNCOMPRESSED: u64 = 0;
pub const DEFLATE_BLOCKTYPE_RESERVED: u64 = 3;
pub const DEFLATE_BLOCKTYPE_STATIC: u64 = 1;
pub const LITLEN_DECODE_BITS: usize =
const_min_usize(DEFLATE_MAX_LITLEN_CODEWORD_LENGTH, LITLEN_TABLE_BITS);
/// Maximum length of a deflate match
pub const DEFLATE_MAX_MATCH_LEN: usize = 258;
/// Number of bytes copied per every loop
pub const FASTCOPY_BYTES: usize = 16;
/// Worst case maximum number of output bytes writtern during each iteration of the
/// fastloop.
pub const FASTLOOP_MAX_BYTES_WRITTEN: usize = 6 + DEFLATE_MAX_MATCH_LEN + (2 * FASTCOPY_BYTES);
zune-inflate-0.2.54/src/crc/crc_tables.rs 0000644 0000000 0000000 00000070476 10461020230 0016340 0 ustar 0000000 0000000 //! crc tables- Data tables for CRC32 computation
//! Gotten from Eric Biggers libdeflate.
pub static CRC32_SLICE1_TABLE: [u32; 256] = [
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3,
0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91,
0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5,
0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f,
0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d,
0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433,
0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457,
0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65,
0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb,
0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9,
0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad,
0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683,
0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1,
0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7,
0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b,
0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79,
0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f,
0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d,
0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21,
0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777,
0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45,
0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db,
0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf,
0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d
];
pub static CRC32_SLICE8_TABLE: [u32; 2048] = [
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3,
0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91,
0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5,
0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f,
0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d,
0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433,
0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457,
0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65,
0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb,
0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9,
0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad,
0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683,
0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1,
0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7,
0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b,
0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79,
0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f,
0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d,
0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21,
0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777,
0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45,
0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db,
0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf,
0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d,
0x00000000, 0x191b3141, 0x32366282, 0x2b2d53c3, 0x646cc504, 0x7d77f445, 0x565aa786, 0x4f4196c7,
0xc8d98a08, 0xd1c2bb49, 0xfaefe88a, 0xe3f4d9cb, 0xacb54f0c, 0xb5ae7e4d, 0x9e832d8e, 0x87981ccf,
0x4ac21251, 0x53d92310, 0x78f470d3, 0x61ef4192, 0x2eaed755, 0x37b5e614, 0x1c98b5d7, 0x05838496,
0x821b9859, 0x9b00a918, 0xb02dfadb, 0xa936cb9a, 0xe6775d5d, 0xff6c6c1c, 0xd4413fdf, 0xcd5a0e9e,
0x958424a2, 0x8c9f15e3, 0xa7b24620, 0xbea97761, 0xf1e8e1a6, 0xe8f3d0e7, 0xc3de8324, 0xdac5b265,
0x5d5daeaa, 0x44469feb, 0x6f6bcc28, 0x7670fd69, 0x39316bae, 0x202a5aef, 0x0b07092c, 0x121c386d,
0xdf4636f3, 0xc65d07b2, 0xed705471, 0xf46b6530, 0xbb2af3f7, 0xa231c2b6, 0x891c9175, 0x9007a034,
0x179fbcfb, 0x0e848dba, 0x25a9de79, 0x3cb2ef38, 0x73f379ff, 0x6ae848be, 0x41c51b7d, 0x58de2a3c,
0xf0794f05, 0xe9627e44, 0xc24f2d87, 0xdb541cc6, 0x94158a01, 0x8d0ebb40, 0xa623e883, 0xbf38d9c2,
0x38a0c50d, 0x21bbf44c, 0x0a96a78f, 0x138d96ce, 0x5ccc0009, 0x45d73148, 0x6efa628b, 0x77e153ca,
0xbabb5d54, 0xa3a06c15, 0x888d3fd6, 0x91960e97, 0xded79850, 0xc7cca911, 0xece1fad2, 0xf5facb93,
0x7262d75c, 0x6b79e61d, 0x4054b5de, 0x594f849f, 0x160e1258, 0x0f152319, 0x243870da, 0x3d23419b,
0x65fd6ba7, 0x7ce65ae6, 0x57cb0925, 0x4ed03864, 0x0191aea3, 0x188a9fe2, 0x33a7cc21, 0x2abcfd60,
0xad24e1af, 0xb43fd0ee, 0x9f12832d, 0x8609b26c, 0xc94824ab, 0xd05315ea, 0xfb7e4629, 0xe2657768,
0x2f3f79f6, 0x362448b7, 0x1d091b74, 0x04122a35, 0x4b53bcf2, 0x52488db3, 0x7965de70, 0x607eef31,
0xe7e6f3fe, 0xfefdc2bf, 0xd5d0917c, 0xcccba03d, 0x838a36fa, 0x9a9107bb, 0xb1bc5478, 0xa8a76539,
0x3b83984b, 0x2298a90a, 0x09b5fac9, 0x10aecb88, 0x5fef5d4f, 0x46f46c0e, 0x6dd93fcd, 0x74c20e8c,
0xf35a1243, 0xea412302, 0xc16c70c1, 0xd8774180, 0x9736d747, 0x8e2de606, 0xa500b5c5, 0xbc1b8484,
0x71418a1a, 0x685abb5b, 0x4377e898, 0x5a6cd9d9, 0x152d4f1e, 0x0c367e5f, 0x271b2d9c, 0x3e001cdd,
0xb9980012, 0xa0833153, 0x8bae6290, 0x92b553d1, 0xddf4c516, 0xc4eff457, 0xefc2a794, 0xf6d996d5,
0xae07bce9, 0xb71c8da8, 0x9c31de6b, 0x852aef2a, 0xca6b79ed, 0xd37048ac, 0xf85d1b6f, 0xe1462a2e,
0x66de36e1, 0x7fc507a0, 0x54e85463, 0x4df36522, 0x02b2f3e5, 0x1ba9c2a4, 0x30849167, 0x299fa026,
0xe4c5aeb8, 0xfdde9ff9, 0xd6f3cc3a, 0xcfe8fd7b, 0x80a96bbc, 0x99b25afd, 0xb29f093e, 0xab84387f,
0x2c1c24b0, 0x350715f1, 0x1e2a4632, 0x07317773, 0x4870e1b4, 0x516bd0f5, 0x7a468336, 0x635db277,
0xcbfad74e, 0xd2e1e60f, 0xf9ccb5cc, 0xe0d7848d, 0xaf96124a, 0xb68d230b, 0x9da070c8, 0x84bb4189,
0x03235d46, 0x1a386c07, 0x31153fc4, 0x280e0e85, 0x674f9842, 0x7e54a903, 0x5579fac0, 0x4c62cb81,
0x8138c51f, 0x9823f45e, 0xb30ea79d, 0xaa1596dc, 0xe554001b, 0xfc4f315a, 0xd7626299, 0xce7953d8,
0x49e14f17, 0x50fa7e56, 0x7bd72d95, 0x62cc1cd4, 0x2d8d8a13, 0x3496bb52, 0x1fbbe891, 0x06a0d9d0,
0x5e7ef3ec, 0x4765c2ad, 0x6c48916e, 0x7553a02f, 0x3a1236e8, 0x230907a9, 0x0824546a, 0x113f652b,
0x96a779e4, 0x8fbc48a5, 0xa4911b66, 0xbd8a2a27, 0xf2cbbce0, 0xebd08da1, 0xc0fdde62, 0xd9e6ef23,
0x14bce1bd, 0x0da7d0fc, 0x268a833f, 0x3f91b27e, 0x70d024b9, 0x69cb15f8, 0x42e6463b, 0x5bfd777a,
0xdc656bb5, 0xc57e5af4, 0xee530937, 0xf7483876, 0xb809aeb1, 0xa1129ff0, 0x8a3fcc33, 0x9324fd72,
0x00000000, 0x01c26a37, 0x0384d46e, 0x0246be59, 0x0709a8dc, 0x06cbc2eb, 0x048d7cb2, 0x054f1685,
0x0e1351b8, 0x0fd13b8f, 0x0d9785d6, 0x0c55efe1, 0x091af964, 0x08d89353, 0x0a9e2d0a, 0x0b5c473d,
0x1c26a370, 0x1de4c947, 0x1fa2771e, 0x1e601d29, 0x1b2f0bac, 0x1aed619b, 0x18abdfc2, 0x1969b5f5,
0x1235f2c8, 0x13f798ff, 0x11b126a6, 0x10734c91, 0x153c5a14, 0x14fe3023, 0x16b88e7a, 0x177ae44d,
0x384d46e0, 0x398f2cd7, 0x3bc9928e, 0x3a0bf8b9, 0x3f44ee3c, 0x3e86840b, 0x3cc03a52, 0x3d025065,
0x365e1758, 0x379c7d6f, 0x35dac336, 0x3418a901, 0x3157bf84, 0x3095d5b3, 0x32d36bea, 0x331101dd,
0x246be590, 0x25a98fa7, 0x27ef31fe, 0x262d5bc9, 0x23624d4c, 0x22a0277b, 0x20e69922, 0x2124f315,
0x2a78b428, 0x2bbade1f, 0x29fc6046, 0x283e0a71, 0x2d711cf4, 0x2cb376c3, 0x2ef5c89a, 0x2f37a2ad,
0x709a8dc0, 0x7158e7f7, 0x731e59ae, 0x72dc3399, 0x7793251c, 0x76514f2b, 0x7417f172, 0x75d59b45,
0x7e89dc78, 0x7f4bb64f, 0x7d0d0816, 0x7ccf6221, 0x798074a4, 0x78421e93, 0x7a04a0ca, 0x7bc6cafd,
0x6cbc2eb0, 0x6d7e4487, 0x6f38fade, 0x6efa90e9, 0x6bb5866c, 0x6a77ec5b, 0x68315202, 0x69f33835,
0x62af7f08, 0x636d153f, 0x612bab66, 0x60e9c151, 0x65a6d7d4, 0x6464bde3, 0x662203ba, 0x67e0698d,
0x48d7cb20, 0x4915a117, 0x4b531f4e, 0x4a917579, 0x4fde63fc, 0x4e1c09cb, 0x4c5ab792, 0x4d98dda5,
0x46c49a98, 0x4706f0af, 0x45404ef6, 0x448224c1, 0x41cd3244, 0x400f5873, 0x4249e62a, 0x438b8c1d,
0x54f16850, 0x55330267, 0x5775bc3e, 0x56b7d609, 0x53f8c08c, 0x523aaabb, 0x507c14e2, 0x51be7ed5,
0x5ae239e8, 0x5b2053df, 0x5966ed86, 0x58a487b1, 0x5deb9134, 0x5c29fb03, 0x5e6f455a, 0x5fad2f6d,
0xe1351b80, 0xe0f771b7, 0xe2b1cfee, 0xe373a5d9, 0xe63cb35c, 0xe7fed96b, 0xe5b86732, 0xe47a0d05,
0xef264a38, 0xeee4200f, 0xeca29e56, 0xed60f461, 0xe82fe2e4, 0xe9ed88d3, 0xebab368a, 0xea695cbd,
0xfd13b8f0, 0xfcd1d2c7, 0xfe976c9e, 0xff5506a9, 0xfa1a102c, 0xfbd87a1b, 0xf99ec442, 0xf85cae75,
0xf300e948, 0xf2c2837f, 0xf0843d26, 0xf1465711, 0xf4094194, 0xf5cb2ba3, 0xf78d95fa, 0xf64fffcd,
0xd9785d60, 0xd8ba3757, 0xdafc890e, 0xdb3ee339, 0xde71f5bc, 0xdfb39f8b, 0xddf521d2, 0xdc374be5,
0xd76b0cd8, 0xd6a966ef, 0xd4efd8b6, 0xd52db281, 0xd062a404, 0xd1a0ce33, 0xd3e6706a, 0xd2241a5d,
0xc55efe10, 0xc49c9427, 0xc6da2a7e, 0xc7184049, 0xc25756cc, 0xc3953cfb, 0xc1d382a2, 0xc011e895,
0xcb4dafa8, 0xca8fc59f, 0xc8c97bc6, 0xc90b11f1, 0xcc440774, 0xcd866d43, 0xcfc0d31a, 0xce02b92d,
0x91af9640, 0x906dfc77, 0x922b422e, 0x93e92819, 0x96a63e9c, 0x976454ab, 0x9522eaf2, 0x94e080c5,
0x9fbcc7f8, 0x9e7eadcf, 0x9c381396, 0x9dfa79a1, 0x98b56f24, 0x99770513, 0x9b31bb4a, 0x9af3d17d,
0x8d893530, 0x8c4b5f07, 0x8e0de15e, 0x8fcf8b69, 0x8a809dec, 0x8b42f7db, 0x89044982, 0x88c623b5,
0x839a6488, 0x82580ebf, 0x801eb0e6, 0x81dcdad1, 0x8493cc54, 0x8551a663, 0x8717183a, 0x86d5720d,
0xa9e2d0a0, 0xa820ba97, 0xaa6604ce, 0xaba46ef9, 0xaeeb787c, 0xaf29124b, 0xad6fac12, 0xacadc625,
0xa7f18118, 0xa633eb2f, 0xa4755576, 0xa5b73f41, 0xa0f829c4, 0xa13a43f3, 0xa37cfdaa, 0xa2be979d,
0xb5c473d0, 0xb40619e7, 0xb640a7be, 0xb782cd89, 0xb2cddb0c, 0xb30fb13b, 0xb1490f62, 0xb08b6555,
0xbbd72268, 0xba15485f, 0xb853f606, 0xb9919c31, 0xbcde8ab4, 0xbd1ce083, 0xbf5a5eda, 0xbe9834ed,
0x00000000, 0xb8bc6765, 0xaa09c88b, 0x12b5afee, 0x8f629757, 0x37def032, 0x256b5fdc, 0x9dd738b9,
0xc5b428ef, 0x7d084f8a, 0x6fbde064, 0xd7018701, 0x4ad6bfb8, 0xf26ad8dd, 0xe0df7733, 0x58631056,
0x5019579f, 0xe8a530fa, 0xfa109f14, 0x42acf871, 0xdf7bc0c8, 0x67c7a7ad, 0x75720843, 0xcdce6f26,
0x95ad7f70, 0x2d111815, 0x3fa4b7fb, 0x8718d09e, 0x1acfe827, 0xa2738f42, 0xb0c620ac, 0x087a47c9,
0xa032af3e, 0x188ec85b, 0x0a3b67b5, 0xb28700d0, 0x2f503869, 0x97ec5f0c, 0x8559f0e2, 0x3de59787,
0x658687d1, 0xdd3ae0b4, 0xcf8f4f5a, 0x7733283f, 0xeae41086, 0x525877e3, 0x40edd80d, 0xf851bf68,
0xf02bf8a1, 0x48979fc4, 0x5a22302a, 0xe29e574f, 0x7f496ff6, 0xc7f50893, 0xd540a77d, 0x6dfcc018,
0x359fd04e, 0x8d23b72b, 0x9f9618c5, 0x272a7fa0, 0xbafd4719, 0x0241207c, 0x10f48f92, 0xa848e8f7,
0x9b14583d, 0x23a83f58, 0x311d90b6, 0x89a1f7d3, 0x1476cf6a, 0xaccaa80f, 0xbe7f07e1, 0x06c36084,
0x5ea070d2, 0xe61c17b7, 0xf4a9b859, 0x4c15df3c, 0xd1c2e785, 0x697e80e0, 0x7bcb2f0e, 0xc377486b,
0xcb0d0fa2, 0x73b168c7, 0x6104c729, 0xd9b8a04c, 0x446f98f5, 0xfcd3ff90, 0xee66507e, 0x56da371b,
0x0eb9274d, 0xb6054028, 0xa4b0efc6, 0x1c0c88a3, 0x81dbb01a, 0x3967d77f, 0x2bd27891, 0x936e1ff4,
0x3b26f703, 0x839a9066, 0x912f3f88, 0x299358ed, 0xb4446054, 0x0cf80731, 0x1e4da8df, 0xa6f1cfba,
0xfe92dfec, 0x462eb889, 0x549b1767, 0xec277002, 0x71f048bb, 0xc94c2fde, 0xdbf98030, 0x6345e755,
0x6b3fa09c, 0xd383c7f9, 0xc1366817, 0x798a0f72, 0xe45d37cb, 0x5ce150ae, 0x4e54ff40, 0xf6e89825,
0xae8b8873, 0x1637ef16, 0x048240f8, 0xbc3e279d, 0x21e91f24, 0x99557841, 0x8be0d7af, 0x335cb0ca,
0xed59b63b, 0x55e5d15e, 0x47507eb0, 0xffec19d5, 0x623b216c, 0xda874609, 0xc832e9e7, 0x708e8e82,
0x28ed9ed4, 0x9051f9b1, 0x82e4565f, 0x3a58313a, 0xa78f0983, 0x1f336ee6, 0x0d86c108, 0xb53aa66d,
0xbd40e1a4, 0x05fc86c1, 0x1749292f, 0xaff54e4a, 0x322276f3, 0x8a9e1196, 0x982bbe78, 0x2097d91d,
0x78f4c94b, 0xc048ae2e, 0xd2fd01c0, 0x6a4166a5, 0xf7965e1c, 0x4f2a3979, 0x5d9f9697, 0xe523f1f2,
0x4d6b1905, 0xf5d77e60, 0xe762d18e, 0x5fdeb6eb, 0xc2098e52, 0x7ab5e937, 0x680046d9, 0xd0bc21bc,
0x88df31ea, 0x3063568f, 0x22d6f961, 0x9a6a9e04, 0x07bda6bd, 0xbf01c1d8, 0xadb46e36, 0x15080953,
0x1d724e9a, 0xa5ce29ff, 0xb77b8611, 0x0fc7e174, 0x9210d9cd, 0x2aacbea8, 0x38191146, 0x80a57623,
0xd8c66675, 0x607a0110, 0x72cfaefe, 0xca73c99b, 0x57a4f122, 0xef189647, 0xfdad39a9, 0x45115ecc,
0x764dee06, 0xcef18963, 0xdc44268d, 0x64f841e8, 0xf92f7951, 0x41931e34, 0x5326b1da, 0xeb9ad6bf,
0xb3f9c6e9, 0x0b45a18c, 0x19f00e62, 0xa14c6907, 0x3c9b51be, 0x842736db, 0x96929935, 0x2e2efe50,
0x2654b999, 0x9ee8defc, 0x8c5d7112, 0x34e11677, 0xa9362ece, 0x118a49ab, 0x033fe645, 0xbb838120,
0xe3e09176, 0x5b5cf613, 0x49e959fd, 0xf1553e98, 0x6c820621, 0xd43e6144, 0xc68bceaa, 0x7e37a9cf,
0xd67f4138, 0x6ec3265d, 0x7c7689b3, 0xc4caeed6, 0x591dd66f, 0xe1a1b10a, 0xf3141ee4, 0x4ba87981,
0x13cb69d7, 0xab770eb2, 0xb9c2a15c, 0x017ec639, 0x9ca9fe80, 0x241599e5, 0x36a0360b, 0x8e1c516e,
0x866616a7, 0x3eda71c2, 0x2c6fde2c, 0x94d3b949, 0x090481f0, 0xb1b8e695, 0xa30d497b, 0x1bb12e1e,
0x43d23e48, 0xfb6e592d, 0xe9dbf6c3, 0x516791a6, 0xccb0a91f, 0x740cce7a, 0x66b96194, 0xde0506f1,
0x00000000, 0x3d6029b0, 0x7ac05360, 0x47a07ad0, 0xf580a6c0, 0xc8e08f70, 0x8f40f5a0, 0xb220dc10,
0x30704bc1, 0x0d106271, 0x4ab018a1, 0x77d03111, 0xc5f0ed01, 0xf890c4b1, 0xbf30be61, 0x825097d1,
0x60e09782, 0x5d80be32, 0x1a20c4e2, 0x2740ed52, 0x95603142, 0xa80018f2, 0xefa06222, 0xd2c04b92,
0x5090dc43, 0x6df0f5f3, 0x2a508f23, 0x1730a693, 0xa5107a83, 0x98705333, 0xdfd029e3, 0xe2b00053,
0xc1c12f04, 0xfca106b4, 0xbb017c64, 0x866155d4, 0x344189c4, 0x0921a074, 0x4e81daa4, 0x73e1f314,
0xf1b164c5, 0xccd14d75, 0x8b7137a5, 0xb6111e15, 0x0431c205, 0x3951ebb5, 0x7ef19165, 0x4391b8d5,
0xa121b886, 0x9c419136, 0xdbe1ebe6, 0xe681c256, 0x54a11e46, 0x69c137f6, 0x2e614d26, 0x13016496,
0x9151f347, 0xac31daf7, 0xeb91a027, 0xd6f18997, 0x64d15587, 0x59b17c37, 0x1e1106e7, 0x23712f57,
0x58f35849, 0x659371f9, 0x22330b29, 0x1f532299, 0xad73fe89, 0x9013d739, 0xd7b3ade9, 0xead38459,
0x68831388, 0x55e33a38, 0x124340e8, 0x2f236958, 0x9d03b548, 0xa0639cf8, 0xe7c3e628, 0xdaa3cf98,
0x3813cfcb, 0x0573e67b, 0x42d39cab, 0x7fb3b51b, 0xcd93690b, 0xf0f340bb, 0xb7533a6b, 0x8a3313db,
0x0863840a, 0x3503adba, 0x72a3d76a, 0x4fc3feda, 0xfde322ca, 0xc0830b7a, 0x872371aa, 0xba43581a,
0x9932774d, 0xa4525efd, 0xe3f2242d, 0xde920d9d, 0x6cb2d18d, 0x51d2f83d, 0x167282ed, 0x2b12ab5d,
0xa9423c8c, 0x9422153c, 0xd3826fec, 0xeee2465c, 0x5cc29a4c, 0x61a2b3fc, 0x2602c92c, 0x1b62e09c,
0xf9d2e0cf, 0xc4b2c97f, 0x8312b3af, 0xbe729a1f, 0x0c52460f, 0x31326fbf, 0x7692156f, 0x4bf23cdf,
0xc9a2ab0e, 0xf4c282be, 0xb362f86e, 0x8e02d1de, 0x3c220dce, 0x0142247e, 0x46e25eae, 0x7b82771e,
0xb1e6b092, 0x8c869922, 0xcb26e3f2, 0xf646ca42, 0x44661652, 0x79063fe2, 0x3ea64532, 0x03c66c82,
0x8196fb53, 0xbcf6d2e3, 0xfb56a833, 0xc6368183, 0x74165d93, 0x49767423, 0x0ed60ef3, 0x33b62743,
0xd1062710, 0xec660ea0, 0xabc67470, 0x96a65dc0, 0x248681d0, 0x19e6a860, 0x5e46d2b0, 0x6326fb00,
0xe1766cd1, 0xdc164561, 0x9bb63fb1, 0xa6d61601, 0x14f6ca11, 0x2996e3a1, 0x6e369971, 0x5356b0c1,
0x70279f96, 0x4d47b626, 0x0ae7ccf6, 0x3787e546, 0x85a73956, 0xb8c710e6, 0xff676a36, 0xc2074386,
0x4057d457, 0x7d37fde7, 0x3a978737, 0x07f7ae87, 0xb5d77297, 0x88b75b27, 0xcf1721f7, 0xf2770847,
0x10c70814, 0x2da721a4, 0x6a075b74, 0x576772c4, 0xe547aed4, 0xd8278764, 0x9f87fdb4, 0xa2e7d404,
0x20b743d5, 0x1dd76a65, 0x5a7710b5, 0x67173905, 0xd537e515, 0xe857cca5, 0xaff7b675, 0x92979fc5,
0xe915e8db, 0xd475c16b, 0x93d5bbbb, 0xaeb5920b, 0x1c954e1b, 0x21f567ab, 0x66551d7b, 0x5b3534cb,
0xd965a31a, 0xe4058aaa, 0xa3a5f07a, 0x9ec5d9ca, 0x2ce505da, 0x11852c6a, 0x562556ba, 0x6b457f0a,
0x89f57f59, 0xb49556e9, 0xf3352c39, 0xce550589, 0x7c75d999, 0x4115f029, 0x06b58af9, 0x3bd5a349,
0xb9853498, 0x84e51d28, 0xc34567f8, 0xfe254e48, 0x4c059258, 0x7165bbe8, 0x36c5c138, 0x0ba5e888,
0x28d4c7df, 0x15b4ee6f, 0x521494bf, 0x6f74bd0f, 0xdd54611f, 0xe03448af, 0xa794327f, 0x9af41bcf,
0x18a48c1e, 0x25c4a5ae, 0x6264df7e, 0x5f04f6ce, 0xed242ade, 0xd044036e, 0x97e479be, 0xaa84500e,
0x4834505d, 0x755479ed, 0x32f4033d, 0x0f942a8d, 0xbdb4f69d, 0x80d4df2d, 0xc774a5fd, 0xfa148c4d,
0x78441b9c, 0x4524322c, 0x028448fc, 0x3fe4614c, 0x8dc4bd5c, 0xb0a494ec, 0xf704ee3c, 0xca64c78c,
0x00000000, 0xcb5cd3a5, 0x4dc8a10b, 0x869472ae, 0x9b914216, 0x50cd91b3, 0xd659e31d, 0x1d0530b8,
0xec53826d, 0x270f51c8, 0xa19b2366, 0x6ac7f0c3, 0x77c2c07b, 0xbc9e13de, 0x3a0a6170, 0xf156b2d5,
0x03d6029b, 0xc88ad13e, 0x4e1ea390, 0x85427035, 0x9847408d, 0x531b9328, 0xd58fe186, 0x1ed33223,
0xef8580f6, 0x24d95353, 0xa24d21fd, 0x6911f258, 0x7414c2e0, 0xbf481145, 0x39dc63eb, 0xf280b04e,
0x07ac0536, 0xccf0d693, 0x4a64a43d, 0x81387798, 0x9c3d4720, 0x57619485, 0xd1f5e62b, 0x1aa9358e,
0xebff875b, 0x20a354fe, 0xa6372650, 0x6d6bf5f5, 0x706ec54d, 0xbb3216e8, 0x3da66446, 0xf6fab7e3,
0x047a07ad, 0xcf26d408, 0x49b2a6a6, 0x82ee7503, 0x9feb45bb, 0x54b7961e, 0xd223e4b0, 0x197f3715,
0xe82985c0, 0x23755665, 0xa5e124cb, 0x6ebdf76e, 0x73b8c7d6, 0xb8e41473, 0x3e7066dd, 0xf52cb578,
0x0f580a6c, 0xc404d9c9, 0x4290ab67, 0x89cc78c2, 0x94c9487a, 0x5f959bdf, 0xd901e971, 0x125d3ad4,
0xe30b8801, 0x28575ba4, 0xaec3290a, 0x659ffaaf, 0x789aca17, 0xb3c619b2, 0x35526b1c, 0xfe0eb8b9,
0x0c8e08f7, 0xc7d2db52, 0x4146a9fc, 0x8a1a7a59, 0x971f4ae1, 0x5c439944, 0xdad7ebea, 0x118b384f,
0xe0dd8a9a, 0x2b81593f, 0xad152b91, 0x6649f834, 0x7b4cc88c, 0xb0101b29, 0x36846987, 0xfdd8ba22,
0x08f40f5a, 0xc3a8dcff, 0x453cae51, 0x8e607df4, 0x93654d4c, 0x58399ee9, 0xdeadec47, 0x15f13fe2,
0xe4a78d37, 0x2ffb5e92, 0xa96f2c3c, 0x6233ff99, 0x7f36cf21, 0xb46a1c84, 0x32fe6e2a, 0xf9a2bd8f,
0x0b220dc1, 0xc07ede64, 0x46eaacca, 0x8db67f6f, 0x90b34fd7, 0x5bef9c72, 0xdd7beedc, 0x16273d79,
0xe7718fac, 0x2c2d5c09, 0xaab92ea7, 0x61e5fd02, 0x7ce0cdba, 0xb7bc1e1f, 0x31286cb1, 0xfa74bf14,
0x1eb014d8, 0xd5ecc77d, 0x5378b5d3, 0x98246676, 0x852156ce, 0x4e7d856b, 0xc8e9f7c5, 0x03b52460,
0xf2e396b5, 0x39bf4510, 0xbf2b37be, 0x7477e41b, 0x6972d4a3, 0xa22e0706, 0x24ba75a8, 0xefe6a60d,
0x1d661643, 0xd63ac5e6, 0x50aeb748, 0x9bf264ed, 0x86f75455, 0x4dab87f0, 0xcb3ff55e, 0x006326fb,
0xf135942e, 0x3a69478b, 0xbcfd3525, 0x77a1e680, 0x6aa4d638, 0xa1f8059d, 0x276c7733, 0xec30a496,
0x191c11ee, 0xd240c24b, 0x54d4b0e5, 0x9f886340, 0x828d53f8, 0x49d1805d, 0xcf45f2f3, 0x04192156,
0xf54f9383, 0x3e134026, 0xb8873288, 0x73dbe12d, 0x6eded195, 0xa5820230, 0x2316709e, 0xe84aa33b,
0x1aca1375, 0xd196c0d0, 0x5702b27e, 0x9c5e61db, 0x815b5163, 0x4a0782c6, 0xcc93f068, 0x07cf23cd,
0xf6999118, 0x3dc542bd, 0xbb513013, 0x700de3b6, 0x6d08d30e, 0xa65400ab, 0x20c07205, 0xeb9ca1a0,
0x11e81eb4, 0xdab4cd11, 0x5c20bfbf, 0x977c6c1a, 0x8a795ca2, 0x41258f07, 0xc7b1fda9, 0x0ced2e0c,
0xfdbb9cd9, 0x36e74f7c, 0xb0733dd2, 0x7b2fee77, 0x662adecf, 0xad760d6a, 0x2be27fc4, 0xe0beac61,
0x123e1c2f, 0xd962cf8a, 0x5ff6bd24, 0x94aa6e81, 0x89af5e39, 0x42f38d9c, 0xc467ff32, 0x0f3b2c97,
0xfe6d9e42, 0x35314de7, 0xb3a53f49, 0x78f9ecec, 0x65fcdc54, 0xaea00ff1, 0x28347d5f, 0xe368aefa,
0x16441b82, 0xdd18c827, 0x5b8cba89, 0x90d0692c, 0x8dd55994, 0x46898a31, 0xc01df89f, 0x0b412b3a,
0xfa1799ef, 0x314b4a4a, 0xb7df38e4, 0x7c83eb41, 0x6186dbf9, 0xaada085c, 0x2c4e7af2, 0xe712a957,
0x15921919, 0xdececabc, 0x585ab812, 0x93066bb7, 0x8e035b0f, 0x455f88aa, 0xc3cbfa04, 0x089729a1,
0xf9c19b74, 0x329d48d1, 0xb4093a7f, 0x7f55e9da, 0x6250d962, 0xa90c0ac7, 0x2f987869, 0xe4c4abcc,
0x00000000, 0xa6770bb4, 0x979f1129, 0x31e81a9d, 0xf44f2413, 0x52382fa7, 0x63d0353a, 0xc5a73e8e,
0x33ef4e67, 0x959845d3, 0xa4705f4e, 0x020754fa, 0xc7a06a74, 0x61d761c0, 0x503f7b5d, 0xf64870e9,
0x67de9cce, 0xc1a9977a, 0xf0418de7, 0x56368653, 0x9391b8dd, 0x35e6b369, 0x040ea9f4, 0xa279a240,
0x5431d2a9, 0xf246d91d, 0xc3aec380, 0x65d9c834, 0xa07ef6ba, 0x0609fd0e, 0x37e1e793, 0x9196ec27,
0xcfbd399c, 0x69ca3228, 0x582228b5, 0xfe552301, 0x3bf21d8f, 0x9d85163b, 0xac6d0ca6, 0x0a1a0712,
0xfc5277fb, 0x5a257c4f, 0x6bcd66d2, 0xcdba6d66, 0x081d53e8, 0xae6a585c, 0x9f8242c1, 0x39f54975,
0xa863a552, 0x0e14aee6, 0x3ffcb47b, 0x998bbfcf, 0x5c2c8141, 0xfa5b8af5, 0xcbb39068, 0x6dc49bdc,
0x9b8ceb35, 0x3dfbe081, 0x0c13fa1c, 0xaa64f1a8, 0x6fc3cf26, 0xc9b4c492, 0xf85cde0f, 0x5e2bd5bb,
0x440b7579, 0xe27c7ecd, 0xd3946450, 0x75e36fe4, 0xb044516a, 0x16335ade, 0x27db4043, 0x81ac4bf7,
0x77e43b1e, 0xd19330aa, 0xe07b2a37, 0x460c2183, 0x83ab1f0d, 0x25dc14b9, 0x14340e24, 0xb2430590,
0x23d5e9b7, 0x85a2e203, 0xb44af89e, 0x123df32a, 0xd79acda4, 0x71edc610, 0x4005dc8d, 0xe672d739,
0x103aa7d0, 0xb64dac64, 0x87a5b6f9, 0x21d2bd4d, 0xe47583c3, 0x42028877, 0x73ea92ea, 0xd59d995e,
0x8bb64ce5, 0x2dc14751, 0x1c295dcc, 0xba5e5678, 0x7ff968f6, 0xd98e6342, 0xe86679df, 0x4e11726b,
0xb8590282, 0x1e2e0936, 0x2fc613ab, 0x89b1181f, 0x4c162691, 0xea612d25, 0xdb8937b8, 0x7dfe3c0c,
0xec68d02b, 0x4a1fdb9f, 0x7bf7c102, 0xdd80cab6, 0x1827f438, 0xbe50ff8c, 0x8fb8e511, 0x29cfeea5,
0xdf879e4c, 0x79f095f8, 0x48188f65, 0xee6f84d1, 0x2bc8ba5f, 0x8dbfb1eb, 0xbc57ab76, 0x1a20a0c2,
0x8816eaf2, 0x2e61e146, 0x1f89fbdb, 0xb9fef06f, 0x7c59cee1, 0xda2ec555, 0xebc6dfc8, 0x4db1d47c,
0xbbf9a495, 0x1d8eaf21, 0x2c66b5bc, 0x8a11be08, 0x4fb68086, 0xe9c18b32, 0xd82991af, 0x7e5e9a1b,
0xefc8763c, 0x49bf7d88, 0x78576715, 0xde206ca1, 0x1b87522f, 0xbdf0599b, 0x8c184306, 0x2a6f48b2,
0xdc27385b, 0x7a5033ef, 0x4bb82972, 0xedcf22c6, 0x28681c48, 0x8e1f17fc, 0xbff70d61, 0x198006d5,
0x47abd36e, 0xe1dcd8da, 0xd034c247, 0x7643c9f3, 0xb3e4f77d, 0x1593fcc9, 0x247be654, 0x820cede0,
0x74449d09, 0xd23396bd, 0xe3db8c20, 0x45ac8794, 0x800bb91a, 0x267cb2ae, 0x1794a833, 0xb1e3a387,
0x20754fa0, 0x86024414, 0xb7ea5e89, 0x119d553d, 0xd43a6bb3, 0x724d6007, 0x43a57a9a, 0xe5d2712e,
0x139a01c7, 0xb5ed0a73, 0x840510ee, 0x22721b5a, 0xe7d525d4, 0x41a22e60, 0x704a34fd, 0xd63d3f49,
0xcc1d9f8b, 0x6a6a943f, 0x5b828ea2, 0xfdf58516, 0x3852bb98, 0x9e25b02c, 0xafcdaab1, 0x09baa105,
0xfff2d1ec, 0x5985da58, 0x686dc0c5, 0xce1acb71, 0x0bbdf5ff, 0xadcafe4b, 0x9c22e4d6, 0x3a55ef62,
0xabc30345, 0x0db408f1, 0x3c5c126c, 0x9a2b19d8, 0x5f8c2756, 0xf9fb2ce2, 0xc813367f, 0x6e643dcb,
0x982c4d22, 0x3e5b4696, 0x0fb35c0b, 0xa9c457bf, 0x6c636931, 0xca146285, 0xfbfc7818, 0x5d8b73ac,
0x03a0a617, 0xa5d7ada3, 0x943fb73e, 0x3248bc8a, 0xf7ef8204, 0x519889b0, 0x6070932d, 0xc6079899,
0x304fe870, 0x9638e3c4, 0xa7d0f959, 0x01a7f2ed, 0xc400cc63, 0x6277c7d7, 0x539fdd4a, 0xf5e8d6fe,
0x647e3ad9, 0xc209316d, 0xf3e12bf0, 0x55962044, 0x90311eca, 0x3646157e, 0x07ae0fe3, 0xa1d90457,
0x579174be, 0xf1e67f0a, 0xc00e6597, 0x66796e23, 0xa3de50ad, 0x05a95b19, 0x34414184, 0x92364a30,
0x00000000, 0xccaa009e, 0x4225077d, 0x8e8f07e3, 0x844a0efa, 0x48e00e64, 0xc66f0987, 0x0ac50919,
0xd3e51bb5, 0x1f4f1b2b, 0x91c01cc8, 0x5d6a1c56, 0x57af154f, 0x9b0515d1, 0x158a1232, 0xd92012ac,
0x7cbb312b, 0xb01131b5, 0x3e9e3656, 0xf23436c8, 0xf8f13fd1, 0x345b3f4f, 0xbad438ac, 0x767e3832,
0xaf5e2a9e, 0x63f42a00, 0xed7b2de3, 0x21d12d7d, 0x2b142464, 0xe7be24fa, 0x69312319, 0xa59b2387,
0xf9766256, 0x35dc62c8, 0xbb53652b, 0x77f965b5, 0x7d3c6cac, 0xb1966c32, 0x3f196bd1, 0xf3b36b4f,
0x2a9379e3, 0xe639797d, 0x68b67e9e, 0xa41c7e00, 0xaed97719, 0x62737787, 0xecfc7064, 0x205670fa,
0x85cd537d, 0x496753e3, 0xc7e85400, 0x0b42549e, 0x01875d87, 0xcd2d5d19, 0x43a25afa, 0x8f085a64,
0x562848c8, 0x9a824856, 0x140d4fb5, 0xd8a74f2b, 0xd2624632, 0x1ec846ac, 0x9047414f, 0x5ced41d1,
0x299dc2ed, 0xe537c273, 0x6bb8c590, 0xa712c50e, 0xadd7cc17, 0x617dcc89, 0xeff2cb6a, 0x2358cbf4,
0xfa78d958, 0x36d2d9c6, 0xb85dde25, 0x74f7debb, 0x7e32d7a2, 0xb298d73c, 0x3c17d0df, 0xf0bdd041,
0x5526f3c6, 0x998cf358, 0x1703f4bb, 0xdba9f425, 0xd16cfd3c, 0x1dc6fda2, 0x9349fa41, 0x5fe3fadf,
0x86c3e873, 0x4a69e8ed, 0xc4e6ef0e, 0x084cef90, 0x0289e689, 0xce23e617, 0x40ace1f4, 0x8c06e16a,
0xd0eba0bb, 0x1c41a025, 0x92cea7c6, 0x5e64a758, 0x54a1ae41, 0x980baedf, 0x1684a93c, 0xda2ea9a2,
0x030ebb0e, 0xcfa4bb90, 0x412bbc73, 0x8d81bced, 0x8744b5f4, 0x4beeb56a, 0xc561b289, 0x09cbb217,
0xac509190, 0x60fa910e, 0xee7596ed, 0x22df9673, 0x281a9f6a, 0xe4b09ff4, 0x6a3f9817, 0xa6959889,
0x7fb58a25, 0xb31f8abb, 0x3d908d58, 0xf13a8dc6, 0xfbff84df, 0x37558441, 0xb9da83a2, 0x7570833c,
0x533b85da, 0x9f918544, 0x111e82a7, 0xddb48239, 0xd7718b20, 0x1bdb8bbe, 0x95548c5d, 0x59fe8cc3,
0x80de9e6f, 0x4c749ef1, 0xc2fb9912, 0x0e51998c, 0x04949095, 0xc83e900b, 0x46b197e8, 0x8a1b9776,
0x2f80b4f1, 0xe32ab46f, 0x6da5b38c, 0xa10fb312, 0xabcaba0b, 0x6760ba95, 0xe9efbd76, 0x2545bde8,
0xfc65af44, 0x30cfafda, 0xbe40a839, 0x72eaa8a7, 0x782fa1be, 0xb485a120, 0x3a0aa6c3, 0xf6a0a65d,
0xaa4de78c, 0x66e7e712, 0xe868e0f1, 0x24c2e06f, 0x2e07e976, 0xe2ade9e8, 0x6c22ee0b, 0xa088ee95,
0x79a8fc39, 0xb502fca7, 0x3b8dfb44, 0xf727fbda, 0xfde2f2c3, 0x3148f25d, 0xbfc7f5be, 0x736df520,
0xd6f6d6a7, 0x1a5cd639, 0x94d3d1da, 0x5879d144, 0x52bcd85d, 0x9e16d8c3, 0x1099df20, 0xdc33dfbe,
0x0513cd12, 0xc9b9cd8c, 0x4736ca6f, 0x8b9ccaf1, 0x8159c3e8, 0x4df3c376, 0xc37cc495, 0x0fd6c40b,
0x7aa64737, 0xb60c47a9, 0x3883404a, 0xf42940d4, 0xfeec49cd, 0x32464953, 0xbcc94eb0, 0x70634e2e,
0xa9435c82, 0x65e95c1c, 0xeb665bff, 0x27cc5b61, 0x2d095278, 0xe1a352e6, 0x6f2c5505, 0xa386559b,
0x061d761c, 0xcab77682, 0x44387161, 0x889271ff, 0x825778e6, 0x4efd7878, 0xc0727f9b, 0x0cd87f05,
0xd5f86da9, 0x19526d37, 0x97dd6ad4, 0x5b776a4a, 0x51b26353, 0x9d1863cd, 0x1397642e, 0xdf3d64b0,
0x83d02561, 0x4f7a25ff, 0xc1f5221c, 0x0d5f2282, 0x079a2b9b, 0xcb302b05, 0x45bf2ce6, 0x89152c78,
0x50353ed4, 0x9c9f3e4a, 0x121039a9, 0xdeba3937, 0xd47f302e, 0x18d530b0, 0x965a3753, 0x5af037cd,
0xff6b144a, 0x33c114d4, 0xbd4e1337, 0x71e413a9, 0x7b211ab0, 0xb78b1a2e, 0x39041dcd, 0xf5ae1d53,
0x2c8e0fff, 0xe0240f61, 0x6eab0882, 0xa201081c, 0xa8c40105, 0x646e019b, 0xeae10678, 0x264b06e6
];
zune-inflate-0.2.54/src/crc.rs 0000644 0000000 0000000 00000002470 10461020230 0014224 0 ustar 0000000 0000000 #![cfg(feature = "gzip")]
use crate::crc::crc_tables::{CRC32_SLICE1_TABLE, CRC32_SLICE8_TABLE};
mod crc_tables;
/// Calculate crc for a data and an initial crc value
#[allow(clippy::identity_op, clippy::zero_prefixed_literal)]
pub fn crc32(data: &[u8], mut crc: u32) -> u32
{
// main loop
for chunk in data.chunks_exact(8)
{
let chunk_loaded = u64::from_le_bytes(chunk.try_into().unwrap());
let v1 = (chunk_loaded & u64::from(u32::MAX)) as u32;
let v2 = (chunk_loaded >> 32) as u32;
crc = CRC32_SLICE8_TABLE[0x700 + (((crc ^ v1) >> 00) & 0xFF) as usize]
^ CRC32_SLICE8_TABLE[0x600 + (((crc ^ v1) >> 08) & 0xFF) as usize]
^ CRC32_SLICE8_TABLE[0x500 + (((crc ^ v1) >> 16) & 0xFF) as usize]
^ CRC32_SLICE8_TABLE[0x400 + (((crc ^ v1) >> 24) & 0xFF) as usize]
^ CRC32_SLICE8_TABLE[0x300 + (((v2 >> 00) & 0xFF) as usize)]
^ CRC32_SLICE8_TABLE[0x200 + (((v2 >> 08) & 0xFF) as usize)]
^ CRC32_SLICE8_TABLE[0x100 + (((v2 >> 16) & 0xFF) as usize)]
^ CRC32_SLICE8_TABLE[0x000 + (((v2 >> 24) & 0xFF) as usize)];
}
// handle remainder
for remainder in data.chunks_exact(8).remainder()
{
crc = (crc >> 8) ^ CRC32_SLICE1_TABLE[((crc & 0xFF) ^ u32::from(*remainder)) as usize];
}
crc
}
zune-inflate-0.2.54/src/decoder.rs 0000644 0000000 0000000 00000200467 10461020230 0015070 0 ustar 0000000 0000000 #![allow(unused_imports)]
use alloc::vec::Vec;
use alloc::{format, vec};
use crate::bitstream::BitStreamReader;
use crate::constants::{
DEFLATE_BLOCKTYPE_DYNAMIC_HUFFMAN, DEFLATE_BLOCKTYPE_RESERVED, DEFLATE_BLOCKTYPE_STATIC,
DEFLATE_BLOCKTYPE_UNCOMPRESSED, DEFLATE_MAX_CODEWORD_LENGTH,
DEFLATE_MAX_LITLEN_CODEWORD_LENGTH, DEFLATE_MAX_NUM_SYMS, DEFLATE_MAX_OFFSET_CODEWORD_LENGTH,
DEFLATE_MAX_PRE_CODEWORD_LEN, DEFLATE_NUM_LITLEN_SYMS, DEFLATE_NUM_OFFSET_SYMS,
DEFLATE_NUM_PRECODE_SYMS, DEFLATE_PRECODE_LENS_PERMUTATION, DELFATE_MAX_LENS_OVERRUN,
FASTCOPY_BYTES, FASTLOOP_MAX_BYTES_WRITTEN, HUFFDEC_END_OF_BLOCK, HUFFDEC_EXCEPTIONAL,
HUFFDEC_LITERAL, HUFFDEC_SUITABLE_POINTER, LITLEN_DECODE_BITS, LITLEN_DECODE_RESULTS,
LITLEN_ENOUGH, LITLEN_TABLE_BITS, OFFSET_DECODE_RESULTS, OFFSET_ENOUGH, OFFSET_TABLEBITS,
PRECODE_DECODE_RESULTS, PRECODE_ENOUGH, PRECODE_TABLE_BITS
};
use crate::errors::{DecodeErrorStatus, InflateDecodeErrors};
#[cfg(feature = "gzip")]
use crate::gzip_constants::{
GZIP_CM_DEFLATE, GZIP_FCOMMENT, GZIP_FEXTRA, GZIP_FHCRC, GZIP_FNAME, GZIP_FOOTER_SIZE,
GZIP_FRESERVED, GZIP_ID1, GZIP_ID2
};
use crate::utils::{copy_rep_matches, fixed_copy_within, make_decode_table_entry};
struct DeflateHeaderTables
{
litlen_decode_table: [u32; LITLEN_ENOUGH],
offset_decode_table: [u32; OFFSET_ENOUGH]
}
impl Default for DeflateHeaderTables
{
fn default() -> Self
{
DeflateHeaderTables {
litlen_decode_table: [0; LITLEN_ENOUGH],
offset_decode_table: [0; OFFSET_ENOUGH]
}
}
}
/// Options that can influence decompression
/// in Deflate/Zlib/Gzip
///
/// To use them, pass a customized options to
/// the deflate decoder.
#[derive(Copy, Clone)]
pub struct DeflateOptions
{
limit: usize,
confirm_checksum: bool,
size_hint: usize
}
impl Default for DeflateOptions
{
fn default() -> Self
{
DeflateOptions {
limit: 1 << 30,
confirm_checksum: true,
size_hint: 37000
}
}
}
impl DeflateOptions
{
/// Get deflate/zlib limit option
///
/// The decoder won't extend the inbuilt limit and will
/// return an error if the limit is exceeded
///
/// # Returns
/// The currently set limit of the instance
/// # Note
/// This is provided as a best effort, correctly quiting
/// is detrimental to speed and hence this should not be relied too much.
pub const fn get_limit(&self) -> usize
{
self.limit
}
/// Set a limit to the internal vector
/// used to store decoded zlib/deflate output.
///
/// # Arguments
/// limit: The new decompressor limit
/// # Returns
/// A modified version of DeflateDecoder
///
/// # Note
/// This is provided as a best effort, correctly quiting
/// is detrimental to speed and hence this should not be relied too much
#[must_use]
pub fn set_limit(mut self, limit: usize) -> Self
{
self.limit = limit;
self
}
/// Get whether the decoder will confirm a checksum
/// after decoding
pub const fn get_confirm_checksum(&self) -> bool
{
self.confirm_checksum
}
/// Set whether the decoder should confirm a checksum
/// after decoding
///
/// Note, you should definitely confirm your checksum, use
/// this with caution, otherwise data returned may be corrupt
///
/// # Arguments
/// - yes: When true, the decoder will confirm checksum
/// when false, the decoder will skip checksum verification
/// # Notes
/// This does not have an influence for deflate decoding as
/// it does not have a checksum
pub fn set_confirm_checksum(mut self, yes: bool) -> Self
{
self.confirm_checksum = yes;
self
}
/// Get the default set size hint for the decompressor
///
/// The decompressor initializes the internal storage for decompressed bytes
/// with this size and will reallocate the vec if the decompressed size becomes bigger
/// than this, but when the user currently knows how big the output will be, can be used
/// to prevent unnecessary re-allocations
pub const fn get_size_hint(&self) -> usize
{
self.size_hint
}
/// Set the size hint for the decompressor
///
/// This can be used to prevent multiple re-allocations
#[must_use]
pub const fn set_size_hint(mut self, hint: usize) -> Self
{
self.size_hint = hint;
self
}
}
/// A deflate decoder instance.
///
/// The decoder manages output buffer as opposed to requiring the caller to provide a pre-allocated buffer
/// it tracks number of bytes written and on successfully reaching the
/// end of the block, will return a vector with exactly
/// the number of decompressed bytes.
///
/// This means that it may use up huge amounts of memory if not checked, but
/// there are [options] that can prevent that
///
/// [options]: DeflateOptions
pub struct DeflateDecoder<'a>
{
data: &'a [u8],
position: usize,
stream: BitStreamReader<'a>,
is_last_block: bool,
static_codes_loaded: bool,
deflate_header_tables: DeflateHeaderTables,
options: DeflateOptions
}
impl<'a> DeflateDecoder<'a>
{
/// Create a new decompressor that will read compressed
/// data from `data` and return a new vector containing new data
///
/// # Arguments
/// - `data`: The compressed data. Data can be of any type
/// gzip,zlib or raw deflate.
///
/// # Returns
/// A decoder instance which will pull compressed data from `data` to inflate the output output
///
/// # Note
///
/// The default output size limit is **1 GiB.**
/// this is to protect the end user against ddos attacks as deflate does not specify it's
/// output size upfront
///
/// The checksum will be verified depending on the called function.
/// this only works for zlib and gzip since deflate does not have a checksum
///
/// These defaults can be overridden via [new_with_options()](Self::new_with_options).
pub fn new(data: &'a [u8]) -> DeflateDecoder<'a>
{
let options = DeflateOptions::default();
Self::new_with_options(data, options)
}
/// Create new decoder with specified options
///
/// This can be used to fine tune the decoder to the user's
/// needs.
///
///
/// # Arguments
/// - `data`: The compressed data. Data can be of any format i.e
/// gzip, zlib or raw deflate.
/// - `options` : A set of user defined options which tune how the decompressor
///
/// # Returns
/// A decoder instance which will pull compressed data from `data` to inflate output
///
/// # Example
/// ```no_run
/// use zune_inflate::{DeflateDecoder, DeflateOptions};
/// let data = [37];
/// let options = DeflateOptions::default()
/// .set_confirm_checksum(true) // confirm the checksum for zlib and gzip
/// .set_limit(1000); // how big I think the input will be
/// let mut decoder = DeflateDecoder::new_with_options(&data,options);
/// // do some stuff and then call decode
/// let data = decoder.decode_zlib();
///
/// ```
pub fn new_with_options(data: &'a [u8], options: DeflateOptions) -> DeflateDecoder<'a>
{
// create stream
DeflateDecoder {
data,
position: 0,
stream: BitStreamReader::new(data),
is_last_block: false,
static_codes_loaded: false,
deflate_header_tables: DeflateHeaderTables::default(),
options
}
}
/// Decode zlib-encoded data returning the uncompressed in a `Vec`
/// or an error if something went wrong.
///
/// Bytes consumed will be from the data passed when the
/// `new` method was called.
///
/// # Arguments
/// - None
/// # Returns
/// Result type containing the decoded data.
///
/// - `Ok(Vec)`: Decoded vector containing the uncompressed bytes
/// - `Err(InflateDecodeErrors)`: Error that occurred during decoding
///
/// It's possible to recover bytes even after an error occurred, bytes up
/// to when error was encountered are stored in [InflateDecodeErrors]
///
///
/// # Note
/// This needs the `zlib` feature enabled to be available otherwise it's a
/// compile time error
///
/// [InflateDecodeErrors]:crate::errors::InflateDecodeErrors
///
#[cfg(feature = "zlib")]
pub fn decode_zlib(&mut self) -> Result, InflateDecodeErrors>
{
use crate::utils::calc_adler_hash;
if self.data.len()
< 2 /* zlib header */
+ 4
/* Deflate */
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::InsufficientData
));
}
// Zlib flags
// See https://www.ietf.org/rfc/rfc1950.txt for
// the RFC
let cmf = self.data[0];
let flg = self.data[1];
let cm = cmf & 0xF;
let cinfo = cmf >> 4;
// let fcheck = flg & 0xF;
// let fdict = (flg >> 4) & 1;
// let flevel = flg >> 5;
// confirm we have the right deflate methods
if cm != 8
{
if cm == 15
{
return Err(InflateDecodeErrors::new_with_error(DecodeErrorStatus::Generic(
"CM of 15 is preserved by the standard,currently don't know how to handle it"
)));
}
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::GenericStr(format!("Unknown zlib compression method {cm}"))
));
}
if cinfo > 7
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::GenericStr(format!(
"Unknown cinfo `{cinfo}` greater than 7, not allowed"
))
));
}
let flag_checks = (u16::from(cmf) * 256) + u16::from(flg);
if flag_checks % 31 != 0
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::Generic("FCHECK integrity not preserved")
));
}
self.position = 2;
let data = self.decode_deflate()?;
if self.options.confirm_checksum
{
// Get number of consumed bytes from the input
let out_pos = self.stream.get_position() + self.position + self.stream.over_read;
// read adler
if let Some(adler) = self.data.get(out_pos..out_pos + 4)
{
let adler_bits: [u8; 4] = adler.try_into().unwrap();
let adler32_expected = u32::from_be_bytes(adler_bits);
let adler32_found = calc_adler_hash(&data);
if adler32_expected != adler32_found
{
let err_msg =
DecodeErrorStatus::MismatchedAdler(adler32_expected, adler32_found);
let err = InflateDecodeErrors::new(err_msg, data);
return Err(err);
}
}
else
{
let err = InflateDecodeErrors::new(DecodeErrorStatus::InsufficientData, data);
return Err(err);
}
}
Ok(data)
}
/// Decode a gzip encoded data and return the uncompressed data in a
/// `Vec` or an error if something went wrong
///
/// Bytes consumed will be from the data passed when the
/// `new` method was called.
///
/// # Arguments
/// - None
/// # Returns
/// Result type containing the decoded data.
///
/// - `Ok(Vec)`: Decoded vector containing the uncompressed bytes
/// - `Err(InflateDecodeErrors)`: Error that occurred during decoding
///
/// It's possible to recover bytes even after an error occurred, bytes up
/// to when error was encountered are stored in [InflateDecodeErrors]
///
/// # Note
/// This needs the `gzip` feature enabled to be available, otherwise it's a
/// compile time error
///
/// [InflateDecodeErrors]:crate::errors::InflateDecodeErrors
///
#[cfg(feature = "gzip")]
pub fn decode_gzip(&mut self) -> Result, InflateDecodeErrors>
{
if self.data.len() < 18
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::InsufficientData
));
}
if self.data[self.position] != GZIP_ID1
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::CorruptData
));
}
self.position += 1;
if self.data[self.position] != GZIP_ID2
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::CorruptData
));
}
self.position += 1;
if self.data[self.position] != GZIP_CM_DEFLATE
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::CorruptData
));
}
self.position += 1;
let flg = self.data[self.position];
self.position += 1;
// skip mtime
self.position += 4;
// skip xfl
self.position += 1;
// skip os
self.position += 1;
if (flg & GZIP_FRESERVED) != 0
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::CorruptData
));
}
// extra field
if (flg & GZIP_FEXTRA) != 0
{
let len_bytes = self.data[self.position..self.position + 2]
.try_into()
.unwrap();
let xlen = usize::from(u16::from_le_bytes(len_bytes));
self.position += 2;
if self.data.len().saturating_sub(self.position) < xlen + GZIP_FOOTER_SIZE
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::CorruptData
));
}
self.position += xlen;
}
// original file name zero terminated
if (flg & GZIP_FNAME) != 0
{
loop
{
if let Some(byte) = self.data.get(self.position)
{
self.position += 1;
if *byte == 0
{
break;
}
}
else
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::InsufficientData
));
}
}
}
// File comment zero terminated
if (flg & GZIP_FCOMMENT) != 0
{
loop
{
if let Some(byte) = self.data.get(self.position)
{
self.position += 1;
if *byte == 0
{
break;
}
}
else
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::InsufficientData
));
}
}
}
// crc16 for gzip header
if (flg & GZIP_FHCRC) != 0
{
self.position += 2;
}
if self.position + GZIP_FOOTER_SIZE > self.data.len()
{
return Err(InflateDecodeErrors::new_with_error(
DecodeErrorStatus::InsufficientData
));
}
let data = self.decode_deflate()?;
let mut out_pos = self.stream.get_position() + self.position + self.stream.over_read;
if self.options.confirm_checksum
{
// Get number of consumed bytes from the input
if let Some(crc) = self.data.get(out_pos..out_pos + 4)
{
let crc_bits: [u8; 4] = crc.try_into().unwrap();
let crc32_expected = u32::from_le_bytes(crc_bits);
let crc32_found = !crate::crc::crc32(&data, !0);
if crc32_expected != crc32_found
{
let err_msg = DecodeErrorStatus::MismatchedCRC(crc32_expected, crc32_found);
let err = InflateDecodeErrors::new(err_msg, data);
return Err(err);
}
}
else
{
let err = InflateDecodeErrors::new(DecodeErrorStatus::InsufficientData, data);
return Err(err);
}
}
//checksum
out_pos += 4;
if let Some(val) = self.data.get(out_pos..out_pos + 4)
{
let actual_bytes: [u8; 4] = val.try_into().unwrap();
let ac = u32::from_le_bytes(actual_bytes) as usize;
if data.len() != ac
{
let err = DecodeErrorStatus::Generic("ISIZE does not match actual bytes");
let err = InflateDecodeErrors::new(err, data);
return Err(err);
}
}
else
{
let err = InflateDecodeErrors::new(DecodeErrorStatus::InsufficientData, data);
return Err(err);
}
Ok(data)
}
/// Decode a deflate stream returning the data as `Vec` or an error
/// indicating what went wrong.
/// # Arguments
/// - None
/// # Returns
/// Result type containing the decoded data.
///
/// - `Ok(Vec)`: Decoded vector containing the uncompressed bytes
/// - `Err(InflateDecodeErrors)`: Error that occurred during decoding
///
/// It's possible to recover bytes even after an error occurred, bytes up
/// to when error was encountered are stored in [InflateDecodeErrors]
///
///
/// # Example
/// ```no_run
/// let data = [42]; // answer to life, the universe and everything
///
/// let mut decoder = zune_inflate::DeflateDecoder::new(&data);
/// let bytes = decoder.decode_deflate().unwrap();
/// ```
///
/// [InflateDecodeErrors]:crate::errors::InflateDecodeErrors
pub fn decode_deflate(&mut self) -> Result, InflateDecodeErrors>
{
self.start_deflate_block()
}
/// Main inner loop for decompressing deflate data
#[allow(unused_assignments)]
fn start_deflate_block(&mut self) -> Result, InflateDecodeErrors>
{
// start deflate decode
// re-read the stream so that we can remove code read by zlib
self.stream = BitStreamReader::new(&self.data[self.position..]);
self.stream.refill();
// Output space for our decoded bytes.
let mut out_block = vec![0; self.options.size_hint];
// bits used
let mut src_offset = 0;
let mut dest_offset = 0;
loop
{
self.stream.refill();
self.is_last_block = self.stream.get_bits(1) == 1;
let block_type = self.stream.get_bits(2);
if block_type == DEFLATE_BLOCKTYPE_UNCOMPRESSED
{
/*
* Uncompressed block: copy 'len' bytes literally from the input
* buffer to the output buffer.
*/
/*
* The RFC says that
* skip any remaining bits in current partially
* processed byte
* read LEN and NLEN (see next section)
* copy LEN bytes of data to output
*/
if self.stream.over_read > usize::from(self.stream.get_bits_left() >> 3)
{
out_block.truncate(dest_offset);
let err_msg = DecodeErrorStatus::Generic("over-read stream");
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
let partial_bits = self.stream.get_bits_left() & 7;
self.stream.drop_bits(partial_bits);
let len = self.stream.get_bits(16) as u16;
let nlen = self.stream.get_bits(16) as u16;
// copy to deflate
if len != !nlen
{
out_block.truncate(dest_offset);
let err_msg = DecodeErrorStatus::Generic("Len and nlen do not match");
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
let len = len as usize;
let start = self.stream.get_position() + self.position + self.stream.over_read;
// ensure there is enough space for a fast copy
if dest_offset + len + FASTCOPY_BYTES > out_block.len()
{
// and if there is not, resize
let new_len = out_block.len() + RESIZE_BY + len;
out_block.resize(new_len, 0);
}
if self.data.get((start + len).saturating_sub(1)).is_none()
{
out_block.truncate(dest_offset);
let err_msg = DecodeErrorStatus::CorruptData;
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
if dest_offset > self.options.limit
{
out_block.truncate(dest_offset);
let err_msg =
DecodeErrorStatus::OutputLimitExceeded(self.options.limit, out_block.len());
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
out_block[dest_offset..dest_offset + len]
.copy_from_slice(&self.data[start..start + len]);
dest_offset += len;
// get the new position to write.
self.stream.position =
len + (self.stream.position - usize::from(self.stream.bits_left >> 3));
self.stream.reset();
if self.is_last_block
{
break;
}
continue;
}
else if block_type == DEFLATE_BLOCKTYPE_RESERVED
{
out_block.truncate(dest_offset);
let err_msg = DecodeErrorStatus::Generic("Reserved block type 0b11 encountered");
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
// build decode tables for static and dynamic tables
match self.build_decode_table(block_type)
{
Ok(_) => (),
Err(value) =>
{
out_block.truncate(dest_offset);
let err_msg = value;
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
};
// Tables are mutated into the struct, so at this point we know the tables
// are loaded, take a reference to them
let litlen_decode_table = &self.deflate_header_tables.litlen_decode_table;
let offset_decode_table = &self.deflate_header_tables.offset_decode_table;
/*
* This is the "fast loop" for decoding literals and matches. It does
* bounds checks on in_next and out_next in the loop conditions so that
* additional bounds checks aren't needed inside the loop body.
*
* To reduce latency, the bit-buffer is refilled and the next litlen
* decode table entry is preloaded before each loop iteration.
*/
let (mut literal, mut length, mut offset, mut entry) = (0, 0, 0, 0);
let mut saved_bitbuf;
'decode: loop
{
let close_src = 3 * FASTCOPY_BYTES < self.stream.remaining_bytes();
if close_src
{
self.stream.refill_inner_loop();
let lit_mask = self.stream.peek_bits::();
entry = litlen_decode_table[lit_mask];
'sequence: loop
{
// Resize the output vector here to ensure we can always have
// enough space for sloppy copies
if dest_offset + FASTLOOP_MAX_BYTES_WRITTEN > out_block.len()
{
let curr_len = out_block.len();
out_block.resize(curr_len + FASTLOOP_MAX_BYTES_WRITTEN + RESIZE_BY, 0)
}
// At this point entry contains the next value of the litlen
// This will always be the case so meaning all our exit paths need
// to load in the next entry.
// recheck after every sequence
// when we hit continue, we need to recheck this
// as we are trying to emulate a do while
let new_check = self.stream.src.len() < self.stream.position + 8;
if new_check
{
break 'sequence;
}
self.stream.refill_inner_loop();
/*
* Consume the bits for the litlen decode table entry. Save the
* original bit-buf for later, in case the extra match length
* bits need to be extracted from it.
*/
saved_bitbuf = self.stream.buffer;
self.stream.drop_bits((entry & 0xFF) as u8);
/*
* Begin by checking for a "fast" literal, i.e. a literal that
* doesn't need a subtable.
*/
if (entry & HUFFDEC_LITERAL) != 0
{
/*
* On 64-bit platforms, we decode up to 2 extra fast
* literals in addition to the primary item, as this
* increases performance and still leaves enough bits
* remaining for what follows. We could actually do 3,
* assuming LITLEN_TABLEBITS=11, but that actually
* decreases performance slightly (perhaps by messing
* with the branch prediction of the conditional refill
* that happens later while decoding the match offset).
*/
literal = entry >> 16;
let new_pos = self.stream.peek_bits::();
entry = litlen_decode_table[new_pos];
saved_bitbuf = self.stream.buffer;
self.stream.drop_bits(entry as u8);
let out: &mut [u8; 2] = out_block
.get_mut(dest_offset..dest_offset + 2)
.unwrap()
.try_into()
.unwrap();
out[0] = literal as u8;
dest_offset += 1;
if (entry & HUFFDEC_LITERAL) != 0
{
/*
* Another fast literal, but this one is in lieu of the
* primary item, so it doesn't count as one of the extras.
*/
// load in the next entry.
literal = entry >> 16;
let new_pos = self.stream.peek_bits::();
entry = litlen_decode_table[new_pos];
out[1] = literal as u8;
dest_offset += 1;
continue;
}
}
/*
* It's not a literal entry, so it can be a length entry, a
* subtable pointer entry, or an end-of-block entry. Detect the
* two unlikely cases by testing the HUFFDEC_EXCEPTIONAL flag.
*/
if (entry & HUFFDEC_EXCEPTIONAL) != 0
{
// Subtable pointer or end of block entry
if (entry & HUFFDEC_END_OF_BLOCK) != 0
{
// block done
break 'decode;
}
/*
* A subtable is required. Load and consume the
* subtable entry. The subtable entry can be of any
* type: literal, length, or end-of-block.
*/
let entry_position = ((entry >> 8) & 0x3F) as usize;
let mut pos = (entry >> 16) as usize;
saved_bitbuf = self.stream.buffer;
pos += self.stream.peek_var_bits(entry_position);
entry = litlen_decode_table[pos.min(LITLEN_ENOUGH - 1)];
self.stream.drop_bits(entry as u8);
if (entry & HUFFDEC_LITERAL) != 0
{
// decode a literal that required a sub table
let new_pos = self.stream.peek_bits::();
literal = entry >> 16;
entry = litlen_decode_table[new_pos];
*out_block.get_mut(dest_offset).unwrap_or(&mut 0) =
(literal & 0xFF) as u8;
dest_offset += 1;
continue;
}
if (entry & HUFFDEC_END_OF_BLOCK) != 0
{
break 'decode;
}
}
// At this point,we dropped at most 22 bits(LITLEN_DECODE is 11 and we
// can do it twice), we now just have 34 bits min remaining.
/*
* Decode the match length: the length base value associated
* with the litlen symbol (which we extract from the decode
* table entry), plus the extra length bits. We don't need to
* consume the extra length bits here, as they were included in
* the bits consumed by the entry earlier. We also don't need
* to check for too-long matches here, as this is inside the
* fast loop where it's already been verified that the output
* buffer has enough space remaining to copy a max-length match.
*/
let entry_dup = entry;
entry = offset_decode_table[self.stream.peek_bits::()];
length = (entry_dup >> 16) as usize;
let mask = (1 << entry_dup as u8) - 1;
length += (saved_bitbuf & mask) as usize >> ((entry_dup >> 8) as u8);
// offset requires a subtable
if (entry & HUFFDEC_EXCEPTIONAL) != 0
{
self.stream.drop_bits(OFFSET_TABLEBITS as u8);
let extra = self.stream.peek_var_bits(((entry >> 8) & 0x3F) as usize);
entry = offset_decode_table[((entry >> 16) as usize + extra) & 511];
// refill to handle some weird edge case where we have
// less bits than needed for reading the lit-len
}
saved_bitbuf = self.stream.buffer;
self.stream.drop_bits((entry & 0xFF) as u8);
let mask = (1 << entry as u8) - 1;
offset = (entry >> 16) as usize;
offset += (saved_bitbuf & mask) as usize >> (((entry >> 8) & 0xFF) as u8);
if offset > dest_offset
{
out_block.truncate(dest_offset);
let err_msg = DecodeErrorStatus::CorruptData;
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
src_offset = dest_offset - offset;
if self.stream.bits_left < 11
{
self.stream.refill_inner_loop();
}
// Copy some bytes unconditionally
// This makes us copy smaller match lengths quicker because we don't need
// a loop + don't send too much pressure to the Memory unit.
fixed_copy_within::(
&mut out_block,
src_offset,
dest_offset
);
entry = litlen_decode_table[self.stream.peek_bits::()];
let mut current_position = dest_offset;
dest_offset += length;
if offset == 1
{
// RLE fill with a single byte
let byte_to_repeat = out_block[src_offset];
out_block[current_position..dest_offset].fill(byte_to_repeat);
}
else if offset <= FASTCOPY_BYTES
&& current_position + offset < dest_offset
{
// The second conditional ensures we only come
// here if the first copy didn't succeed to copy just enough bytes for a rep
// match to be valid, i.e we want this path to be taken the least amount
// of times possible
// the unconditional copy above copied some bytes
// don't let it go into waste
// Increment the position we are in by the number of correct bytes
// currently copied
let mut src_position = src_offset + offset;
let mut dest_position = current_position + offset;
// loop copying offset bytes in place
// notice this loop does fixed copies but increments in offset bytes :)
// that is intentional.
loop
{
fixed_copy_within::(
&mut out_block,
src_position,
dest_position
);
src_position += offset;
dest_position += offset;
if dest_position > dest_offset
{
break;
}
}
}
else if length > FASTCOPY_BYTES
{
current_position += FASTCOPY_BYTES;
// fast non-overlapping copy
//
// We have enough space to write the ML+FAST_COPY bytes ahead
// so we know this won't come to shoot us in the foot.
//
// An optimization is to copy FAST_COPY_BITS per invocation
// Currently FASTCOPY_BYTES is 16, this fits in nicely as we
// it's a single SIMD instruction on a lot of things, i.e x86,Arm and even
// wasm.
// current position of the match
let mut dest_src_offset = src_offset + FASTCOPY_BYTES;
// Number of bytes we are to copy
// copy in batches of FAST_BYTES
'match_lengths: loop
{
// Safety: We resized out_block hence we know it can handle
// sloppy copies without it being out of bounds
//
// Reason: This is a latency critical loop, even branches start
// to matter
fixed_copy_within::(
&mut out_block,
dest_src_offset,
current_position
);
dest_src_offset += FASTCOPY_BYTES;
current_position += FASTCOPY_BYTES;
if current_position > dest_offset
{
break 'match_lengths;
}
}
}
if dest_offset > self.options.limit
{
out_block.truncate(dest_offset);
let err_msg = DecodeErrorStatus::OutputLimitExceeded(
self.options.limit,
dest_offset
);
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
if self.stream.src.len() < self.stream.position + 8
{
// close to input end, move to the slower one
break 'sequence;
}
}
}
// generic loop that does things a bit slower but it's okay since it doesn't
// deal with a lot of things
// We can afford to be more careful here, checking that we do
// not drop non-existent bits etc etc as we do not have the
// assurances of the fast loop bits above.
loop
{
self.stream.refill();
if self.stream.over_read > usize::from(self.stream.bits_left >> 3)
{
out_block.truncate(dest_offset);
let err_msg = DecodeErrorStatus::CorruptData;
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
let literal_mask = self.stream.peek_bits::();
entry = litlen_decode_table[literal_mask];
saved_bitbuf = self.stream.buffer;
self.stream.drop_bits((entry & 0xFF) as u8);
if (entry & HUFFDEC_SUITABLE_POINTER) != 0
{
let extra = self.stream.peek_var_bits(((entry >> 8) & 0x3F) as usize);
entry = litlen_decode_table[(entry >> 16) as usize + extra];
saved_bitbuf = self.stream.buffer;
self.stream.drop_bits((entry & 0xFF) as u8);
}
length = (entry >> 16) as usize;
if (entry & HUFFDEC_LITERAL) != 0
{
resize_and_push(&mut out_block, dest_offset, length as u8);
dest_offset += 1;
continue;
}
if (entry & HUFFDEC_END_OF_BLOCK) != 0
{
break 'decode;
}
let mask = (1 << entry as u8) - 1;
length += (saved_bitbuf & mask) as usize >> ((entry >> 8) as u8);
self.stream.refill();
entry = offset_decode_table[self.stream.peek_bits::()];
if (entry & HUFFDEC_EXCEPTIONAL) != 0
{
// offset requires a subtable
self.stream.drop_bits(OFFSET_TABLEBITS as u8);
let extra = self.stream.peek_var_bits(((entry >> 8) & 0x3F) as usize);
entry = offset_decode_table[((entry >> 16) as usize + extra) & 511];
}
// ensure there is enough space for a fast copy
if dest_offset + length + FASTCOPY_BYTES > out_block.len()
{
let new_len = out_block.len() + RESIZE_BY + length;
out_block.resize(new_len, 0);
}
saved_bitbuf = self.stream.buffer;
let mask = (1 << (entry & 0xFF) as u8) - 1;
offset = (entry >> 16) as usize;
offset += (saved_bitbuf & mask) as usize >> ((entry >> 8) as u8);
if offset > dest_offset
{
out_block.truncate(dest_offset);
let err_msg = DecodeErrorStatus::CorruptData;
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
src_offset = dest_offset - offset;
self.stream.drop_bits(entry as u8);
let (dest_src, dest_ptr) = out_block.split_at_mut(dest_offset);
if src_offset + length + FASTCOPY_BYTES > dest_offset
{
// overlapping copy
// do a simple rep match
copy_rep_matches(&mut out_block, src_offset, dest_offset, length);
}
else
{
dest_ptr[0..length]
.copy_from_slice(&dest_src[src_offset..src_offset + length]);
}
dest_offset += length;
if dest_offset > self.options.limit
{
out_block.truncate(dest_offset);
let err_msg =
DecodeErrorStatus::OutputLimitExceeded(self.options.limit, dest_offset);
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
}
}
/*
* If any of the implicit appended zero bytes were consumed (not just
* refilled) before hitting end of stream, then the data is bad.
*/
if self.stream.over_read > usize::from(self.stream.bits_left >> 3)
{
out_block.truncate(dest_offset);
let err_msg = DecodeErrorStatus::CorruptData;
let error = InflateDecodeErrors::new(err_msg, out_block);
return Err(error);
}
if self.is_last_block
{
break;
}
}
// decompression. DONE
// Truncate data to match the number of actual
// bytes written.
out_block.truncate(dest_offset);
Ok(out_block)
}
/// Build decode tables for static and dynamic
/// huffman blocks.
fn build_decode_table(&mut self, block_type: u64) -> Result<(), DecodeErrorStatus>
{
const COUNT: usize =
DEFLATE_NUM_LITLEN_SYMS + DEFLATE_NUM_OFFSET_SYMS + DELFATE_MAX_LENS_OVERRUN;
let mut lens = [0_u8; COUNT];
let mut precode_lens = [0; DEFLATE_NUM_PRECODE_SYMS];
let mut precode_decode_table = [0_u32; PRECODE_ENOUGH];
let mut litlen_decode_table = [0_u32; LITLEN_ENOUGH];
let mut offset_decode_table = [0; OFFSET_ENOUGH];
let mut num_litlen_syms = 0;
let mut num_offset_syms = 0;
if block_type == DEFLATE_BLOCKTYPE_DYNAMIC_HUFFMAN
{
const SINGLE_PRECODE: usize = 3;
self.static_codes_loaded = false;
// Dynamic Huffman block
// Read codeword lengths
if !self.stream.has(5 + 5 + 4)
{
return Err(DecodeErrorStatus::InsufficientData);
}
num_litlen_syms = 257 + (self.stream.get_bits(5)) as usize;
num_offset_syms = 1 + (self.stream.get_bits(5)) as usize;
let num_explicit_precode_lens = 4 + (self.stream.get_bits(4)) as usize;
self.stream.refill();
if !self.stream.has(3)
{
return Err(DecodeErrorStatus::InsufficientData);
}
let first_precode = self.stream.get_bits(3) as u8;
let expected = (SINGLE_PRECODE * num_explicit_precode_lens.saturating_sub(1)) as u8;
precode_lens[usize::from(DEFLATE_PRECODE_LENS_PERMUTATION[0])] = first_precode;
self.stream.refill();
if !self.stream.has(expected)
{
return Err(DecodeErrorStatus::InsufficientData);
}
for i in DEFLATE_PRECODE_LENS_PERMUTATION[1..]
.iter()
.take(num_explicit_precode_lens - 1)
{
let bits = self.stream.get_bits(3) as u8;
precode_lens[usize::from(*i)] = bits;
}
self.build_decode_table_inner(
&precode_lens,
&PRECODE_DECODE_RESULTS,
&mut precode_decode_table,
PRECODE_TABLE_BITS,
DEFLATE_NUM_PRECODE_SYMS,
DEFLATE_MAX_CODEWORD_LENGTH
)?;
/* Decode the litlen and offset codeword lengths. */
let mut i = 0;
loop
{
if i >= num_litlen_syms + num_offset_syms
{
// confirm here since with a continue loop stuff
// breaks
break;
}
let rep_val: u8;
let rep_count: u64;
if !self.stream.has(DEFLATE_MAX_PRE_CODEWORD_LEN + 7)
{
self.stream.refill();
}
// decode next pre-code symbol
let entry_pos = self
.stream
.peek_bits::<{ DEFLATE_MAX_PRE_CODEWORD_LEN as usize }>();
let entry = precode_decode_table[entry_pos];
let presym = entry >> 16;
if !self.stream.has(entry as u8)
{
return Err(DecodeErrorStatus::InsufficientData);
}
self.stream.drop_bits(entry as u8);
if presym < 16
{
// explicit codeword length
lens[i] = presym as u8;
i += 1;
continue;
}
/* Run-length encoded codeword lengths */
/*
* Note: we don't need verify that the repeat count
* doesn't overflow the number of elements, since we've
* sized the lens array to have enough extra space to
* allow for the worst-case overrun (138 zeroes when
* only 1 length was remaining).
*
* In the case of the small repeat counts (presyms 16
* and 17), it is fastest to always write the maximum
* number of entries. That gets rid of branches that
* would otherwise be required.
*
* It is not just because of the numerical order that
* our checks go in the order 'presym < 16', 'presym ==
* 16', and 'presym == 17'. For typical data this is
* ordered from most frequent to least frequent case.
*/
if presym == 16
{
if i == 0
{
return Err(DecodeErrorStatus::CorruptData);
}
if !self.stream.has(2)
{
return Err(DecodeErrorStatus::InsufficientData);
}
// repeat previous length three to 6 times
rep_val = lens[i - 1];
rep_count = 3 + self.stream.get_bits(2);
lens[i..i + 6].fill(rep_val);
i += rep_count as usize;
}
else if presym == 17
{
if !self.stream.has(3)
{
return Err(DecodeErrorStatus::InsufficientData);
}
/* Repeat zero 3 - 10 times. */
rep_count = 3 + self.stream.get_bits(3);
lens[i..i + 10].fill(0);
i += rep_count as usize;
}
else
{
if !self.stream.has(7)
{
return Err(DecodeErrorStatus::InsufficientData);
}
// repeat zero 11-138 times.
rep_count = 11 + self.stream.get_bits(7);
lens[i..i + rep_count as usize].fill(0);
i += rep_count as usize;
}
if i >= num_litlen_syms + num_offset_syms
{
break;
}
}
}
else if block_type == DEFLATE_BLOCKTYPE_STATIC
{
if self.static_codes_loaded
{
return Ok(());
}
self.static_codes_loaded = true;
lens[000..144].fill(8);
lens[144..256].fill(9);
lens[256..280].fill(7);
lens[280..288].fill(8);
lens[288..].fill(5);
num_litlen_syms = 288;
num_offset_syms = 32;
}
// build offset decode table
self.build_decode_table_inner(
&lens[num_litlen_syms..],
&OFFSET_DECODE_RESULTS,
&mut offset_decode_table,
OFFSET_TABLEBITS,
num_offset_syms,
DEFLATE_MAX_OFFSET_CODEWORD_LENGTH
)?;
self.build_decode_table_inner(
&lens,
&LITLEN_DECODE_RESULTS,
&mut litlen_decode_table,
LITLEN_TABLE_BITS,
num_litlen_syms,
DEFLATE_MAX_LITLEN_CODEWORD_LENGTH
)?;
self.deflate_header_tables.offset_decode_table = offset_decode_table;
self.deflate_header_tables.litlen_decode_table = litlen_decode_table;
Ok(())
}
/// Build the decode table for the precode
#[allow(clippy::needless_range_loop)]
fn build_decode_table_inner(
&mut self, lens: &[u8], decode_results: &[u32], decode_table: &mut [u32],
table_bits: usize, num_syms: usize, mut max_codeword_len: usize
) -> Result<(), DecodeErrorStatus>
{
const BITS: u32 = usize::BITS - 1;
let mut len_counts: [u32; DEFLATE_MAX_CODEWORD_LENGTH + 1] =
[0; DEFLATE_MAX_CODEWORD_LENGTH + 1];
let mut offsets: [u32; DEFLATE_MAX_CODEWORD_LENGTH + 1] =
[0; DEFLATE_MAX_CODEWORD_LENGTH + 1];
let mut sorted_syms: [u16; DEFLATE_MAX_NUM_SYMS] = [0; DEFLATE_MAX_NUM_SYMS];
let mut i;
// count how many codewords have each length, including 0.
for sym in 0..num_syms
{
len_counts[usize::from(lens[sym])] += 1;
}
/*
* Determine the actual maximum codeword length that was used, and
* decrease table_bits to it if allowed.
*/
while max_codeword_len > 1 && len_counts[max_codeword_len] == 0
{
max_codeword_len -= 1;
}
/*
* Sort the symbols primarily by increasing codeword length and
* A temporary array of length @num_syms.
* secondarily by increasing symbol value; or equivalently by their
* codewords in lexicographic order, since a canonical code is assumed.
*
* For efficiency, also compute 'codespace_used' in the same pass over
* 'len_counts[]' used to build 'offsets[]' for sorting.
*/
offsets[0] = 0;
offsets[1] = len_counts[0];
let mut codespace_used = 0_u32;
for len in 1..max_codeword_len
{
offsets[len + 1] = offsets[len] + len_counts[len];
codespace_used = (codespace_used << 1) + len_counts[len];
}
codespace_used = (codespace_used << 1) + len_counts[max_codeword_len];
for sym in 0..num_syms
{
let pos = usize::from(lens[sym]);
sorted_syms[offsets[pos] as usize] = sym as u16;
offsets[pos] += 1;
}
i = (offsets[0]) as usize;
/*
* Check whether the lengths form a complete code (exactly fills the
* codespace), an incomplete code (doesn't fill the codespace), or an
* overfull code (overflows the codespace). A codeword of length 'n'
* uses proportion '1/(2^n)' of the codespace. An overfull code is
* nonsensical, so is considered invalid. An incomplete code is
* considered valid only in two specific cases; see below.
*/
// Overfull code
if codespace_used > 1 << max_codeword_len
{
return Err(DecodeErrorStatus::Generic("Overflown code"));
}
// incomplete code
if codespace_used < 1 << max_codeword_len
{
let entry = if codespace_used == 0
{
/*
* An empty code is allowed. This can happen for the
* offset code in DEFLATE, since a dynamic Huffman block
* need not contain any matches.
*/
/* sym=0, len=1 (arbitrary) */
make_decode_table_entry(decode_results, 0, 1)
}
else
{
/*
* Allow codes with a single used symbol, with codeword
* length 1. The DEFLATE RFC is unclear regarding this
* case. What zlib's decompressor does is permit this
* for the litlen and offset codes and assume the
* codeword is '0' rather than '1'. We do the same
* except we allow this for precodes too, since there's
* no convincing reason to treat the codes differently.
* We also assign both codewords '0' and '1' to the
* symbol to avoid having to handle '1' specially.
*/
if codespace_used != 1 << (max_codeword_len - 1) || len_counts[1] != 1
{
return Err(DecodeErrorStatus::Generic(
"Cannot work with empty pre-code table"
));
}
make_decode_table_entry(decode_results, usize::from(sorted_syms[i]), 1)
};
/*
* Note: the decode table still must be fully initialized, in
* case the stream is malformed and contains bits from the part
* of the codespace the incomplete code doesn't use.
*/
decode_table.fill(entry);
return Ok(());
}
/*
* The lengths form a complete code. Now, enumerate the codewords in
* lexicographic order and fill the decode table entries for each one.
*
* First, process all codewords with len <= table_bits. Each one gets
* '2^(table_bits-len)' direct entries in the table.
*
* Since DEFLATE uses bit-reversed codewords, these entries aren't
* consecutive but rather are spaced '2^len' entries apart. This makes
* filling them naively somewhat awkward and inefficient, since strided
* stores are less cache-friendly and preclude the use of word or
* vector-at-a-time stores to fill multiple entries per instruction.
*
* To optimize this, we incrementally double the table size. When
* processing codewords with length 'len', the table is treated as
* having only '2^len' entries, so each codeword uses just one entry.
* Then, each time 'len' is incremented, the table size is doubled and
* the first half is copied to the second half. This significantly
* improves performance over naively doing strided stores.
*
* Note that some entries copied for each table doubling may not have
* been initialized yet, but it doesn't matter since they're guaranteed
* to be initialized later (because the Huffman code is complete).
*/
let mut codeword = 0;
let mut len = 1;
let mut count = len_counts[1];
while count == 0
{
len += 1;
if len >= len_counts.len()
{
break;
}
count = len_counts[len];
}
let mut curr_table_end = 1 << len;
while len <= table_bits
{
// Process all count codewords with length len
loop
{
let entry = make_decode_table_entry(
decode_results,
usize::from(sorted_syms[i]),
len as u32
);
i += 1;
// fill first entry for current codeword
decode_table[codeword] = entry;
if codeword == curr_table_end - 1
{
// last codeword (all 1's)
for _ in len..table_bits
{
decode_table.copy_within(0..curr_table_end, curr_table_end);
curr_table_end <<= 1;
}
return Ok(());
}
/*
* To advance to the lexicographically next codeword in
* the canonical code, the codeword must be incremented,
* then 0's must be appended to the codeword as needed
* to match the next codeword's length.
*
* Since the codeword is bit-reversed, appending 0's is
* a no-op. However, incrementing it is nontrivial. To
* do so efficiently, use the 'bsr' instruction to find
* the last (highest order) 0 bit in the codeword, set
* it, and clear any later (higher order) 1 bits. But
* 'bsr' actually finds the highest order 1 bit, so to
* use it first flip all bits in the codeword by XOR' ing
* it with (1U << len) - 1 == cur_table_end - 1.
*/
let adv = BITS - (codeword ^ (curr_table_end - 1)).leading_zeros();
let bit = 1 << adv;
codeword &= bit - 1;
codeword |= bit;
count -= 1;
if count == 0
{
break;
}
}
// advance to the next codeword length
loop
{
len += 1;
if len <= table_bits
{
// dest is decode_table[curr_table_end]
// source is decode_table(start of table);
// size is curr_table;
decode_table.copy_within(0..curr_table_end, curr_table_end);
//decode_table.copy_within(range, curr_table_end);
curr_table_end <<= 1;
}
count = len_counts[len];
if count != 0
{
break;
}
}
}
// process codewords with len > table_bits.
// Require sub-tables
curr_table_end = 1 << table_bits;
let mut subtable_prefix = usize::MAX;
let mut subtable_start = 0;
let mut subtable_bits;
loop
{
/*
* Start a new sub-table if the first 'table_bits' bits of the
* codeword don't match the prefix of the current subtable.
*/
if codeword & ((1_usize << table_bits) - 1) != subtable_prefix
{
subtable_prefix = codeword & ((1 << table_bits) - 1);
subtable_start = curr_table_end;
/*
* Calculate the subtable length. If the codeword has
* length 'table_bits + n', then the subtable needs
* '2^n' entries. But it may need more; if fewer than
* '2^n' codewords of length 'table_bits + n' remain,
* then the length will need to be incremented to bring
* in longer codewords until the subtable can be
* completely filled. Note that because the Huffman
* code is complete, it will always be possible to fill
* the sub-table eventually.
*/
subtable_bits = len - table_bits;
codespace_used = count;
while codespace_used < (1 << subtable_bits)
{
subtable_bits += 1;
if subtable_bits + table_bits > 15
{
return Err(DecodeErrorStatus::CorruptData);
}
codespace_used = (codespace_used << 1) + len_counts[table_bits + subtable_bits];
}
/*
* Create the entry that points from the main table to
* the subtable.
*/
decode_table[subtable_prefix] = (subtable_start as u32) << 16
| HUFFDEC_EXCEPTIONAL
| HUFFDEC_SUITABLE_POINTER
| (subtable_bits as u32) << 8
| table_bits as u32;
curr_table_end = subtable_start + (1 << subtable_bits);
}
/* Fill the sub-table entries for the current codeword. */
let stride = 1 << (len - table_bits);
let mut j = subtable_start + (codeword >> table_bits);
let entry = make_decode_table_entry(
decode_results,
sorted_syms[i] as usize,
(len - table_bits) as u32
);
i += 1;
while j < curr_table_end
{
decode_table[j] = entry;
j += stride;
}
//advance to the next codeword
if codeword == (1 << len) - 1
{
// last codeword
return Ok(());
}
let adv = BITS - (codeword ^ ((1 << len) - 1)).leading_zeros();
let bit = 1 << adv;
codeword &= bit - 1;
codeword |= bit;
count -= 1;
while count == 0
{
len += 1;
count = len_counts[len];
}
}
}
}
const RESIZE_BY: usize = 1024 * 4; // 4 kb
/// Resize vector if its current space wont
/// be able to store a new byte and then push an element to that new space
#[inline(always)]
fn resize_and_push(buf: &mut Vec, position: usize, elm: u8)
{
if buf.len() <= position
{
let new_len = buf.len() + RESIZE_BY;
buf.resize(new_len, 0);
}
buf[position] = elm;
}
zune-inflate-0.2.54/src/encoder/fast_match_finder.rs 0000644 0000000 0000000 00000000421 10461020230 0020526 0 ustar 0000000 0000000 /*
* Copyright (c) 2023.
*
* This software is free software;
*
* You can redistribute it or modify it under terms of the MIT, Apache License or Zlib license
*/
//! Fast match finder
//!
//! This implements a fast match finder that can match up to one previous match
zune-inflate-0.2.54/src/encoder.rs 0000644 0000000 0000000 00000014034 10461020230 0015073 0 ustar 0000000 0000000 /*
* Copyright (c) 2023.
*
* This software is free software; You can redistribute it or modify it under terms of the MIT, Apache License or Zlib license
*/
use alloc::vec;
use alloc::vec::Vec;
use crate::constants::DEFLATE_BLOCKTYPE_UNCOMPRESSED;
mod fast_match_finder;
const _SEQ_LENGTH_SHIFT: u32 = 23;
const _SEQ_LITRUNLEN_MASK: u32 = (1_u32 << _SEQ_LENGTH_SHIFT) - 1;
pub(crate) struct _Sequence
{
/*
* Bits 0..22: the number of literals in this run. This may be 0 and
* can be at most MAX_BLOCK_LENGTH. The literals are not stored
* explicitly in this structure; instead, they are read directly from
* the uncompressed data.
*
* Bits 23..31: the length of the match which follows the literals, or 0
* if this literal run was the last in the block, so there is no match
* which follows it.
*/
litrunlen_and_length: u32
}
#[derive(Debug, Copy, Clone)]
pub enum DeflateEncodingStrategy
{
NoCompression
}
impl DeflateEncodingStrategy
{
#[allow(dead_code)]
fn to_level(self) -> u8
{
match self
{
Self::NoCompression => 0
}
}
}
pub struct DeflateEncodingOptions
{
strategy: DeflateEncodingStrategy
}
impl Default for DeflateEncodingOptions
{
fn default() -> Self
{
DeflateEncodingOptions {
strategy: DeflateEncodingStrategy::NoCompression
}
}
}
/// A simple Deflate Encoder.
///
/// Not yet complete
pub struct DeflateEncoder<'a>
{
data: &'a [u8],
options: DeflateEncodingOptions,
output_position: usize,
input_position: usize,
output: Vec
}
impl<'a> DeflateEncoder<'a>
{
/// Create a new deflate encoder.
///
/// The
pub fn new(data: &'a [u8]) -> DeflateEncoder<'a>
{
DeflateEncoder::new_with_options(data, DeflateEncodingOptions::default())
}
pub fn new_with_options(data: &'a [u8], options: DeflateEncodingOptions) -> DeflateEncoder<'a>
{
let length = data.len() + 1024;
let out_array = vec![0; length];
DeflateEncoder {
data,
options,
output_position: 0,
input_position: 0,
output: out_array
}
}
#[cfg(feature = "zlib")]
fn write_zlib_header(&mut self)
{
const ZLIB_CM_DEFLATE: u16 = 8;
const ZLIB_CINFO_32K_WINDOW: u16 = 7;
let level_hint = self.options.strategy.to_level();
let mut hdr = (ZLIB_CM_DEFLATE << 8) | (ZLIB_CINFO_32K_WINDOW << 12);
hdr |= u16::from(level_hint) << 6;
hdr |= 31 - (hdr % 31);
self.output[self.output_position..self.output_position + 2]
.copy_from_slice(&hdr.to_be_bytes());
}
/// Encode a deflate data block with no compression
///
/// # Argument
/// - `bytes`: number of bytes to compress from input as non-compressed
/// bytes
fn encode_no_compression(&mut self, bytes: usize)
{
let final_position = self.input_position + bytes;
/*
* If the input is zero-length, we still must output a block in order
* for the output to be a valid DEFLATE stream. Handle this case
* specially to avoid potentially passing NULL to memcpy() below.
*/
if self.data.is_empty()
{
/* BFINAL and BTYPE */
self.output[self.output_position] = (1 | (DEFLATE_BLOCKTYPE_UNCOMPRESSED << 1)) as u8;
self.output_position += 1;
/* LEN and NLEN */
let num: u32 = 0xFFFF0000;
self.output[self.output_position..self.output_position + 4]
.copy_from_slice(&num.to_le_bytes());
self.output_position += 4;
return;
}
loop
{
let mut bfinal = 0;
let mut len = usize::from(u16::MAX);
if final_position - self.input_position <= usize::from(u16::MAX)
{
bfinal = 1;
len = final_position - self.input_position;
}
/*
* Output BFINAL and BTYPE. The stream is already byte-aligned
* here, so this step always requires outputting exactly 1 byte.
*/
self.output[self.output_position] =
(bfinal | (DEFLATE_BLOCKTYPE_UNCOMPRESSED << 1)) as u8;
self.output_position += 1;
// output len and nlen
let len_u16 = len as u16;
self.output[self.output_position..self.output_position + 2]
.copy_from_slice(&len_u16.to_le_bytes());
self.output_position += 2;
self.output[self.output_position..self.output_position + 2]
.copy_from_slice(&(!len_u16).to_le_bytes());
self.output_position += 2;
// copy from input to output
self.output[self.output_position..self.output_position + len]
.copy_from_slice(&self.data[self.input_position..self.input_position + len]);
self.output_position += len;
self.input_position += len;
if self.input_position == final_position
{
break;
}
}
}
/// Encode a deflate stream
pub fn encode_deflate(&mut self)
{
match self.options.strategy
{
DeflateEncodingStrategy::NoCompression =>
{
self.encode_no_compression(self.data.len());
}
}
}
#[cfg(feature = "zlib")]
pub fn encode_zlib(&mut self) -> Vec
{
let extra = 40 * ((self.data.len() + 41) / 40);
self.output = vec![0_u8; self.data.len() + extra];
self.write_zlib_header();
self.output_position = 2;
self.encode_deflate();
// add adler hash
let hash = crate::utils::calc_adler_hash(self.data);
self.output[self.output_position..self.output_position + 4]
.copy_from_slice(&hash.to_be_bytes());
self.output_position += 4;
self.output.truncate(self.output_position);
core::mem::take(&mut self.output)
}
}
zune-inflate-0.2.54/src/errors.rs 0000644 0000000 0000000 00000006714 10461020230 0014776 0 ustar 0000000 0000000 //! Errors possible when decoding deflate/zlib/gzip
//! streams
use alloc::string::String;
use alloc::vec;
use alloc::vec::Vec;
use core::fmt::{Debug, Display, Formatter};
/// A struct returned when decompression fails
///
/// This struct contains two fields,
///
/// - `error`:Tells you the error that actually occured.
/// - `data`: Gives you decoded data up until that point when
/// the error was encountered.
///
/// One can recover data up to the error if they so wish but
/// guarantees about data state is not given
pub struct InflateDecodeErrors
{
/// reason why decompression fails
pub error: DecodeErrorStatus,
/// Decoded data up until that decompression error
pub data: Vec
}
impl InflateDecodeErrors
{
/// Create a new decode wrapper with data being
/// how many bytes we actually decoded before hitting an error
///
/// # Arguments
/// - `error`: Error encountered during decoding
/// - `data`: Data up to that point of decoding
///
/// # Returns
/// Itself
pub fn new(error: DecodeErrorStatus, data: Vec) -> InflateDecodeErrors
{
InflateDecodeErrors { error, data }
}
/// Create a new decode wrapper with an empty vector
///
/// # Arguments
/// - `error`: Error encountered during decoding.
pub fn new_with_error(error: DecodeErrorStatus) -> InflateDecodeErrors
{
InflateDecodeErrors::new(error, vec![])
}
}
impl Debug for InflateDecodeErrors
{
fn fmt(&self, f: &mut Formatter<'_>) -> core::fmt::Result
{
writeln!(f, "{:?}", self.error)
}
}
pub enum DecodeErrorStatus
{
/// Input data is not enough to construct
/// a full output
InsufficientData,
/// Anything that isn't significant
Generic(&'static str),
/// Anything that isn't significant but we need to
/// pass back information to the user as to what went wrong
GenericStr(String),
///Input data was malformed.
CorruptData,
/// Limit set by the user was exceeded by
/// decompressed output
OutputLimitExceeded(usize, usize),
/// Output CRC does not match stored CRC.
///
/// Only present for zlib
MismatchedCRC(u32, u32),
/// Output Adler does not match stored adler
///
/// Only present for gzip
MismatchedAdler(u32, u32)
}
impl Debug for DecodeErrorStatus
{
fn fmt(&self, f: &mut Formatter<'_>) -> core::fmt::Result
{
match self
{
Self::InsufficientData => writeln!(f, "Insufficient data"),
Self::Generic(reason) => writeln!(f, "{reason}"),
Self::GenericStr(reason) => writeln!(f, "{reason}"),
Self::CorruptData => writeln!(f, "Corrupt data"),
Self::OutputLimitExceeded(limit, current) => writeln!(
f,
"Output limit exceeded, set limit was {limit} and output size is {current}"
),
Self::MismatchedCRC(expected, found) =>
{
writeln!(f, "Mismatched CRC, expected {expected} but found {found}")
}
Self::MismatchedAdler(expected, found) =>
{
writeln!(f, "Mismatched Adler, expected {expected} but found {found}")
}
}
}
}
impl Display for InflateDecodeErrors
{
#[allow(clippy::uninlined_format_args)]
fn fmt(&self, f: &mut Formatter<'_>) -> core::fmt::Result
{
writeln!(f, "{:?}", self)
}
}
#[cfg(feature = "std")]
impl std::error::Error for InflateDecodeErrors {}
zune-inflate-0.2.54/src/gzip_constants.rs 0000644 0000000 0000000 00000000522 10461020230 0016516 0 ustar 0000000 0000000 #![cfg(feature = "gzip")]
pub const GZIP_ID1: u8 = 0x1F;
pub const GZIP_ID2: u8 = 0x8B;
pub const GZIP_CM_DEFLATE: u8 = 8;
pub const GZIP_FRESERVED: u8 = 0xE0;
pub const GZIP_FEXTRA: u8 = 0x04;
pub const GZIP_FOOTER_SIZE: usize = 8;
pub const GZIP_FHCRC: u8 = 0x02;
pub const GZIP_FNAME: u8 = 0x08;
pub const GZIP_FCOMMENT: u8 = 0x10;
zune-inflate-0.2.54/src/lib.rs 0000644 0000000 0000000 00000006337 10461020230 0014231 0 ustar 0000000 0000000 /*
* Copyright (c) 2023.
*
* This software is free software; You can redistribute it or modify it under terms of the MIT, Apache License or Zlib license
*/
//! An incredibly spiffy deflate decoder.
//!
//! This crate features a deflate/zlib decoder inspired by
//! Eric Bigger's [libdeflate].
//!
//! This libary has a smaller set of features hence you should use it
//! if it aligns with your end goals.
//!
//! Use it if
//! - You want a smaller library footprint when compared to flate/miniz-oxide
//! - You want faster speeds than zlib-ng/zlib/miniz-oxide.
//! - You do full buffer decompression and not streaming decompression.
//! - You don't need compression support for now, it will come soon enough.
//! - You want a 100% safe, pure rust implementation with above.
//!
//!Do not use it if
//! - You want compression support, not yet there
//! - You stream your data, not compatible with this library
//!
//! ## Alternatives
//!- For the fastest speeds, check out [libdeflate] (C), if using Rust there is [libdeflater] which
//! provides bindings to [libdeflate]
//!
//!- For streaming support use [flate2-rs] with an appropriate backend(zlib-ng is recommended for speed)
//!
//! # Features
//! You can disable features depending on what you need. the following are
//! features present
//! - gzip: Enable gzip decoding
//! - zlib: Enable zlib decoding
//!
//! These features are enabled by default
//!
//! To disable a feature , modify Cargo.toml to disable default features
//! and add the needed feature , e.g below will include zlib decoding and disable gzip decoding
//! ```toml
//! zune-inflate={ version="0.2",default-features=false,feature=["zlib"]}
//! ```
//!
//! # Errors
//! In case of an error, the library returns the error and the decoded
//! data up to when the error was encountered hence that data can be recovered
//! but no data further than that can be recovered
//!
//!
//! # Usage
//!
//! Decoding delfate data
//
//! ```no_run
//! use zune_inflate::DeflateDecoder;
//! let totally_valid_data = [0;23];
//! let mut decoder = DeflateDecoder::new(&totally_valid_data);
//!
//! let decompressed =decoder.decode_deflate().unwrap();
//! ```
//!
//! Decoding zlib data
//! ```no_run
//! use zune_inflate::DeflateDecoder;
//! // yea this isn't valid
//! let totally_valid_data = [0;23];
//! let mut decoder = DeflateDecoder::new(&totally_valid_data);
//!
//! let decompressed =decoder.decode_zlib().unwrap();
//! ```
//!
//! Decoding zlib data without confirming the adler32 checksum
//! ```no_run
//! use zune_inflate::DeflateDecoder;
//! use zune_inflate::DeflateOptions;
//! let totally_valid_data=[0;23];
//! let mut options = DeflateOptions::default()
//! .set_confirm_checksum(false);
//! let decoder = DeflateDecoder::new_with_options(&totally_valid_data,options);
//!
//! ```
//!
//! [libdeflate]: https://github.com/ebiggers/libdeflate
//! [libdeflater]: https://github.com/adamkewley/libdeflater
//! [flate2-rs]: https://github.com/rust-lang/flate2-rs
//!
#![cfg_attr(not(feature = "std"), no_std)]
extern crate alloc;
pub use crate::decoder::{DeflateDecoder, DeflateOptions};
pub use crate::encoder::DeflateEncoder;
mod bitstream;
mod constants;
mod crc;
mod decoder;
mod encoder;
pub mod errors;
mod gzip_constants;
mod utils;
zune-inflate-0.2.54/src/utils.rs 0000644 0000000 0000000 00000005142 10461020230 0014614 0 ustar 0000000 0000000 use core::cell::Cell;
/// make_decode_table_entry() creates a decode table entry for the given symbol
/// by combining the static part 'decode_results[sym]' with the dynamic part
/// 'len', which is the remaining codeword length (the codeword length for main
/// table entries, or the codeword length minus TABLEBITS for subtable entries).
///
/// In all cases, we add 'len' to each of the two low-order bytes to create the
/// appropriately-formatted decode table entry. See the definitions of the
/// *_decode_results[] arrays below, where the entry format is described.
pub(crate) fn make_decode_table_entry(decode_results: &[u32], sym: usize, len: u32) -> u32
{
decode_results[sym] + (len << 8) + len
}
/// A safe version of src.copy_within that helps me because I tend to always
/// confuse the arguments
pub fn fixed_copy_within(dest: &mut [u8], src_offset: usize, dest_offset: usize)
{
// for debug builds ensure we don't go out of bounds
debug_assert!(
dest_offset + SIZE <= dest.len(),
"[dst]: End position {} out of range for slice of length {}",
dest_offset + SIZE,
dest.len()
);
dest.copy_within(src_offset..src_offset + SIZE, dest_offset);
}
#[inline(always)]
pub fn copy_rep_matches(dest: &mut [u8], offset: usize, dest_offset: usize, length: usize)
{
// This is a slightly complicated rep match copier that has
// no bounds check.
// The only invariant we need to uphold is dest[dest_offset] should
// copy from dest[offset]
// i.e in the first iteration, the first entry in the window will point
// to dest[offset] and the
// last entry will point to dest[dest_offset]
// it's easy to prove dest[offset] since we take our slice
// from offset.
// but proving dest[dest_offset] is trickier
// If we were at offset, to get to dest_offset, we could
// 1. Get difference between dest_offset and offset
// 2. Add that difference to offset.
//
let diff = dest_offset - offset + 1;
// note
for window in Cell::from_mut(&mut dest[offset..dest_offset + length + 2])
.as_slice_of_cells()
.windows(diff)
{
window.last().unwrap().set(window[0].get());
}
}
/// Return the minimum of two usizes in a const context
#[rustfmt::skip]
pub const fn const_min_usize(a: usize, b: usize) -> usize
{
if a < b { a } else { b }
}
/// Calculate the adler hash of a piece of data.
#[inline(never)]
#[cfg(feature = "zlib")]
pub fn calc_adler_hash(data: &[u8]) -> u32
{
use simd_adler32::Adler32;
let mut hasher = Adler32::new();
hasher.write(data);
hasher.finish()
}