Changes since 1.5.0:
- The Rust crate is now compatible with Miri.
- ~1% performance improvement on Arm NEON contributed by @divinity76 (#384).
- Various fixes and improvements in the CMake build.
- The MSRV of b3sum is now 1.74.1. (The MSRV of the library crate is
unchanged, 1.66.1.)
Changes since 1.4.1:
- The Rust crate's Hasher type has gained new helper methods for common
forms of IO: update_reader, update_mmap, and update_mmap_rayon. The
latter matches the default behavior of b3sum. The mmap methods are
gated by the new "mmap" Cargo feature.
- Most of the Rust crate's public types now implement the Zeroize trait.
This is gated by the new "zeroize" Cargo feature.
- The Rust crate's Hash types now implements the serde Serialize and
Deserialize traits. This is gated by the new "serde" Cargo feature.
- The C library now uses atomics to cache detected CPU features under
most compilers other than MSVC. Previously this was a non-atomic
write, which was probably "benign" but made TSan unhappy.
- NEON support is now disabled by default on big-endian AArch64.
Previously this was a build error if the caller didn't explicitly
disable it.
New methods:
- update_reader
- update_mmap
- update_mmap_rayon
These are more discoverable, more convenient, and safer.
There are two problems I want to avoid by taking a `Path` instead of a
`File`. First, exposing `Mmap` objects to the caller is fundamentally
unsafe, and making `maybe_mmap_file` private avoids that issue. Second,
taking a `File` raises questions about whether memory mapped reads
should behave like regular file reads. (Should they respect the current
seek position? Should they update the seek position?) Taking a `Path`
from the caller and opening the `File` internally avoids these
questions.
Changes since 1.4.0:
- Improved performance in the ARM NEON implementation for both C and
Rust callers. This affects AArch64 targets by default and ARMv7
targets that explicitly enable (and support) NEON. The size of the
improvement depends on the microarchitecture, but I've benchmarked
~1.3x on a Cortex-A53 and ~1.2x on an Apple M1. Contributed by
@sdlyyxy in #319.
- The MSRV is now 1.66.1 for both the `blake3` crate and `b3sum`.
Changes since 1.3.3:
- The C implementation provides a `CMakeLists.txt` for callers who build
with CMake. The CMake build is not yet stable, and callers should
expect breaking changes in patch version updates. The "by hand" build
will always continue to be supported and documented.
- `b3sum` supports the `--seek` flag, to set the starting position in
the output stream.
- `b3sum --check` prints a summary of errors to stderr.
- `Hash::as_bytes` is const.
- `Hash` supports `from_bytes`, which is const.
Changes since 1.3.2:
- Fix incorrect output from AVX-512 intrinsics under GCC 5.4 and 6.1 in
debug mode. This bug was found in unit tests and probably doesn't
affect the public API in practice. See
https://github.com/BLAKE3-team/BLAKE3/issues/271.
Changes since 1.3.1:
- Dependency updates only. This includes updating Clap to v4, which
changes the format of the `b3sum --help` output. The new MSRV is
1.59.0 for `blake3` and 1.60.0 for `b3sum`. Note that this project
doesn't have any particular MSRV policy, and we don't consider MSRV
bumps to be breaking changes.
Changes since 1.3.0:
- The unstable `traits-preview` feature now includes an implementation
of `crypto_common::BlockSizeUser`, AKA
`digest::core_api::BlockSizeUser`. This allows `blake3::Hasher` to be
used with `hmac::SimpleHmac`.
Changes since 1.2.0:
- Added blake3_hasher_reset to the C API, for parity with the Rust API.
- Updated digest to v0.10. This version merged the crypto-mac crate with
digest, so the dependency on crypto-mac has been removed. These trait
implementations are still gated behind the "traits-preview" feature.
- Updated clap to v3.
Adjust to the following changes that happened in digest:
- The crypto-mac crate has been merged into digest (with "mac" feature
enabled)
- Various traits have been split up
- The Digest and Mac traits now share their update/finalize/reset
implementations
- The BlockInput trait was dropped without replacement apparently (as
long as the low-level core API is not used)
Changes since 1.1.0:
- SECURITY FIX: Fixed an instance of undefined behavior in the Windows
SSE2 assembly implementations, which affected both the Rust and C
libraries in their default build configurations. See
https://github.com/BLAKE3-team/BLAKE3/issues/206. The cause was a
vector register that wasn't properly saved and restored. This bug has
been present since SSE2 support was initially added in v0.3.7. The
effects of this bug depend on surrounding code and compiler
optimizations; see test_issue_206_windows_sse2 for an example of this
bug causing incorrect hash output. Note that even when surrounding
code is arranged to trigger this bug, the SSE2 implementation is
normally only invoked on CPUs where SSE4.1 (introduced in 2007) isn't
supported. One notable exception, however, is if the Rust library is
built in `no_std` mode, with `default_features = false` or similar. In
that case, runtime CPU feature detection is disabled, and since LLVM
assumes that all x86-64 targets support SSE2, the SSE2 implementation
will be invoked. For that reason, Rust callers who build `blake3` in
`no_std` mode for x86-64 Windows targets are the most likely to
trigger this bug. We found this bug in internal testing, and we aren't
aware of any callers encountering it in practice.
- Added the Hasher::count() method.
Changes since 1.0.0:
- The NEON implementation is now enabled by default on AArch64 targets.
Previously it was disabled without the "neon" Cargo feature in Rust or
the "BLAKE3_USE_NEON=1" preprocessor flag in C. This is still the case
on ARM targets other than AArch64, because of the lack of dynamic CPU
feature detection on ARM. Contributed by @rsdy.
- The previous change leads to some build incompatibilities,
particularly in C. If you build the C implementation for AArch64
targets, you now need to include blake3_neon.c, or else you'll get a
linker error like "undefined reference to `blake3_hash_many_neon'". If
you don't want the NEON implementation, you need to explicitly set
"BLAKE3_USE_NEON=0". On the Rust side, AArch64 targets now require the
C toolchain by default. build.rs includes workarounds for missing or
very old C compilers for x86, but it doesn't currently include such
workarounds for AArch64. If we hear about build breaks related to
this, we can add more workarounds as appropriate.
- C-specific Git tags ("c-0.3.7" etc.) have been removed, and all the
projects in this repo (Rust "blake3", Rust "b3sum", and the C
implementation) will continue to be versioned in lockstep for the
foreseeable future.
Changes since 0.3.8:
- Add Hash::from_hex() and implement FromStr for Hash.
- Implement Display for Hash, equivalent to Hash::to_hex().
- Implement PartialEq<[u8]> for Hash, using constant_time_eq.
- Change derive_key() to return a 32-byte array. As with hash() and
keyed_hash(), callers who want a non-default output length can use
Hasher::finalize_xof().
- Replace Hasher::update_with_join() with Hasher::update_rayon(). The
former was excessively generic, and the Join trait leaked
implementation details. As part of this change, the Join trait is no
longer public.
- Upgraded arrayvec to 0.7.0, which uses const generics. This bumps the
minimum supported Rust compiler version to 1.51.
- Gate the digest and crypto-mac trait implementations behind an
unstable feature, "traits-preview". As part of this change upgrade
crypto-mac to 0.11.0.
These clutter the toplevel API, and their prominence might lead callers
to prefer them as a first resort, which probably isn't a good idea.
Restricting multithreading to `Hasher::update_rayon` feels better,
similar to what we've done with `Hasher::finalize_xof`. (But I think
`update_rayon` is still an improvement over the trait-based interface
that it replaced.)
Changes since 0.3.6:
- BUGFIX: The C implementation was incorrect on big endian systems for
inputs longer than 1024 bytes. This bug affected all previous versions
of the C implementation. Little endian platforms like x86 were
unaffected. The Rust implementation was also unaffected.
@jakub-zwolakowski and @pascal-cuoq from TrustInSoft reported this
bug: https://github.com/BLAKE3-team/BLAKE3/pull/118
- BUGFIX: The C build on x86-64 was producing binaries with an
executable stack. @tristanheaven reported this bug:
https://github.com/BLAKE3-team/BLAKE3/issues/109
- @mkrupcale added optimized implementations for SSE2. This improves
performance on older x86 processors that don't support SSE4.1.
- The C implementation now exposes the
`blake3_hasher_init_derive_key_raw` function, to make it easier to
implement language bindings. Added by @k0001.
Wire up basic functions and features for SSE2 support using the SSE4.1 version
as a basis without implementing the SSE2 instructions yet.
* Cargo.toml: add no_sse2 feature
* benches/bench.rs: wire SSE2 benchmarks
* build.rs: add SSE2 rust intrinsics and assembly builds
* c/Makefile.testing: add SSE2 C and assembly targets
* c/README.md: add SSE2 to C build instructions
* c/blake3_c_rust_bindings/build.rs: add SSE2 C rust binding builds
* c/blake3_c_rust_bindings/src/lib.rs: add SSE2 C rust bindings
* c/blake3_dispatch.c: add SSE2 C dispatch
* c/blake3_impl.h: add SSE2 C function prototypes
* c/blake3_sse2.c: add SSE2 C intrinsic file starting with SSE4.1 version
* c/blake3_sse2_x86-64_{unix.S,windows_gnu.S,windows_msvc.asm}: add SSE2
assembly files starting with SSE4.1 version
* src/ffi_sse2.rs: add rust implementation using SSE2 C rust bindings
* src/lib.rs: add SSE2 rust intrinsics and SSE2 C rust binding rust SSE2 module
configurations
* src/platform.rs: add SSE2 rust platform detection and dispatch
* src/rust_sse2.rs: add SSE2 rust intrinsic file starting with SSE4.1 version
* tools/instruction_set_support/src/main.rs: add SSE2 feature detection
Changes since 0.3.4:
- The `digest` dependency is now v0.9 and the `crypto-mac` dependency is
now v0.8.
- Intel CET is supported in the assembly implementations.
- `b3sum` error output includes filepaths again.
Changes since 0.3.3:
- `b3sum` now supports the `--check` flag. This is intended to be a
drop-in replacement for e.g. `md5sum --check` from Coreutils. The
behavior is somewhat stricter than Coreutils with respect to invalid
Unicode in filenames. For a complete description of how `--check`
works, see the file `b3sum/what_does_check_do.md`.
- To support the `--check` feature, backslashes and newlines that appear
in filenames are now escaped in the output of `b3sum`. This is done
the same way as in Coreutils.
- To support `--check` interoperability between Unix and Windows,
backslashes in filepaths on Windows are now replaced with forward
slashes in the output of `b3sum`. Note that this is different from
Coreutils.
I've tested manually and found that 1.0.4 is the oldest version of `cc`
that builds successfully for us. (Version 1.0.3 is missing the
`is_flag_supported` method.)
This change might help with
https://github.com/BLAKE3-team/BLAKE3/issues/83. That said, the
underlying issue there is related to "minimum supported Rust versions",
and `blake3` does not yet have an MSRV other than latest stable.
Changes since 0.3.0:
- The x86 build now automatically falls back to "pure" Rust intrinsics,
under either of two possible conditions:
1. The `cc` crate fails to invoke a C compiler at all, indicating that
nothing of the right name (e.g. "cc" or "$CC" on Unix) is installed.
2. The `cc` crate detects that the compiler doesn't support AVX-512
flags, usually because it's too old.
The end result should be that most callers successfully build the
assembly implementations, and that callers who can't build those see a
warning but not an error. (And note that Cargo suppresses warnings for
non-path depencies.)
There are two scenarios where compiling AVX-512 C or assembly code might
not work:
1. There might not be a C compiler installed at all. Most commonly this
is either in cross-compiling situations, or with the Windows GNU
target.
2. The installed C compiler might not support e.g. -mavx512f, because
it's too old.
In both of these cases, print a relevant warning, and then automatically
fall back to using the pure Rust intrinsics build.
Note that this only affects x86 targets. Other targets always use pure
Rust, unless the "neon" feature is enabled.
Changes since version 0.2.3:
- The optimized assembly implementations are now built by default. They
perform better than the intrinsics implementations, and they compile
much more quickly. Bringing the default behavior in line with reported
benchmark figures should also simplify things for people running their
own benchmarks. Previously this crate only built Rust intrinsics
implementations by default, and the assembly implementations were
gated by the (slightly confusingly named) "c" feature. Now the "c"
feature is gone, and applications that need the old behavior can use
the new "pure" feature. Mainly this will be applications that don't
want to require a C compiler. Note that the `b3sum` crate previously
activated the "c" feature by default, so its behavior hasn't changed.
The biggest change here is that assembly implementations are enabled by
default.
Added features:
- "pure" (Pure Rust, with no C or assembly implementations.)
Removed features:
- "c" (Now basically the default.)
Renamed features;
- "c_prefer_intrinsics" -> "prefer_intrinsics"
- "c_neon" -> "neon"
Unchanged:
- "rayon"
- "std" (Still the only feature on by default.)
Changes since version 0.2.2:
- Bug fix: Commit 13556be fixes a crash on Windows when using the SSE4.1
assembly implementation (--features=c, set by default for b3sum). This
is undefined behavior and therefore a potential security issue.
- b3sum now supports the --num-threads flag.
- The C API now includes a blake3_hasher_finalize_seek() function, which
returns output from any position in the extended output stream.
- Build fix: Commit 5fad419 fixes a compiler error in the AVX-512 C
intrinsics implementation targeting the Windows GNU ABI.
Jack O'Connor
Ralph Minderhoud
Banyc
Elichai Turkel
jbis9051
Matthias Schiffer
rsdy
Paul Grandperrin
Jack O'Connor
Matthew Krupcale
Justus K