Trending / Packages / crates.io

tk-serialize

crates.io

The reader for the canonical `tokenizer.json` format: a `hifijson` parse and a builder that constructs `tk-encode`'s runtime pipeline directly, with no serde anywhere. Legacy config shapes are `tk-convert`'s job, not this crate's.

Latest v0.1.0-rc.3 · Oct 19, 2026

Downloads

No download history yet.

Linked repositories

##RepositoryRepositoryConnectionConnection
1huggingface/tokenizersSource package · Manifest

Recent releases

VersionPublished
0.1.0-rc.3Oct 19, 2026
0.1.0-rc.2Sep 21, 2026
0.1.0-rc.1Sep 21, 2026
0.1.0-dev.0Sep 21, 2026

Metadata updated Oct 25, 2026 · Downloads updated Oct 30, 2026

Download counts include automated and repeat installs. Gaps indicate missing observations.