tk-serialize
crates.ioThe reader for the canonical `tokenizer.json` format: a `hifijson` parse and a builder that constructs `tk-encode`'s runtime pipeline directly, with no serde anywhere. Legacy config shapes are `tk-convert`'s job, not this crate's.
Latest v0.1.0-rc.3 · Oct 19, 2026
Downloads
No download history yet.
Linked repositories
| ## | RepositoryRepository | ConnectionConnection |
|---|---|---|
| 1 | huggingface/tokenizers | Source package · Manifest |
Recent releases
| Version | Published |
|---|---|
| 0.1.0-rc.3 | Oct 19, 2026 |
| 0.1.0-rc.2 | Sep 21, 2026 |
| 0.1.0-rc.1 | Sep 21, 2026 |
| 0.1.0-dev.0 | Sep 21, 2026 |