dev-python/tiktoken (ha-bleeding-edge)

Search

Package Information

Description:
Fast BPE tokeniser for use with OpenAI's models.
Homepage:
https://github.com/openai/tiktoken
License:
MIT Apache-2.0 Unicode-3.0

Versions

Version EAPI Keywords Slot
0.12.0 8 ~amd64 0

Metadata

No metadata available.

Lint Warnings

Manifest

Type File Size Versions
Unmatched Entries
Type File Size
DIST aho-corasick-1.1.3.crate 183311 bytes
DIST autocfg-1.5.0.crate 18729 bytes
DIST bit-set-0.5.3.crate 14470 bytes
DIST bit-vec-0.6.3.crate 19927 bytes
DIST bstr-1.12.0.crate 351557 bytes
DIST fancy-regex-0.13.0.crate 85930 bytes
DIST heck-0.5.0.crate 11517 bytes
DIST indoc-2.0.6.crate 17164 bytes
DIST libc-0.2.177.crate 792045 bytes
DIST memchr-2.7.6.crate 97616 bytes
DIST memoffset-0.9.1.crate 9032 bytes
DIST once_cell-1.21.3.crate 34534 bytes
DIST portable-atomic-1.11.1.crate 185506 bytes
DIST proc-macro2-1.0.101.crate 53886 bytes
DIST pyo3-0.26.0.crate 1151579 bytes
DIST pyo3-build-config-0.26.0.crate 34309 bytes
DIST pyo3-ffi-0.26.0.crate 78247 bytes
DIST pyo3-macros-0.26.0.crate 8906 bytes
DIST pyo3-macros-backend-0.26.0.crate 81809 bytes
DIST quote-1.0.41.crate 31408 bytes
DIST regex-1.12.2.crate 163843 bytes
DIST regex-automata-0.4.13.crate 625250 bytes
DIST regex-syntax-0.8.8.crate 359141 bytes
DIST rustc-hash-2.1.1.crate 14154 bytes
DIST serde-1.0.228.crate 83652 bytes
DIST serde_core-1.0.228.crate 63111 bytes
DIST serde_derive-1.0.228.crate 59605 bytes
DIST syn-2.0.107.crate 301716 bytes
DIST target-lexicon-0.13.3.crate 28498 bytes
DIST tiktoken-0.12.0.tar.gz 37806 bytes
DIST unicode-ident-1.0.19.crate 47480 bytes
DIST unindent-0.2.4.crate 7422 bytes