Gentoo Packages

Search

Install

Install this package:

emerge -a dev-python/tiktoken

Package Information

Description:

Fast BPE tokeniser for use with OpenAI's models.

Homepage:

https://github.com/openai/tiktoken

License:

MIT Apache-2.0 Unicode-3.0

Versions

Metadata

Maintainers

• Andrew Udvare (audvare@gmail.com) - Type: person

Upstream

• Remote IDs:
  • github: openai/tiktoken
  • pypi: tiktoken

<pkgmetadata>
	<maintainer type="person">
		<email>audvare@gmail.com</email>
		<name>Andrew Udvare</name>
	</maintainer>
	<upstream>
		<remote-id type="github">openai/tiktoken</remote-id>
		<remote-id type="pypi">tiktoken</remote-id>
	</upstream>
</pkgmetadata>

Lint Warnings

• [Error] Manifest entry for unused DIST file 'aho-corasick-1.1.5.crate'
• [Error] Manifest entry for unused DIST file 'bit-set-0.8.0.crate'
• [Error] Manifest entry for unused DIST file 'bit-vec-0.8.0.crate'
• [Error] Manifest entry for unused DIST file 'bstr-1.13.1.crate'
• [Error] Manifest entry for unused DIST file 'fancy-regex-0.19.0.crate'
• [Error] Manifest entry for unused DIST file 'heck-0.5.0.crate'
• [Error] Manifest entry for unused DIST file 'libc-0.2.189.crate'
• [Error] Manifest entry for unused DIST file 'memchr-2.8.3.crate'
• [Error] Manifest entry for unused DIST file 'once_cell-1.21.4.crate'
• [Error] Manifest entry for unused DIST file 'portable-atomic-1.15.0.crate'
• [Error] Manifest entry for unused DIST file 'proc-macro2-1.0.107.crate'
• [Error] Manifest entry for unused DIST file 'pyo3-0.29.2.crate'
• [Error] Manifest entry for unused DIST file 'pyo3-build-config-0.29.2.crate'
• [Error] Manifest entry for unused DIST file 'pyo3-ffi-0.29.2.crate'
• [Error] Manifest entry for unused DIST file 'pyo3-macros-0.29.2.crate'
• [Error] Manifest entry for unused DIST file 'pyo3-macros-backend-0.29.2.crate'
• [Error] Manifest entry for unused DIST file 'quote-1.0.47.crate'
• [Error] Manifest entry for unused DIST file 'regex-1.13.1.crate'
• [Error] Manifest entry for unused DIST file 'regex-automata-0.4.18.crate'
• [Error] Manifest entry for unused DIST file 'regex-syntax-0.8.11.crate'
• [Error] Manifest entry for unused DIST file 'rustc-hash-2.1.3.crate'
• [Error] Manifest entry for unused DIST file 'serde_core-1.0.229.crate'
• [Error] Manifest entry for unused DIST file 'serde_derive-1.0.229.crate'
• [Error] Manifest entry for unused DIST file 'syn-2.0.119.crate'
• [Error] Manifest entry for unused DIST file 'syn-3.0.4.crate'
• [Error] Manifest entry for unused DIST file 'target-lexicon-0.13.5.crate'
• [Error] Manifest entry for unused DIST file 'tiktoken-0.14.0.tar.gz'
• [Error] Manifest entry for unused DIST file 'unicode-ident-1.0.24.crate'
• [Error] Repository is missing a repo-name in metadata/layout.conf or profiles/repo_name
• [Notice] Package has only unstable keywords for arch(es): [ amd64 ], all versions are unstable: [ 0.14.0 ]

Manifest