Gentoo Packages

Search

Install

Install this package:

emerge -a dev-python/tiktoken

Package Information

Description:

A fast BPE tokeniser for use with OpenAI's models

Homepage:

https://github.com/openai/tiktoken https://pypi.org/project/tiktoken/

License:

MIT Apache-2.0-with-LLVM-exceptions MIT Unicode-3.0

Versions

Metadata

Maintainers

• Florian Albrechtskirchinger (falbrechtskirchinger@gmail.com) - Type: person

Upstream

• Remote IDs:
  • pypi: tiktoken
  • github: openai/tiktoken

<pkgmetadata>
	<maintainer type="person">
		<email>falbrechtskirchinger@gmail.com</email>
		<name>Florian Albrechtskirchinger</name>
	</maintainer>
	<upstream>
		<remote-id type="pypi">tiktoken</remote-id>
		<remote-id type="github">openai/tiktoken</remote-id>
	</upstream>
</pkgmetadata>

Lint Warnings

• [Warning] Ebuild 0.14.0 has a line exceeding 80 positions on line 30.
• [Warning] Ebuild 0.14.0 has a line exceeding 80 positions on line 38.
• [Warning] Ebuild 0.14.0 has a line exceeding 80 positions on line 39.
• [Warning] Ebuild 0.14.0 has a line exceeding 80 positions on line 78.
• [Error] Manifest entry for unused DIST file 'pyo3-0.29.0.crate'
• [Error] Manifest entry for unused DIST file 'pyo3-build-config-0.29.0.crate'
• [Error] Manifest entry for unused DIST file 'pyo3-ffi-0.29.0.crate'
• [Error] Manifest entry for unused DIST file 'pyo3-macros-0.29.0.crate'
• [Error] Manifest entry for unused DIST file 'pyo3-macros-backend-0.29.0.crate'
• [Error] Manifest entry for unused DIST file 'tiktoken-0.14.0-crates.tar.xz'
• [Error] Manifest entry for unused DIST file 'tiktoken-0.14.0.gh.tar.gz'
• [Error] Manifest entry for unused DIST file 'tiktoken-encodings-v2026.03.26.tar.xz'
• [Error] Repository is missing a repo-name in metadata/layout.conf or profiles/repo_name
• [Notice] Package has only unstable keywords for arch(es): [ amd64, arm64 ], all versions are unstable: [ 0.14.0 ]

Files

• tiktoken-0.13.0-special-token-roudtrip.patch

Manifest