Gentoo Packages

Search

Package Information

Description:

Implementation of today's most used tokenizers

Homepage:

https://github.com/huggingface/tokenizers

License:

Apache-2.0 Apache-2.0-with-LLVM-exceptions BSD-2 BSD ISC MIT MPL-2.0 Unicode-DFS-2016

Versions

Metadata

Maintainers

• Tupone Alfredo (tupone@gentoo.org) - Type: person

Upstream

• Remote IDs:
  • github: huggingface/tokenizers

<pkgmetadata>
	<maintainer type="person">
		<email>tupone@gentoo.org</email>
		<name>Tupone Alfredo</name>
	</maintainer>
	<upstream>
		<remote-id type="github">huggingface/tokenizers</remote-id>
	</upstream>
</pkgmetadata>

Lint Warnings

• [Warning] Missing md5-cache for version 0.22.1. Run 'ebuild <ebuild_file> manifest' or 'egencache' to generate it.

Files

• tokenizers-0.21.2-test.patch

Manifest