Gentoo Packages

Search

Install

Install this package:

emerge -a dev-python/curated-tokenizers

Package Information

Description:

Cython-compiled tokenizer building blocks for spaCy's curated- transformers integration. Slim WordPiece / SentencePiece / BBPE. Direct dep of dev-python/spacy-curated-transformers.

Homepage:

https://github.com/explosion/curated-tokenizers https://pypi.org/project/curated-tokenizers/

License:

MIT

Versions

Metadata

Description

• Cython-compiled tokenizer building blocks for spaCy's curated- transformers integration. Slim WordPiece / SentencePiece / BBPE. Direct dep of dev-python/spacy-curated-transformers.

Maintainers

• Ivan S. Titov (iohann.s.titov@gmail.com) - Type: person

Upstream

• Remote IDs:
  • github: explosion/curated-tokenizers
  • pypi: curated-tokenizers

<pkgmetadata>
	<maintainer type="person">
		<email>iohann.s.titov@gmail.com</email>
		<name>Ivan S. Titov</name>
	</maintainer>
	<longdescription lang="en">
    Cython-compiled tokenizer building blocks for spaCy's curated-
    transformers integration. Slim WordPiece / SentencePiece / BBPE.
    Direct dep of dev-python/spacy-curated-transformers.
  </longdescription>
	<upstream>
		<remote-id type="github">explosion/curated-tokenizers</remote-id>
		<remote-id type="pypi">curated-tokenizers</remote-id>
	</upstream>
</pkgmetadata>

Lint Warnings

• [Error] Missing md5-cache for ebuild curated-tokenizers-0.0.9

Manifest