GigaToken: ~1000x faster Language model tokenization
(github.com)
160 points
by syrusakbary
2 hours ago |
27 comments
()
()
()
()
()
()
()
()
()
