Scaling Hierarchical Networks (cartesia.ai)

Its cool, but it goes against the whole optimization that NLP has gone through with BPE and redundancy/repetition in large data in general.

Put this on a coding task or English and you will underperform against tokenizer that exploit multi-char patterns and can decode much faster.

Given where things are going with spec decode (I've seen 32 token spec-decode pop up in the frontier models), you will need a very domain-specific use for this to be useful imo like massive tabular data, but at that point you already have everything structured, and might as well use an LLM with polars/sql/arrow.

Oh man, that haskell quicksort tokenization has got me feeling great