gpt-tokenizer
PublicThe fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4 / GPT-4o / GPT-o1. Port of OpenAI's tiktoken with additional features.
Creat:2023-03-22T12:32:09
Update:2025-03-26T03:51:14
https://gpt-tokenizer.dev
596
Stars
0
Stars Increase