HomeAI Tutorial

gpt-tokenizer

Public

The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4 / GPT-4o / GPT-o1. Port of OpenAI's tiktoken with additional features.

Heure de création2023-03-22T12:32:09
Heure de mise à jour2025-03-26T03:51:14
https://gpt-tokenizer.dev
703
Stars
2
Stars Increase