AIbase
Product LibraryTool NavigationMCP

lm-vocab-trimmer

Public

Vocabulary Trimming (VT) is a model compression technique, which reduces a multilingual LM vocabulary to a target language by deleting irrelevant tokens from its vocabulary. This repository contains a python-library vocabtrimmer, that remove irrelevant tokens from a multilingual LM vocabulary for the target language.

Creat2023-02-25T21:24:18
Update2025-03-02T19:13:20
41
Stars
0
Stars Increase

Related projects