Lemur is the first open-source pre-trained language model that excels in both text and code tasks. By pre-training on a large-scale code dataset, it maintains competitive performance in text understanding and knowledge tasks while surpassing other models in programming benchmarks. The contributions of Lemur are already reshaping the landscape of language model technology, providing a powerful tool for developers, researchers, and organizations.