The Deep Ignorance Model Suite is a collection of 18 large language models with 6.9 billion parameters each. It aims to study methods of preventing models from learning unsafe technical capabilities (such as CBRN-related capabilities) by filtering pretrained data. This suite demonstrates that filtering data can effectively avoid the learning of bad knowledge while maintaining general performance and having anti-tampering capabilities.

Natural Language ProcessingTransformersEnglish

A fine-tuned version based on the EleutherAI_pythia-1b-deduped model for generating concise summaries

Natural Language ProcessingTransformers

Pile-T5 Base is an encoder-decoder model trained on The Pile dataset using the T5x library, trained for 2 million steps with MLM objective, approximately 2 trillion tokens.

Natural Language ProcessingTransformersEnglish

Pile-T5 XXL is an encoder-decoder model trained on The Pile dataset using the T5x library, employing a MLM objective similar to the original T5 model, trained for 2 million steps (approximately 2 trillion tokens).

Natural Language ProcessingTransformersEnglish

A 31-million parameter conversational model fine-tuned from EleutherAI/pythia-31m, supporting multi-turn dialogues and various task scenarios

Natural Language ProcessingTransformersEnglish

Llemma 34B is a language model specialized in the field of mathematics, initialized based on the weights of Code Llama 34B and trained on 50 billion tokens from the Proof-Pile-2 dataset.

Natural Language ProcessingTransformersEnglish

Llemma 34B is a language model specialized in mathematics, initialized with Code Llama 34B weights and trained on the Proof-Pile-2 dataset.

Natural Language ProcessingTransformersEnglish

A Korean multilingual large model fine-tuned on 260,000 instruction-following datasets based on EleutherAI/polyglot-ko-12.8b

Natural Language ProcessingTransformersKorean

Pile-T5 Large is an encoder-decoder model trained on The Pile dataset based on the T5x library, primarily used for English text-to-text generation tasks.

Natural Language ProcessingTransformersEnglish

A chat model trained on multiple Korean instruction datasets based on the EleutherAI/polyglot-ko-5.8b model

Natural Language ProcessingTransformersKorean

KULLM-Polyglot-5.8B-v2 is a large language model version that performs parameter-efficient fine-tuning on the KULLM v2 dataset based on EleutherAI/polyglot-ko-5.8b. It is specifically optimized for Korean natural language processing tasks and has a scale of 5.8 billion parameters.

Natural Language ProcessingTransformersKorean

KULLM v2 fine-tuned version based on EleutherAI/polyglot-ko-12.8b, supporting Korean multilingual large language model

Natural Language ProcessingTransformersKorean

A multilingual Korean text generation model fine-tuned on the Korean Alpaca dataset v1.1b based on EleutherAI/polyglot-ko-12.8b

Natural Language ProcessingTransformersKorean

A 7-billion-parameter open-source dialogue model fine-tuned from EleutherAI Pythia-7B, trained on over 40 million instructions using 100% carbon-negative computing resources

Natural Language ProcessingTransformersEnglish

Pythia-1B is a language model specialized for interpretability research developed by EleutherAI, belonging to the 1-billion-parameter version in the Pythia suite, trained on The Pile dataset.

Natural Language ProcessingTransformersEnglish

Pythia-12B is the largest model in EleutherAI's scalable language model suite, with 12 billion parameters, specifically designed to advance scientific research on large language models

Natural Language ProcessingTransformersEnglish

Pythia-12B-deduped is a large language model with 12B parameters developed by EleutherAI, designed specifically for interpretability research and trained on the deduplicated Pile dataset.

Natural Language ProcessingTransformersEnglish

Pythia-6.9B is a large-scale language model developed by EleutherAI, part of the Pythia scalable suite, specifically designed to facilitate interpretability research.

Natural Language ProcessingTransformersEnglish

Pythia-1B Deduplicated is a language model developed by EleutherAI specifically for interpretability research, trained on the deduplicated Pile dataset using Transformer architecture with 1 billion parameters

Natural Language ProcessingTransformersEnglish

Pythia is a series of causal language models developed by EleutherAI, specifically designed for interpretability research. It includes 8 model sizes ranging from 70 million to 12 billion parameters, providing 154 training checkpoints.

Natural Language ProcessingTransformersEnglish