AIbase
Home

MoE-LLM

Public

A modular, custom-made Mixture-of-Experts transformer LLM showcasing modern architectural features from recent LLM research, including RoPE, GQA, sliding-window attention, and curriculum training.

attention-mechanismscurriculum-learningdeep-learninglanguage-modelinglarge-language-modelsmachine-learningmixture-of-expertspytorchresearch-coderotary-embeddings
Creat：2025-08-02T18:00:10
Update：2025-11-25T01:32:58
0
Stars
0
Stars Increase

Related projects

Computer Science

Hot
awesome-list

? Path to a free self-taught education in Computer Science!

198852
1年前
+153today

Tensorflow

deep-learning

An Open Source Machine Learning Framework for Everyone

192719
3年前
+38today

Stable Diffusion Webui

Hot
ai

Stable Diffusion web UI

158833
2年前
+73today

Transformers

Hot
bert

? Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

153615
3年前
+136today

30 Seconds Of Code

astro

Coding articles to level up your development skills

125990
1年前
+35today

Pytorch

Hot
autograd

Tensors and Dynamic neural networks in Python with strong GPU acceleration

95701
1年前
+88today

Opencv

c-plus-plus

Open Source Computer Vision Library

85202
8年前
+42today

Netdata

alerting

X-Ray Vision for your infrastructure!

76944
1年前
+31today

Deep Live Cam

Hot
ai

real time face swap and one-click video deepfake with only a single image

76195
1年前
+63today

D2l Zh

Hot
book

《动手学深度学习》：面向中文读者、能运行、可讨论。中英文版被70多个国家的500多所大学用于教学。

74277
1年前
+60today