Popular GitHub Repositories for Sparse

Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors

A package for the sparse identification of nonlinear dynamical systems from data

[MICRO'23, MLSys'22] TorchSparse: Efficient Training and Inference Framework for Sparse Convolution on GPUs.

[ICLR'23 Spotlight?] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"

A super fast Graph Database uses GraphBLAS under the hood for its sparse adjacency matrix graph representation. Our goal is to provide the best Knowledge Graph for LLM (GraphRAG).

PyTorch Extension Library of Optimized Autograd Sparse Matrix Operations

Programmable CUDA/C++ GPU Graph Analytics

Library for specialized dense and sparse matrix operations, and deep learning primitives.

SPLADE: sparse neural search (SIGIR21, SIGIR22)

VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking (CVPR 2023)

From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

Awesome Multitask Learning Resources

SpargeAttention: A training-free sparse attention that can accelerate any model inference.

ICRA 2019 "Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera"

[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding

Hierarchical unsupervised and semi-supervised topic models for sparse count data with CorEx

A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible.

[CVPR 2022] SparseInst: Sparse Instance Activation for Real-Time Instance Segmentation