Popular GitHub Repositories for Sparse Matrices

Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors

A package for the sparse identification of nonlinear dynamical systems from data

[MICRO'23, MLSys'22] TorchSparse: Efficient Training and Inference Framework for Sparse Convolution on GPUs.

[ICLR'23 Spotlight?] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"

A super fast Graph Database uses GraphBLAS under the hood for its sparse adjacency matrix graph representation. Our goal is to provide the best Knowledge Graph for LLM (GraphRAG).

PyTorch Extension Library of Optimized Autograd Sparse Matrix Operations

Programmable CUDA/C++ GPU Graph Analytics

Library for specialized dense and sparse matrix operations, and deep learning primitives.

SPLADE: sparse neural search (SIGIR21, SIGIR22)

VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking (CVPR 2023)

From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

SpargeAttention: A training-free sparse attention that can accelerate any model inference.

Awesome Multitask Learning Resources

ICRA 2019 "Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera"

Hierarchical unsupervised and semi-supervised topic models for sparse count data with CorEx

[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding

A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible.

[CVPR 2022] SparseInst: Sparse Instance Activation for Real-Time Instance Segmentation