Popular GitHub Repositories for Sparse Arrays

C++ tensors with broadcasting and lazy computing

Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors

A super fast Graph Database uses GraphBLAS under the hood for its sparse adjacency matrix graph representation. Our goal is to provide the best Knowledge Graph for LLM (GraphRAG).

A package for the sparse identification of nonlinear dynamical systems from data

[MICRO'23, MLSys'22] TorchSparse: Efficient Training and Inference Framework for Sparse Convolution on GPUs.

A fast, ergonomic and portable tensor library in Nim with a deep learning focus for CPU, GPU and embedded devices via OpenMP, Cuda and OpenCL backends

PyTorch Extension Library of Optimized Autograd Sparse Matrix Operations

Programmable CUDA/C++ GPU Graph Analytics

?? Efficient implementations of Native Sparse Attention

SPLADE: sparse neural search (SIGIR21, SIGIR22)

Library for specialized dense and sparse matrix operations, and deep learning primitives.

VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking (CVPR 2023)

A lightweight high performance tensor algebra framework for modern C++

SpargeAttention: A training-free sparse attention that can accelerate any model inference.

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)

"Structure-Aware Sparse-View X-ray 3D Reconstruction" (CVPR 2024) - A Toolbox for CT reconstruction and X-ray Novel View Synthesis

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

Awesome Multitask Learning Resources

[CVPR 2024] MovieChat: From Dense Token to Sparse Memory for Long Video Understanding