Popular GitHub Repositories for Massively Parallel

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

A hyperparameter optimization framework

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Massively Parallel Deep Reinforcement Learning. ?

:stuck_out_tongue_closed_eyes: TensorFlowTTS: Real-Time State-of-the-art Speech Synthesis for Tensorflow 2 (supported including English, French, Korean, Chinese, German and Easy to adapt for other languages)

Fast inference engine for Transformer models

Massively parallel rigidbody physics simulation on accelerator hardware.

MuseV: Infinite-length and High Fidelity Virtual Human Video Generation with Visual Conditioned Parallel Denoising

Parallel Computing and Scientific Machine Learning (SciML): Methods and Applications (MIT 18.337J/6.338J)

GenAI Processors is a lightweight Python library that enables efficient, parallel content processing.

[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling

Unofficial Parallel WaveGAN (+ MelGAN & Multi-band MelGAN & HiFi-GAN & StyleMelGAN) with Pytorch

Must read research papers and links to tools and datasets that are related to using machine learning for compilers and systems optimisation

A fast, ergonomic and portable tensor library in Nim with a deep learning focus for CPU, GPU and embedded devices via OpenMP, Cuda and OpenCL backends

The PyTorch Implementation based on YOLOv4 of the paper: "Complex-YOLO: Real-time 3D Object Detection on Point Clouds"

:chart_with_upwards_trend: Adaptive: parallel active learning of mathematical functions

C++-based high-performance parallel environment execution engine (vectorized env) for general RL environments.