llm-batch

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

A high-throughput and memory-efficient inference and serving engine for LLMs

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

? Serverless Framework – Effortlessly build apps that auto-scale, incur zero costs when idle, and require minimal maintenance using AWS Lambda and other managed cloud services.

Port of OpenAI's Whisper model in C/C++

Making large AI models cheaper, faster and more accessible

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A generative speech model for daily dialogue.

Cross-platform, customizable ML solutions for live and streaming media.