Popular GitHub Repositories for Apache Kafka

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ?Docker-friendly.?Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

Open source platform for the machine learning lifecycle

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/

Simple and Distributed Machine Learning

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

PipelineAI

The ops and reference implementation for Apache Dubbo.

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

中文 NLP 预处理、解析工具包，准确、高效、易用 A Chinese NLP Preprocessing & Parsing Package www.jionlp.com

Alink is the Machine Learning algorithm platform based on Flink, developed by the PAI team of Alibaba computing platform.

BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray

RF-DETR is a real-time object detection model architecture developed by Roboflow, released under the Apache 2.0 license.

TransmogrifAI (pronounced tr?ns-m?g?r?-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

TFX is an end-to-end platform for deploying production ML pipelines

Elyra extends JupyterLab with an AI centric approach.

A toolkit to run Ray applications on Kubernetes

Feathr – A scalable, unified data and AI engineering platform for enterprise

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.