Popular GitHub Repositories for Apache Airflow

Open source platform for the machine learning lifecycle

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/

Simple and Distributed Machine Learning

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

PipelineAI

The ops and reference implementation for Apache Dubbo.

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

中文 NLP 预处理、解析工具包，准确、高效、易用 A Chinese NLP Preprocessing & Parsing Package www.jionlp.com

BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray

RF-DETR is a real-time object detection model architecture developed by Roboflow, released under the Apache 2.0 license.

TransmogrifAI (pronounced tr?ns-m?g?r?-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

TFX is an end-to-end platform for deploying production ML pipelines

Elyra extends JupyterLab with an AI centric approach.

A toolkit to run Ray applications on Kubernetes

Feathr – A scalable, unified data and AI engineering platform for enterprise

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Apache OpenNLP

PySpark + Scikit-learn = Sparkit-learn