Popular GitHub Repositories for Kafka Brokers

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. ?Docker-friendly.?Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

PipelineAI

Alink is the Machine Learning algorithm platform based on Flink, developed by the PAI team of Alibaba computing platform.

Trench — Open-Source Analytics Infrastructure. A single production-ready Docker image built on ClickHouse, Kafka, and Node.js for tracking events, page views. Easily build product analytics dashboards, LLM RAGs, observability platforms, or any other analytics product.

Machine Learning Platform and Recommendation Engine built on Kubernetes

Metering and Billing for AI, API and DevOps. Collect and aggregate millions of usage events in real-time and enable usage-based billing.

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

This project contains examples which demonstrate how to deploy analytic models to mission-critical, scalable production environments leveraging Apache Kafka and its Streams API. Models are built with Python, H2O, TensorFlow, Keras, DeepLearning4 and other technologies.

In this repository, I have developed the entire server-side principal architecture for real-time stock market prediction with Machine Learning. I have used Tensorflow.js for constructing ml model architecture, and Kafka for real-time data streaming and pipelining.

Modular e-commerce backend with a GraphQL gateway and gRPC microservices for accounts, products, orders, and recommendations.

LangStream. Event-Driven Developer Platform for Building and Running LLM AI Apps. Powered by Kubernetes and Kafka.

Real Time Big Data / IoT Machine Learning (Model Training and Inference) with HiveMQ (MQTT), TensorFlow IO and Apache Kafka - no additional data store like S3, HDFS or Spark required

? Open Source App Framework to build streaming apps with real-time data - ? Build real-time data pipelines and make real-time data universally accessible - ? Join historical and real-time data in the stream to create smarter ML and AI applications. - ? Standardize complex data ingestion and stream data to apps with pre-built connectors

Kafka Streams + Java + gRPC + TensorFlow Serving => Stream Processing combined with RPC / Request-Response

Tangseng search engine including full text search and vector search base on golang. 基于go语言的搜索引擎，信息检索系统

Repo that relates to the Medium blog 'Keeping your ML model in shape with Kafka, Airflow' and?MLFlow'

Oracle database CDC (Change Data Capture)

Examples for Deep Learning/Feature Store/Spark/Flink/Hive/Kafka jobs and Jupyter notebooks on Hops

Using Kafka-Python to illustrate a ML production pipeline