Popular GitHub Repositories for Data Pipelines

Fair-code workflow automation platform with native AI capabilities. Combine visual building with custom code, self-host or cloud, 400+ integrations.

For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.

Deep Learning for humans

1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

? Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

LlamaIndex is the leading framework for building LLM-powered agents over your data.

Learn how to design, develop, deploy and iterate on production-grade ML applications.

Making large AI models cheaper, faster and more accessible

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Build and share delightful machine learning apps, all in Python. ? Star to support our work!

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

AI's query engine - Platform for building AI that can learn and answer questions over large scale federated data.

Collection of various algorithms in mathematics, machine learning, computer science and physics implemented in C++ for educational purposes.

? Industrial-strength Natural Language Processing (NLP) in Python

Easily train a good VC model with voice data <= 10 mins!

Roadmap to becoming an Artificial Intelligence Expert in 2022

Code and documentation to train Stanford's Alpaca models, and generate the data.

Pretrain, finetune ANY AI model of ANY size on multiple GPUs, TPUs with zero code changes.

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.