pathway

Workflow Engine for Kubernetes

A curated list of awesome big data frameworks, ressources and other awesomeness.

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

An orchestration platform for the development, production, and observation of data assets.

Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.

Self-serve BI to 10x your data team ??

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

Build data pipelines, the easy way ?️

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

A system for agentic LLM-powered data processing and ETL