Popular GitHub Repositories for Parquet

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

GigAPI：基于DuckDB的Parquet存储引擎和OLAP数据池

Comprehensive Vector Data Tooling. The universal interface for all vector database, datasets and RAG platforms. Easily export, import, backup, re-embed (using any model) or access your vector data from any vector databases or repository.

?? RumbleDB 1.22.0 "Pyrenean oak" ? for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

Firebolt Core is a free, self-hosted edition of Firebolt's distributed query engine (https://www.firebolt.io/); it provides high-performance data warehousing capabilities that can be deployed anywhere from a single laptop to enterprise datacenters.

Parquet-based ML data format optimized for working with unstructured data

用于可视化和分析 Parquet 文件的交互式 Python 命令行界面

Icebird: JavaScript Iceberg Client

Hyperparam local dataset viewer

Generate massive fake datasets for your datalake, fast. By SOMA

A real-time data cleaning pipeline for medical and healthcare data using Apache Spark, SparkNLP, Spark Streaming, and Kafka.

Python COTTAS library for compressing and querying RDF

Contextual Multi-Armed Bandit Reward Tracker & Model Trainer

Supercharge Your Compute for Analytics & AI

ParquetToHuggingFace processes raw audio data, converts it into Parquet files, and uploads them to Hugging Face. The README explains how to set up the environment, configure paths, and run the scripts to generate and upload the data.

将你的音频文件转换成 DuckDB 或 Parquet 文件

MCP分析师是一个MCP服务器，它使Claude能够分析本地CSV或Parquet文件。