Popular GitHub Repositories for Cleaning

The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

Refine high-quality datasets and visual AI models

文本挖掘和预处理工具（文本清洗、新词发现、情感分析、实体识别链接、关键词抽取、知识抽取、句法分析等），无监督或弱监督方法

A Python toolkit/library for reality-centric machine/deep learning and data mining on partially-observed time series, including SOTA neural network models for scientific analysis tasks of imputation/classification/clustering/forecasting/anomaly detection/cleaning on incomplete industrial (irregularly-sampled) multivariate TS with NaN missing values

Prepping tables for machine learning

R usage tips for data cleaning, data visualisation, statistical modelling and machine learning

The toolkit to test, validate, and evaluate your models and surface, curate, and prioritize the most valuable data for labeling.

?️ A book and repo to get you started programming voice computing applications in Python (10 chapters and 200+ scripts).

Code used for sourcing and cleaning the BigScience ROOTS corpus

LLM-based text extraction from unstructured data like PDFs, Words and HTMLs. Transform and cluster the text into your desired format. Less information loss, more interpretation, and faster R&D!

? An automated machine learning framework for audio, text, image, video, or .CSV files (50+ featurizers and 15+ model trainers). Python 3.6 required.

Inspired by google c4, here is a series of colossal clean data cleaning scripts focused on CommonCrawl data processing. Including Chinese data processing and cleaning methods in MassiveText.

OpenDataVal: a Unified Benchmark for Data Valuation in Python (NeurIPS 2023)

An Extension for Automatic1111 Webui that helps cleaning up prompts

A curated list of awesome open source tools and commercial products for monitoring data quality, monitoring model performance, and profiling data ?

Semi-supervised tool for cleaning image dataset , using deep learning.

Simplify Your Visual Data Ops. Find and visualize issues with your computer vision datasets such as duplicates, anomalies, data leakage, mislabels and others.

Power up your data science workflow with ChatGPT.

Analyzing the Drugs Descriptions, conditions, reviews and then recommending it using Deep Learning Models, for each Health Condition of a Patient.

FP-Age: Leveraging Face Parsing Attention for Facial Age Estimation in the Wild