Popular GitHub Repositories for Ingest

Parse files (e.g. code repos) and websites to clipboard or a file for ingestions by AI / LLMs

Python based Open Source ETL tools for file crawling, document processing (text extraction, OCR), content analysis (Entity Extraction & Named Entity Recognition) & data enrichment (annotation) pipelines & ingestor to Solr or Elastic search index & linked data graph database

? A extension can help you open git ingest to turn any git repository into a prompt-friendly text ingest for LLMs.

The goal is to parse,process and ingest codebase/s to form a pluggable knowledge Graph and become the unified code context provider

GraphRAG for Second Brain. Ingest knowledge -> build knowledge graphs -> Query relevant knowledge | Explore connections

Template for Retreival-Augmented Generation (RAG) application using Milvus for vector storage, LangGraph for ReAct agentic workflows, and Streamlit for a front end. Runs locally on Ollama with a pipeline to ingest markdown documentation.

Llama2 + Supabase powered tool to ingest and chat with github documents

RAG-Ingest: A tool for converting PDFs to markdown and indexing them for enhanced Retrieval Augmented Generation (RAG) capabilities.

Elevate your data apps with seamless, production-ready chat integration. Effortlessly ingest, personalize, and deploy using just a single line of code.

ETL data pipeline with Python, dbt, Dagster, DuckDB, Apache Kafka, and PySpark to ingest, extract, transform and load stock data in real-time.

Manage ingesting mangas, placing them in the right places and merging multiple sources together, while also enabling automatic upscaling of said mangas.

RAG-powered document/url Q&A system with ChromaDB + Groq LLM. Upload docs, ingest URLs, get AI answers.

Self-hosted RAG prototype to ingest PDFs/HTML and chat with them via a local UI

This is the backend for a RAG system that runs on Docker Compose. It registers documents in a wide range of file formats, which can be searched using the MCP server.

Project is created by "AI/ML Clarity" newsletter author for RAG demo purposes. It allows users to ingest documents and perform semantic search and summarization using vector databases. Project also includes console-based chat with the RAG-powered LLM from Azure Open AI.

This project aims to build an end-to-end loan default prediction system for a Non-Banking Financial Institution (NBFI). The system is designed to ingest, clean, process, and predict loan default probabilities while ensuring model deployment, monitoring, and automated CI/CD.