Popular GitHub Repositories for Late Chunking

? CHONK your texts with Chonkie ? - The no-nonsense RAG chunking library

NCRF++, a Neural Sequence Labeling Toolkit. Easy use to any sequence labeling tasks (e.g. NER, POS, Segmentation). It includes character LSTM/CNN, word LSTM/CNN and softmax/CRF components.

? RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with PostgreSQL or SQLite

Use late-interaction multi-modal models such as ColPali in just a few lines of code.

Production-ready Inference, Ingestion and Indexing built in Rust ?

Late Interaction Models Training & Retrieval

Fully neural approach for text chunking

A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

A package for parsing PDFs and analyzing their content using LLMs.

The RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.

? CHONK your texts with Chonkie ✨ Type-friendly, light-weight, fast and super-simple chunking library

A TensorFlow implementation of Neural Sequence Labeling model, which is able to tackle sequence labeling tasks such as POS Tagging, Chunking, NER, Punctuation Restoration and etc.

? GUI for editing LLM vector embeddings. No more blind chunking. Upload content in any file extension, join and split chunks, edit metadata and embedding tokens + remove stop-words and punctuation with one click, add images, and download in .veml to share it with your team.

Completely local RAG. Chat with your PDF documents (with open LLM) and UI to that uses LangChain, Streamlit, Ollama (Llama 3.1), Qdrant and advanced methods like reranking and semantic chunking.

? semantic-chunking ? semantically create chunks from large document for passing to LLM workflows

a modular multimodal framework for ai applications

An LLM GUI application; enables you to interact with your files, offering dynamic parameters that can modify response behavior during runtime.

Postgres extensions to support end-to-end Retrieval-Augmented Generation (RAG) pipelines

smart-llm-loader is a lightweight yet powerful Python package that transforms any document into LLM-ready chunks. Spend less time on preprocessing headaches and more time building what matters. From RAG systems to chatbots to document Q&A, SmartLLMLoader handles the heavy lifting so you can focus on creating exceptional AI applications.

Implementation of the dynamic chunking mechanism in H-net by Hwang et al. of Carnegie Mellon