Popular GitHub Repositories for Opensearch Ingestion

File Parser optimised for LLM Ingestion with no loss ? Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.

Scalable Multi-modal RAG platform

LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.

Specify a github or local repo, github pull request, arXiv or Sci-Hub paper, Youtube transcript or documentation URL on the web and scrape into a text file and clipboard for easier LLM ingestion

A modular and comprehensive solution to deploy a Multi-LLM and Multi-RAG powered chatbot (Amazon Bedrock, Anthropic, HuggingFace, OpenAI, Meta, AI21, Cohere, Mistral) using AWS CDK on AWS

Jcseg is a light weight NLP framework developed with Java. Provide CJK and English segmentation based on MMSEG algorithm, With also keywords extraction, key sentence extraction, summary extraction implemented based on TEXTRANK algorithm. Jcseg had a build-in http server and search modules for lucene,solr,elasticsearch,opensearch

Production-ready Inference, Ingestion and Indexing built in Rust ?

? Sycamore is an LLM-powered search and analytics platform for unstructured data.

AWS Generative AI CDK Constructs are sample implementations of AWS CDK for common generative AI patterns.

? Open Source App Framework to build streaming apps with real-time data - ? Build real-time data pipelines and make real-time data universally accessible - ? Join historical and real-time data in the stream to create smarter ML and AI applications. - ? Standardize complex data ingestion and stream data to apps with pre-built connectors

The modular, open-source backend for building AI-native software — powered by knowledge, not static data.

A Python library that enables ML teams to share, load, and transform data in a collaborative, flexible, and efficient way :chestnut:

Python based Open Source ETL tools for file crawling, document processing (text extraction, OCR), content analysis (Entity Extraction & Named Entity Recognition) & data enrichment (annotation) pipelines & ingestor to Solr or Elastic search index & linked data graph database

Python toolkit for building GraphRAG applications

RAG with langchain using Amazon Bedrock and Amazon OpenSearch

Amazon Bedrock Foundation models with Amazon Opensearch Serverless as a Vector DB

? A extension can help you open git ingest to turn any git repository into a prompt-friendly text ingest for LLMs.

Search with BERT vectors in Solr, Elasticsearch, OpenSearch and GSI APU

A solution guidance for Generative BI using Amazon Bedrock, Amazon OpenSearch with RAG

Fully managed RAG solution implemented using Knowledge Bases for Amazon Bedrock