AIbase
Product LibraryTool NavigationMCP

LLMWebCrawler

Public

A Web Crawler based on LLMs implemented with Ray and Huggingface. The embeddings are saved into a vector database for fast clustering and retrieval. Use it for your RAG.

Creat2023-09-28T17:52:05
Update2025-03-14T05:25:50
95
Stars
1
Stars Increase

Related projects