Popular GitHub Repositories for Scrap

WebSage is an AI Engine that extracts content from any URL, generates summaries, and enables interaction using AI models. Choose between Closed Source (OpenAI) or Open Source (Ollama) for flexible summarization and conversation.

This LLM based reddit bot uses OpenAI API to scrap comments from given subreddits posts links and allow the admin to have a QnA session in the context of whole comments of that specific post. Build in python using pineconeDB and docker.

Multilingual library for scraping, preprocessing, topic modeling, and summarization. This is basic logic for Prepo service.

A project to scrap cybersecurity news articles from multiple websites and display them. Uses Gemini API for purification, summarization, FAQ and vector search to search articles. CRUD ops use Cosmocloud API.

Simple Job Offer Scraping tool basing on polish job offer site "pracuj.pl" using libraries like: Selenium, Pandas and Spacy. Written in Python 3.10

This project contains several scrappers to scrap images from various platforms like altcensored, civitai, haicut, kemono, leonardo, midjourney, openart, pinterest and etc

My GenAI assignment to scrap Roman Urdu Poetry from internet and then preprocess it. Implement a LSTM-based recurrent neural network for poetry generation. Integrated with Gradio for an interactive web-based interface.