Popular GitHub Repositories for Rainforest Scraper

Python scraper based on AI

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Use LLMs to dig out what you care about from massive amounts of information and a variety of sources daily.

Turn any webpage into structured data using LLMs

A community-driven way to read and chat with AI bots - powered by chatGPT.

Easily collect responses from ChatGPT scraper by providing a prompt along with valid Web Scraper API credentials.

Oxylabs AI Studio python SDK

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Automate webpages at scale, scrape web data completely and accurately with high performance, distributed AI-RPA.

Browser4: a lightning-fast, coroutine-safe browser for your AI.

A multithreaded ?? web crawler that recursively crawls a website and creates a ? markdown file for each page, designed for LLM RAG

Monitor and download Manga automatically in a Docker-Container.

Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.

The GPT-based Universal Web Scraper MVP is a solution that leverages GPT models and web scraping libraries to generate scraper code based on user input and website analysis, simplifying the web scraping process.

Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.

Lego AI Parser is an open-source application that uses OpenAI to parse visible text of HTML elements.

ScraperAI is an open-source, AI-powered tool designed to simplify web scraping for users of all skill levels.

Developed an AI application using LLM to analyze user resumes and provided the summarization, strengths, weaknesses, suggestions, suitable job titles, and also scraping job details from LinkedIn using Selenium. This application reduces time by 30% and helps candidates tailor their resumes effectively.

Unofficial Claude API supporting direct HTTP chat creation/deletion/retrieval, messages with multiple file attachments and auto session gathering using Firefox with geckodriver.

Financial Web Scraper & Sentiment Classifier