Popular GitHub Repositories for Spiders

A Machine Learning Project implemented from scratch which involves web scraping, data engineering, exploratory data analysis and machine learning to predict housing prices in New York Tri-State Area.

Automated web scraping spider generation using Browser Use and LLMs. Streamline the creation of Playwright-based spiders with minimal manual coding. Ideal for large enterprises with recurring data extraction needs.

分布式爬虫系统

A products scraper, that uses Scrapy spiders to crawl a list of websites, gather relevant data from them, a data filtering script and a Spacy model for NER, that will be trained to detect products.