cc-pyspark

INFO-SPIDER 是一个集众多数据源于一身的爬虫工具箱?，旨在安全快捷的帮助用户拿回自己的数据，工具代码开源，流程透明。支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。

the portable Python dataframe library

Simple and Distributed Machine Learning

State of the Art Natural Language Processing

Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.

:jack_o_lantern: 常用正则表达式 - 收集一些在平时项目开发中经常用到的正则表达式。

AnyCrawl ?: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.

news-please - an integrated web crawler and information extractor for news that just works

Crawl a website starting from a URL, find relevant pages, and extract data – all guided by your natural language prompt.

Implementing best practices for PySpark ETL jobs and applications.