cc-pyspark

INFO-SPIDER 是一个集众多数据源于一身的爬虫工具箱?，旨在安全快捷的帮助用户拿回自己的数据，工具代码开源，流程透明。支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。

the portable Python dataframe library

Simple and Distributed Machine Learning

State of the Art Natural Language Processing

Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.

:jack_o_lantern: 常用正则表达式 - 收集一些在平时项目开发中经常用到的正则表达式。

news-please - an integrated web crawler and information extractor for news that just works

Implementing best practices for PySpark ETL jobs and applications.

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

A curated list of awesome Apache Spark packages and resources.