Popular GitHub Repositories for Pyspark Mllib

Simple and Distributed Machine Learning

State of the Art Natural Language Processing

A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Hopsworks - Data-Intensive AI platform with a Feature Store

Starter code to solve real world text data problems. Includes: Gensim Word2Vec, phrase embeddings, Text Classification with Logistic Regression, word count with pyspark, simple text preprocessing, pre-trained embeddings and more.

PySpark + Scikit-learn = Sparkit-learn

CTR prediction model based on spark(LR, GBDT, DNN)

A Comprehensive Framework for Building End-to-End Recommendation Systems with State-of-the-Art Models

Distributed scikit-learn meta-estimators in PySpark

MorphL Community Edition uses big data and machine learning to predict user behaviors in digital products and services with the end goal of increasing KPIs (click-through rates, conversion rates, etc.) through personalization

Isolation Forest on Spark

Toolkit for Apache Spark ML for Feature clean-up, feature Importance calculation suite, Information Gain selection, Distributed SMOTE, Model selection and training, Hyper parameter optimization and selection, Model interprability.

Build reliable AI and agentic applications with DataFrames

商品类目预测，使用 Spring Boot 开发框架和 Spark MLlib 机器学习框架，通过 TF-IDF 和 Bayes 算法，训练出一个商品类目预测模型。该模型可以根据商品名称自动预测出商品类目。项目对外提供 RESTFul 接口。

:star2: :sparkles: Analyze and visualize Twitter Sentiment on a world map using Spark MLlib

BERT, AWS RDS, AWS Forecast, EMR Spark Cluster, Hive, Serverless, Google Assistant + Raspberry Pi, Infrared, Google Cloud Platform Natural Language, Anomaly detection, Tensorflow, Mathematics

Relation Extraction using Deep learning(CNN)

Spark 2.0 Python Machine Learning examples