Popular GitHub Repositories for Dataproc Clusters

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

Simple, scalable AI model deployment on GPU clusters

prima.cpp: Speeding up 70B-scale LLM inference on low-resource everyday home clusters

Enhances construction site safety using YOLO for object detection, identifying hazards like workers without helmets or safety vests, and proximity to machinery or vehicles. HDBSCAN clusters safety cone coordinates to create monitored zones. Post-processing algorithms improve detection accuracy.

An open-source toolkit for deploying and managing high performance clusters for HPC, AI, and data analytics workloads.

BERT, AWS RDS, AWS Forecast, EMR Spark Cluster, Hive, Serverless, Google Assistant + Raspberry Pi, Infrared, Google Cloud Platform Natural Language, Anomaly detection, Tensorflow, Mathematics

Efficient LLM inference on Slurm clusters using vLLM.

Create, List, Update, Delete Amazon EKS clusters. Deploy and manage software on EKS. Run distributed model training and inference examples.

dask-pytorch-ddp is a Python package that makes it easy to train PyTorch models on dask clusters using distributed data parallel.

Inference in Bayesian Belief Networks using Probability Propagation in Trees of Clusters (PPTC) and Gibbs sampling

This repository aims to showcase how to finetune a FM model in Amazon EKS cluster using, JupyterHub to provision notebooks and craft both serving and training scripts, RayOperator to manage Ray Clusters and Karpenter to manage Node Scaling.

An application using Machine Learning to automatically create clusters of queries from Google Search Console.

TwinGraph is a Python framework for distributed container orchestration using Kubernetes clusters, Docker Compose/Swarm or cloud resources on AWS (AWS Lambda, AWS Batch, Amazon EKS). Applications include high-throughput simulations, simulation-driven optimization, Digital Twins and machine learning.

Interactive tree-maps with SBERT & Hierarchical Clustering (HAC)

In this work, we propose a deterministic version of Local Interpretable Model Agnostic Explanations (LIME) and the experimental results on three different medical datasets shows the superiority for Deterministic Local Interpretable Model-Agnostic Explanations (DLIME).

[ICML 2024] Serving LLMs on heterogeneous decentralized clusters.

High performance implementation of Deep neuroevolution in pytorch using mpi4py. Intended for use on HPC clusters

Sort similar looking faces to clusters

Clustering a set of word/tags using K-Means with word2vec or wordnet distance

An unofficial implementation using Pytorch for "Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types". Improve the algorithm with DINO pretrained ViT. Implement algorithms based on PatchCore.