Popular GitHub Repositories for Optimizer

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

AdalFlow: The library to build & auto-optimize LLM applications.

torch-optimizer -- collection of optimizers for Pytorch

? Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

Adlik: Toolkit for Accelerating Deep Learning Inference

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.

Visualize Tensorflow's optimizers.

Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.

Awesome-LLM-Prompt-Optimization: a curated list of advanced prompt optimization and tuning methods in Large Language Models

Deep learning library in plain Numpy.

syftr is an agent optimizer that helps you find the best agentic workflows for your budget.

A very fast, very minimal prompt optimizer

Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch

APOLLO: SGD-like Memory, AdamW-level Performance

Must-read Papers on Large Language Model (LLM) as Optimizers and Automatic Optimization for Prompting LLMs.

Explore the energy-efficient dataflow scheduling for neural networks.

Q-Learning Based Cryptocurrency Trader and Portfolio Optimizer for the Poloniex Exchange