Popular GitHub Repositories for Activations

This project extends the idea of the innovative architecture of Kolmogorov-Arnold Networks (KAN) to the Convolutional Layers, changing the classic linear transformation of the convolution to learnable non linear activations in each pixel.

Compare SELUs (scaled exponential linear units) with other activations on MNIST, CIFAR10, etc.

This is an open-source version of the representation engineering framework for stopping harmful outputs or hallucinations on the level of activations. 100% free, self-hosted and open-source.

RNN and general weights, gradients, & activations visualization in Keras & TensorFlow

Code accompanying the paper "Massive Activations in Large Language Models"

Quantize weights and activations in Recurrent Neural Networks.

Twitch Streamer GPT is a NodeJS-based Twitch enhancement tool, offering interactive stream experiences with AI-powered automated responses, voice command activations, and advanced modules. It's easy to set up and suited for users of all tech levels.

Compression schema for gradients of activations in backward pass

Visualising what each LSTM cell learns from data.

A tool to visualize convolutional layer activations on an input image.

Quantization is a technique to reduce the computational and memory costs of running inference by representing the weights and activations with low-precision data types like 8-bit integer (int8) instead of the usual 32-bit floating point (float32).

This CNN-based language model leverages causal and dilated convolutions, gated activations, residual connections, and layer normalization to effectively model textual data for generation tasks.

A plug-and-play adapter architecture that efficiently adapts large language models to downstream tasks. Essential for client-server architecture language model services. A practical approach to adapting LLMs' hidden activations without modifying the base model.

Exploration of an alternative approach to extracting steering vectors. Instead of using the classical contrastive method we investigate whether comparing activations between a base model and its fine-tuned deceptive version reveals a more meaningful latent direction.

Very simple version of the code used for the experiments in the paper Hidden Activations Are Not Enough: A General Approach to Neural Networks Predictions.

Cairngorms SOTA activations award

Implementation of a Quantized Neural Network with low bitwidth of weights, activations and gradients.