Popular GitHub Repositories for Latency Tester

A robust, efficient, low-latency speech-to-text library with advanced voice activity detection, wake word activation and instant transcription.

The world’s first real-time, distributed, cloud-edge collaborative multimodal AI Agent Framework that simultaneously supports C/C++/Go/Python/JS/TS

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

HelixDB is a powerful, graph-vector database built in Rust for millisecond query latency and ease of use.

Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database.

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

? Stateful Serverless Framework for Geo-distributed Edge AI Infra. with function calling support, write once, run on any model.

Aura is like Siri, but in your browser. An AI voice assistant optimized for low latency responses.

TEN VAD: low-latency high-performance Voice Activity Detector

The Qualcomm? AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm? devices.

Real-time inference for Stable Diffusion - 0.88s latency. Covers AITemplate, nvFuser, TensorRT, FlashAttention. Join our Discord communty: https://discord.com/invite/TgHXuSJEk6

[ICLR 2020] "FasterSeg: Searching for Faster Real-time Semantic Segmentation" by Wuyang Chen, Xinyu Gong, Xianming Liu, Qian Zhang, Yuan Li, Zhangyang Wang

Latency and Memory Analysis of Transformer Models for Training and Inference

Low latency JSON generation using LLMs ??

A low-latency & high-throughput serving engine for LLMs

A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

??Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM

Real-time GCC-NMF Blind Speech Separation and Enhancement

PyTorch Library for Low-Latency, High-Throughput Graph Learning on GPUs.

Low latency ai companion voice talk in 60 lines of code using faster_whisper and elevenlabs input streaming