Popular GitHub Repositories for Black Box Benchmarking

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.

BoxMOT: pluggable SOTA tracking modules for segmentation, object detection and pose estimation models

An adversarial example library for constructing attacks, building defenses, and benchmarking both

[ECCV 2022] ByteTrack: Multi-Object Tracking by Associating Every Detection Box

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including OpenAI Agents SDK, CrewAI, Langchain, Autogen, AG2, and CamelAI

Merlion: A Machine Learning Framework for Time Series Intelligence

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

A library for prototyping realtime hand detection (bounding box), directly in the browser.

Deep learning software for colorizing black and white images with a few clicks.

Just draw a bounding box and you can remove the object you want to remove.

Repository for benchmarking graph neural networks (JMLR 2023)

Collection of popular and reproducible image denoising works.

基于图像识别的自动化黑盒测试框架 | An automation black-box testing framework based on image recognition

[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

?? Build Your Own chatgpt Bot|? Discord/Slack/Kook/Telegram |? ToolCall|? Plugin Support | ? out-of-box | gpt-4o

Unifying Variational Autoencoder (VAE) implementations in Pytorch (NeurIPS 2022)

OneDiff: An out-of-the-box acceleration library for diffusion models.

A streamlined and customizable framework for efficient large model evaluation and performance benchmarking

MobileNetV1, MobileNetV2, VGG based SSD/SSD-lite implementation in Pytorch 1.0 / Pytorch 0.4. Out-of-box support for retraining on Open Images dataset. ONNX and Caffe2 support. Experiment Ideas like CoordConv.

[ACL 2024 ?] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.