Popular GitHub Repositories for Video Captioning

Auto transcribe tool based on whisper

A new multi-shot video understanding benchmark Shot2Story with comprehensive video summaries and detailed shot-level captions.

这是一个基于Pytorch平台、Transformer框架实现的视频描述生成 (Video Captioning) 深度学习模型。 视频描述生成任务指的是：输入一个视频，输出一句描述整个视频内容的文字（前提是视频较短且可以用一句话来描述）。本repo主要目的是帮助视力障碍者欣赏网络视频、感知周围环境，促进“无障碍视频”的发展。

CrossCLR: Cross-modal Contrastive Learning For Multi-modal Video Representations, ICCV 2021

Attention Bidirectional Video Recurrent Net

? Video Captioning: ICCV '15 paper implementation

MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models (CVPR 2023)

This is the project webpage for 'Any2Caption'.

AI VTuber with LLM, ASR, TTS, OCR, CV and more technologies to live stream or play Minecraft with you.

Deep learning works for ADLxMLDS (CSIE 5431) in NTU

Interactive multimedia captioning with Keras

Source Code for Captionomaly: A Deep Learning Toolbox for Anomaly Captioning in Surveillance Videos

text-only training or language-free training for multimodal tasks (image/audio/video caption, retrieval, text2image)

[ACM MMGR '24] ? Shotluck Holmes: A family of small-scale LLVMs for shot-level video understanding

Aggregation framework for annotating datasets in computer vision tasks (detection, segmentation, video captioning etc.)

To build attention based encoder-decoder model for video captioning on the MSVD dataset

Video Captioning using Scene Change Detection and Image Captioning

[ACM Multimedia 2024] Observe before Generate: Emotion-Cause aware Video Caption for Multimodal Emotion Cause Generation in Conversations