Popular GitHub Repositories for Factuality

Loki: Open-source solution designed to automate the process of verifying factuality

FacTool: Factuality Detection in Generative AI

Benchmarking long-form factuality in large language models. Original code for our paper "Long-form factuality in large language models".

Official implementation for the paper "DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models"

RefChecker provides automatic checking pipeline and benchmark dataset for detecting fine-grained hallucinations generated by Large Language Models.

A package to evaluate factuality of long-form generation. Original implementation of our EMNLP 2023 paper "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation"

open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality

Code for the EMNLP 2024 paper "Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps"

[EMNLP'24] RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models

Code for the arXiv paper: "LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond"

This AI fact-checking system, built with LangGraph, dissects text into verifiable claims, cross-referencing them with real-world evidence via web searches. It then generates detailed accuracy reports, ideal for combating misinformation in LLM outputs, news, or any text.

This AI fact-checking system, built with LangGraph, dissects text into verifiable claims, cross-referencing them with real-world evidence via web searches. It then generates detailed accuracy reports, ideal for combating misinformation in LLM outputs, news, or any text.

The implementation for EMNLP 2023 paper ”Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators“

This AI fact-checking system, built with LangGraph, dissects text into verifiable claims, cross-referencing them with real-world evidence via web searches. It then generates detailed accuracy reports, ideal for combating misinformation in LLM outputs, news, or any text.

SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Model https://arxiv.org/pdf/2411.02433

Code for paper "Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators"

Code and data for the Dreyer et al (2023) paper on abstractiveness and factuality in abstractive summarization

A lightweight, agent-style framework for fact-checking atomic claims using iterative retrieval and verification. Reduces LLM and search cost while maintaining strong factuality performance.

A retrieval-augmented generation-based framework for automatically constructing a hierarchy of aspects typically considered when addressing a nuanced claim and enriching them with corpus-specific perspectives.

CLI tool to evaluate LLM factuality on MMLU benchmark.