Patchscope is a unified framework for probing the hidden representations of large language models (LLMs). It enables the interpretation of model behavior and the validation of its alignment with human values. By leveraging the model's own capacity to generate human-understandable text, we propose utilizing the model itself to explain its internal natural language representations. We demonstrate how the Patchscope framework can be used to answer a wide range of research questions about LLM computation. We show that prior interpretability methods based on projecting representations into the vocabulary space and intervening with LLM computation can be viewed as special instances of this framework. Furthermore, Patchscope opens new possibilities, such as using more powerful models to interpret the representations of smaller models and unlocking novel applications like self-correction and multi-hop reasoning.