Recently, research teams from Stanford University and West Virginia University have proposed a method called "Verbalized Sampling" (VS), aimed at enhancing the creative diversity of generative AI models. The study shows that by adding a simple sentence to the prompt: "Generate five responses and their corresponding probabilities, sampling from the full distribution," large language models (LLMs) and image generation models can demonstrate more diverse creativity in their outputs.
When generating content, generative AI models typically select the next information unit (token) based on predictions. This means that when answering questions like "What is the capital of France?", the model will choose "Paris" as the answer from the probability distribution. However, many users find that AI outputs often seem repetitive and monotonous. This phenomenon is known as mode collapse, which limits the potential of the model, especially in areas such as creative writing, communication, strategy, and illustrations.
The VS method restores the rich diversity of the model's original pre-training by allowing the model to display a set of possible responses and their relative probabilities. Test results from the research team show that in creative writing, VS significantly improves the diversity of outputs while maintaining quality. When simulating conversations, models using VS are better able to mimic human thought changes and emotional fluctuations. In open-ended question-answering tasks, the answers generated by the model are closer to real-world data, showing greater diversity.
This method not only achieves significant results in output diversity but also allows users to adjust the diversity of the generated content by setting probability thresholds. Users can adjust the sampling threshold according to their needs to achieve more creative outputs. The implementation of VS is simple, does not require retraining the model, and supports various large language models, demonstrating its wide application potential.
Key Points:
🌟 A research team has proposed the "Verbalized Sampling" method, which significantly enhances the output diversity of generative AI through a simple prompt.
✍️ Using the VS method, AI demonstrates more human-like richness in tasks such as creative writing and conversation simulation.
🚀 Users can further control the diversity of outputs by adjusting probability thresholds, making it simple and easy to use without requiring model retraining.
