Reinforcement learning (RL) is a key method for fine-tuning large language models (LLMs) in complex reasoning tasks. However, a common problem with RL is exploration collapse, where the model focuses on a narrow set of dominant strategies. This results in high performance on initial accuracy (pass@1) but limits diversity in the solutions it generates. The authors argue that traditional approaches tend to reward local behaviors instead of promoting a variety of distinct solutions, which stifles creative problem-solving. To tackle this issue, the authors introduce Uniqueness-Aware Reinforcement Learning. This method emphasizes rewarding solutions that apply rare, effective strategies, rather than just those that are frequently used. They achieve this by developing a judge mechanism based on LLMs that organizes potential solutions into clusters according to their underlying strategies. The model then adjusts the reward structure so that correct solutions with fewer occurrences in the clusters receive higher rewards. This approach encourages the generation of novel solutions, enhancing the overall diversity of outcomes. The authors validate their method across various benchmarks in mathematics, physics, and medical reasoning. They report consistent improvements in pass@$k$ metrics, demonstrating that their approach not only boosts the overall performance but also increases the area under the pass@$k$ curve (AUC@$K$). Notably, this improvement occurs without sacrificing the initial accuracy (pass@1). The results indicate that their framework allows for broader exploration and discovery of unique solution strategies, which can be particularly beneficial in fields requiring creative thinking.