LLM Strawberry

Source: https://github.com/hijkzzz/Awesome-LLM-Strawberry at 3f17fc9ee49c
License: Apache-2.0
Read the English original

This edition is an AI-assisted translation.

LLM Strawberry (OpenAI o1) 연구 자료

OpenAI Strawberry(o1) 및 추론에 관한 연구 논문과 블로그를 모았습니다. 이것은 연구 논문 및 블로그의 컬렉션입니다. OpenAI Strawberry(o1)와 리빙. LLM 추론의 최전선을 추적하기 위해 저장소를 계속 업데이트합니다. 그리고 저장소는 지속적으로 LLM Reasoning의 국경을 추적하기 위해 업데이트 될 것입니다.

OpenAI 문서

• https://platform.openai.com/docs/guides/reasoning
• 

(주)

• [영업] Introducing deep research
• [영업] o3 preview & o3 mini
• [영업] Introducing ChatGPT Pro
• [구글 DeepMind] Gemini 2.0 Flash Thinking
• [일상] AI with reasoning power will be less predictable
• [SemianAlysis] Scaling Laws – O1 Pro Architecture, Reasoning Training Infrastructure, Orion and Claude 3.5 Opus “Failures”
• [직접] DeepSeek-R1-Lite-Preview is now live: unleashing supercharged reasoning power!
• [문서] 数学对标o1系列，搜索再次进化，Kimi 新推理模型与你一起拓展智能边界
• [문서] Kimi 发布视觉思考模型 k1，多项理科测试行业领先
• [인터nLM] 强推理模型书生InternThinker开放体验：自主生成高智力密度数据、具备元动作思考能力
• [新智元] 万字独家爆光，首揭o1 pro架构！惊人反转，Claude 3.5 Opus没失败？

블로그

• [영업] Learning to Reason with LLMs
• [영업] OpenAI o1-mini Advancing cost-efficient reasoning
• [영업] Finding GPT-4’s mistakes with GPT-4
• [ARC-AGI] (주) OpenAI o3 Breakthrough High Score on ARC-AGI-Pub
• [공지] Building effective agents
• [히즈즈] Stabilizing MoE RL Without Router Replay: The Online IcePop Solution
• [히즈즈] REINFORCE++-baseline is all you need in RLVR
• [히즈즈] Exploring OpenAI O1 Model Replication
• [나단 Lambert] OpenAI’s Strawberry, LM self-talk, inference scaling laws, and spending more on inference
• [나단 Lambert] Reverse engineering OpenAI’s o1
• [Andreas Stuhlmüller, 정프테원] Supervise Process, not Outcomes
• [노하 Dziri] Have o1 Models Cracked Human Reasoning?
• [Rishabh 아가왈] Improving LLM Reasoning using Self-generated data: RL and Verifiers
• [웨이 Shen] Generalization Progress in RLHF: Insights into the Impact of Reward Models and PPO
• [도미나터069] Codeforces - Analyzing how good O1-Mini actually is
• [티버 블로호] Summary of what we have learned during AMA hour with the OpenAI o1 team

한국어

• [노암 브라운] Parables on the Power of Planning in AI: From Poker to Diplomacy
• [노암 브라운] OpenAI o1 and Teaching LLMs to Reason Better
• [본사] Don’t teach. Incentivize.

한국어

• [DeepLearning입니다. 사이트 맵 Reasoning with o1

사이트 맵

OpenAI 개발자

• All the questions addressed by the API team during the December 17, 2024 AMA
• We’re hosting an AMA for developers from 10–11 AM PT today.
• Today we previewed Reinforcement Fine-Tuning
• 

노암 브라운

• 
• 
• 
• 

제이슨 웨이

• 
• 
• 

이름 *

• 
• 
• 

공지사항

모델 번호:

• [Alibaba Qwen 팀] Qwen3
• [Alibaba Qwen 팀] QwQ
• [Alibaba Qwen 팀] QvQ
• [직접] DeepSeek R1
• [네비디아] Nemotron-Research-Reasoning-Qwen-1.5B
• [스카이워크] Skywork R1V2
• [rLLM] DeepScaler
• [나바스카] Sky-T1
• [GAIR-NLP] O1 Replication Journey: A Strategic Progress Report
• [OpenO1팀] Open-Source O1
• [요금] DRT-o1
• [아리바] Marco-o1
• [CUHK-SZ]의 특징 HuatuoGPT-o1

코드베이스

• [OpenRLHF 팀] OpenRLHF
• [OpenRLHF 팀] REINFORCE++ | REINFORCE++-baseline | Code
• [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] [NovaSky-AI] SkyRL
• [RUCAIBox] (주) STILL: Slow Thinking with LLMs
• [시간] Simple Reinforcement Learning for Reasoning
  • 이것은 DeepSeek-R1-Zero 및 DeepSeek-R1 교육의 복제는 제한된 데이터로 작은 모델에
• [유비량] Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
• [특징] Open-Reasoner-Zero
• [티드드라] LMM-R1
• [모듈] MM-EUREKA: Exploring Visual Aha Moment with Rule-based Large-scale Reinforcement Learning
• [R1-V 팀] R1-V
• [LLaMA 공장 팀] EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework
• [아리바] ROLL | Paper
• [바다 AI 연구소] Dr. GRPO
• [Berkeley AI 연구] TinyZero
• [미트릭스.org] LLM Reasoners

회사 소개

format:
- [title](paper link) [links]
  - author1, author2, and author3...
  - publisher
  - code
  - experimental environments and datasets

o1 모델에 대한 기술 보고서

• INTELLECT-3: Technical Report
  • 인텔리전트 팀
• DeepSeek V3.2
  • 딥스카이 AI
• MiMo-V2 Flash
  • 샤오 미
• Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
  • 다운로드
• Qwen3 Technical Report
  • Qwen 팀
• Scaling Agents via Continual Pre-training
  • Qwen 팀
• WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents
  • Qwen 팀
• Towards General Agentic Intelligence via Environment Scaling
  • Qwen 팀
• Magistral
  • 미스트랄 AI
• ERINE-4.5
  • 로드 중 …
• LongCat Flash
  • 운영 정보
• GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
  • 채용 정보
• Seed Thinking v1.5
  • Bytedance 씨앗
• DeepSeek-V3 Technical Report
  • 딥스카이
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
  • 딥스카이 AI
• DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
  • 딥스카이 AI
• MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
  • 미니 맥스
• Kimi k2: Open agentic intelligence
  • 문샷
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
  • 문샷
• KIMI-VL TECHNICAL REPORT
  • 문샷
• Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning
  • MoonShot & 누나
• Llama-Nemotron: Efficient Reasoning Models
  • 다운로드
• Skywork Open Reasoner 1 Technical Report
  • 관련 기사

2025

• Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
  • 다운로드
• QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory
  • Qwen 팀
• DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
  • 딥스카이-AI
• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices
  • Qwen 팀
• The Art of Scaling Reinforcement Learning Compute for LLMs
  • Devvrit Khatri, Lovish Madaan, Rishabh Tiwari, Rachit Bansal, Sai Surya Duvvuri, Manzil Zaheer, Inderjit S. Dhillon, 데이비드 Brandfonbrener, Rishabh Agarwal
• BroRL: Scaling Reinforcement Learning via Broadened Exploration
  • Jian Hu, Mingjie Liu, Ximing Lu, Fang Wu, Zaid Harchaoui, Shizhe Diao, Yejin Choi, Pavlo Molchanov, Jun Yang, Jan Kautz, 이 동
• Why Language Models Hallucinate
  • 오시는 길
• rStar2-Agent: Agentic Reasoning Technical Report
  • 닝보, Yifei Liu, 이 주, Li Lyna 장, Weijiang Xu, Xinyu Guan, Buze 장, Bingcheng Dong, Xudong Zhou, Bowen 장, Ying Xin, Ziming Miao, Scarlett Li, 팬 양, Mao 양
• RL Is Neither a Panacea Nor a Mirage: Understanding Supervised vs. Reinforcement Learning Fine-Tuning for LLMs
  • Hangzhan 진, Sicheng, Sifan 우, 모하메드 Hamdaqa
• DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization
  • Shuaijie 그녀, Yu Bao, Yu Lu, Lu Xu, Tao Li, Wenhao 주, Shujian Huang, Shanbo 청, Lu Lu, Yuxuan 왕
• Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning
  • Vaishnavi Shrivastava, Ahmed Awadallah, Vidhisha Balachandran, Shivam Garg, Harkirat Behl, Dimitris Papailiopoulos
• ProRL V2 - Prolonged Training Validates RL Scaling Laws
  • Jian Hu, Mingjie Liu, Shizhe Diao, Ximing Lu, Xin Dong, Pavlo Molchanov, Yejin Choi, Jan Kautz, 이동
• Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning
  • Zihe Liu, Jiashun Liu, Yancheng He, Weixun 왕, Jiaheng Liu, 링 팬, Xinyu Hu, Shaopan Xiong, Ju Huang, Jian Hu, Shengyi Huang, Siran 양, Jiamang 왕, Wenbo Su, Bo Zheng
• Group Sequence Policy Optimization
  • Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, 양, Jingren Zhou, Junyang Lin
• Your Efficient RL Framework Secretly Brings You Off-Policy RL Training
  • Feng Yao, Liyuan Liu, Dinghuai 장, Chengyu Dong, Jingbo Shang, Jianfeng Gao
• Gemini 2.5 Pro Capable of Winning Gold at IMO 2025
  • Yichen Huang, 린 F. 양
• DeepSWE: Training a Fully Open-sourced, State-of-the-Art Coding Agent by Scaling RL
  • 한국어
• OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
  • Zengzhi 왕, 팬 Zhou, Xuefeng Li, Pengfei Liu
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
  • Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, 이동
• REINFORCE++: An Efficient RLHF Algorithm with Robustness to Both Prompt and Reward Models
  • Jian Hu, Jason Klein Liu, 웨이 Shen
  • 이름: REINFORCE++-baseline
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
  • Qwen 팀
• The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
  • Xinyu 주, Mengzhou Xia, Zhepei Wei, Wei-Lin Chen, Danqi Chen, Yu Meng
• Thinking with Generated Images
  • Ethan Chern, Zhulin Hu, Steffi Chern, Siqi Kou, Jiadi Su, Yan Ma, Zhijie Deng, Pengfei Liu
• Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning
  • Alex Su, Haozhe 왕, Weiming Ren, Fangzhen 린, 원후 첸
• DeepEyes: Incentivizing “Thinking with Images” via Reinforcement Learning
  • Ziwei Zheng, 마이클 양, 잭 홍, Chenxiao Zhao, Guohai Xu, 르 양, 조 Shen, Xing Yu
• Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging
  • Shiqi Chen, Jinghan 장, Tongyao 주, 웨이 리우, Siyang Gao, Miao Xiong, Manling Li, Junxian 그는
• QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
  • Fanqi Wan, Weizhou Shen, Shengyi Liao, Yingcheng Shi, Chenliang Li, Ziyi 양, Ji 장, Fei Huang, Jingren Zhou, Ming Yan
• Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving
  • Xinji Mai, Haotian Xu, Xing W, Weinong 왕, Yingying 장, Wenqiang 장
  • 이름: https://github.com/yyht/openrlhf_async_pipline
• A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
  • Wei Xiong, Jiarui Yao, Yuhui Xu, Bo Pang, Lei Wang, Doyen Sahoo, Junnan Li, Nan Jiang, Tong 장, Caiming Xiong, Hanze Dong
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example
  • Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, Yelong Shen
• Process Reward Models That Think
  • Muhammad Khalifa, Rishabh Agarwal, Lajanugen Logeswaran, Jaekyeom Kim, Hao Peng, Moontae Lee, Honglak Lee, 루 왕
• M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
  • Junxiong 왕, Wen-Ding Li, Daniele Paliotta, Daniel Ritter, Alexander M. Rush, Tri Dao
• A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility
  • 안드레아스 Hochlehnert, Hardik Bhatnagar, Vishaal Udandarao, Samuel Albanie, Ameya Prabhu, Matthias Bethge
• Concise Reasoning via Reinforcement Learning
  • Mehdi Fatemi, Banafsheh Rafiee, Mingjie 탕, Kartik Talamadupula
• VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks
  • YuYue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian 팬, Zhengyin Du, Xiangpeng Wei, Gaohong Liu, Juncai Liu, Lingjun Liu, Haibin Lin, Zhiqi Lin, Bole Ma, 등.
• Inference-Time Scaling for Generalist Reward Modeling
  • Zijun Liu, Peiyi 왕, Runxin Xu, Shirong Ma, 충 Ruan, Peng Li, 양 Liu, Yu Wu
• JudgeLRM: Large Reasoning Models as a Judge
  • Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, Bingsheng 그는
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
  • Qiying Yu, Zheng 장, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian 팬, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, 등.
• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
  • Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, 톰 골드슈타인
• Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning
  • Yuxiao Qu, Matthew Y. R. Yang, Amrith Setlur, Lewis Tunstall, Edward Emanuel Beeching, Ruslan Salakhutdinov, Aviral 쿠마르
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
  • Huatong 노래, Jinhao 장, Yingqian 분, Jie 첸, Zhipeng 첸, 웨인 Xin Zhao, Lei Fang, Ji-Rong 원
• Visual-RFT: Visual Reinforcement Fine-Tuning
  • Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, Jiaqi 왕
• Introducing Visual Perception Token into Multimodal Large Language Model
  • Runpeng Yu, Xinyin Ma, Xinchao 왕
• Scaling Test-Time Compute Without Verification or RL is Suboptimal
  • Amrith Setlur, Nived Rajaraman, Sergey Levine, Aviral 쿠마르
• LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
  • Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Shishir G. Patil, Matei Zaharia, 조셉 E. Gonzalez, 이온 Stoica
• Demystifying Long Chain-of-Thought Reasoning in LLMs
  • 에드워드 여, Yuxuan 통, 모리스 Niu, Graham Neubig, Xiang Yue
• LIMR: Less is More for RL Scaling
  • Xuefeng Li, 호양 Zou, Pengfei Liu
• LIMO: Less is More for Reasoning
  • Yixin 그렇습니다, Zhen Huang, 양 Xiao, Ethan Chern, Shijie Xia, Pengfei Liu
• s1: Simple test-time scaling
  • Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, Tatsunori Hashimoto
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
  • Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, 이마
• Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  • Zhenyu Hou, Xin Lv, Rui Lu, Jiajie 장, Yujiang Li, Zijun Yao, Juanzi Li, Jie Tang, Yuxiao Dong
• Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
  • Maohao Shen, Guangtao Zeng, Zhenting Qi, Zhang-Wei Hong, Zhenfang Chen, Wei Lu, Gregory Wornell, Subhro Das, 데이비드 Cox, Chuang Gan
• Distillation Quantification for Large Language Models
  • Sunbowen Lee, Junting Zhou, Chao Ao 등.
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
  • Xinyu Guan, Li Lyna 장, Yifei Liu, Ning Shang, Youran 일요일, 이 주, 팬 양, Mao 양
• Evolving Deeper LLM Thinking
  • Kuang-Huei Lee, Ian Fischer, Yueh-Hua Wu, Dave Marwood, Shumeet Baluja, Dale Schuurmans, 신현 첸
• The Lessons of Developing Process Reward Models in Mathematical Reasoning
  • Zhenru 장, Chujie Zheng, Yangzhen Wu, Beichen 장, Runji 린, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin
• Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
  • Violet Xiang, Charlie Snell, Kanishk Gandhi, Alon Albalak, Anikait Singh, Chase Blagden, Duy Phung, Rafael Rafailov, Nathan Lile, Dakota Mahan, Louis Castricato, Jan-Philipp Franken, Nick Haber, 첼시 핀
• PRMBENCH: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
  • Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou, 유 청
• Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
  • Yifan Du, Zikang Liu, Yifan Li, 웨인 Xin Zhao, Yuqi Huo, Bingning 왕, Weipeng 첸, Zheng Liu, Zhongyuan 왕, Ji-Rong 원
• Imagine while Reasoning in Space: Multimodal Visualization-of-Thought
  • Chengzu Li, Wenshan Wu, Huanyu 장, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, 후루 웨이
• LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs
  • Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev, Mubarak Shah, Fahad Shahbaz Khan, 살만 칸
• From chaos to order: The atomic reasoner framework for fine-grained reasoning in large language models
  • Jinyi Liu, Yan Zheng, Rong 청, Qiyu Wu, Wei Guo, Fei Ni, Hebin Liang, Yifu Yuan, Hangyu Mao, Fuzheng 장, Jianye Hao

2024

• Deliberative alignment: reasoning enables safer language models
  • 오시는 길
• MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
  • Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan May, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Lilian Weng, Aleksander Mądry
• From Medprompt to o1: Exploration of Run-Time Strategies for Medical Challenge Problems and Beyond
  • 스콧 맥키니
• LLM Critics Help Catch LLM Bugs
  • Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, 마하 Trebacz, Jan Leike
• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
  • Yuexiang Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Shengbang Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, Sergey Levine
• ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
  • Haozhan Shen, Kangjia Zhao, Tiancheng Zhao, Ruochen Xu, Zilun 장, Mingwei 주, Jianwei Yin
• Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
  • Wenbin 왕, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Dacheng Tao
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
  • Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral 쿠마르
• An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
  • 양주 Wu, Zhiqing 일요일, Shanda Li, Sean Welleck의 Yiming 양
• Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling
  • Hritik Bansal, Arian Hosseini, Rishabh Agarwal, Vinh Q. Tran, 메 란 카제미
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
  • 브래들리 브라운, 요르단 Juravsky, 라이언 Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, Azalia Mirhoseini
• Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems
  • Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang, Xiaoxue Cheng, Huatong Song, 웨인 Xin Zhao, Zheng Liu, Zhongyuan 왕, Ji-Rong Wen
• Training Language Models to Self-Correct via Reinforcement Learning
  • Aviral 쿠마르, 빈센트 Zhuang, Rishabh Agarwal, 이 수, 존 D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal Bebahhani, Al Fatandra
• Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
  • Xingyu 첸, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng 장, Rui 왕, Zhaopeng Tu, Haitao Mi, Dong Yu
• MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes
  • Asma Ben Abacha, Wen-wai Yim, Yujuan Fu, Zhaoyi Sun, Meliha Yetisgen, Fei Xia, 토마스 린
• Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
  • 양, Beichen 장, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, Zhenru 장
• Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
  • Zhenyu Hou, Pengfan Du, Yilin Niu, Zhengxiao Du, Aohan Zeng, Xiao Liu, Minlie Huang, 홍닝 왕, Jie Tang, Yuxiao Dong
• Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering
  • Xinyan Guan, Yanjiang Liu, Xinyu Lu, Boxi Cao, Ben He, Xianpei Han, Le Sun, Jie Lou, Bowen Yu, Yaojie Lu, 홍유 린
• Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective
  • Zhiyuan Zeng, Qinyuan 청, Zhangyue Yin, Bo 왕, Shimin Li, Yunhua Zhou, Qipeng Guo, Xuanjing Huang, Xipeng Qiu
• Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
  • Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, 노아 D. Goodman
  • https://github.com/ezelikman/quiet-star
• Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
  • Zhiheng Xi, Dingwen Yang, Jixuan Huang, Jiafu Tang, Guanyu Li, Yiwen Ding, Wei He, Boyang Hong, Shihan Do, Wenyu Zhan, Xiao Wang, Rui Zheng, Tao Ji, Xiaowei Shi, Yitao Zhai, Rongxiang Weng, Jingang Wang, Xunliang Cai, Tao Gui, Zuxuan Wu, Qi Zhang, Xipeng Qiu, Xjing Huang, 장강, Yu
  • https://mathcritique.github.io/
• On Designing Effective RL Reward at Training Time for LLM Reasoning
  • Jiaxuan Gao, Shusheng Xu, Wenjie 예, Weilin Liu, Chuyi 그는, Wei Fu, Zhiyu Mei, Guangju 왕, 이 우
• Generative Verifiers: Reward Modeling as Next-Token Prediction
  • Lunjun 장, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, Rishabh Agarwal
• Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
  • Agarwal, Alekh Agarwal, Jonathan Berant, Aviral Kumar
• Improve Mathematical Reasoning in Language Models by Automated Process Supervision
  • Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, Abhinav Rastogi
• Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
  • Peiyi 왕, Lei Li, Zhihong Shao, R.X. Xu, Damai Dai, Yifei Li, Deli Chen, Y.Wu, Zhifang Sui
• Planning In Natural Language Improves LLM Search For Code Generation
  • Evan 왕, Federico Cassano, 캐서린 Wu, Yunfeng Bai, 윌 송, Vaskar Nath, Ziwen 한, Sean Hendryx, 여름 Yue, 장 휴
• PROCESSBENCH: Identifying Process Errors in Mathematical Reasoning
  • Chujie Zheng, Zhenru 장, Beichen 장, Runji 린, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang 린
• AFlow: Automating Agentic Workflow Generation
  • Jiayi 장, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, Chenglin Wu
• Interpretable Contrastive Monte Carlo Tree Search Reasoning
  • Zitian Gao, Boye Niu, Xuzheng He, Haotian Xu, Hongzhang Liu, Aiwei Liu, Xuming Hu, Lijie Wen
• Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
  • Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, 첼시 핀, Divyansh Garg, Rafael Rafailov
• Mixture-of-Agents Enhances Large Language Model Capabilities
  • Junlin 왕, Jue 왕, 벤 Athiwaratkun, Ce 장, 제임스 Zou
• Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models
  • Zhiyuan Hu, Chumin Liu, Xidong Feng, Yilun Zhao, See-Kiong Ng, Anh Tuan Luu, Junxian 그는, Pang Wei Koh, Bryan Hooi
• Advancing LLM Reasoning Generalists with Preference Trees
  • Lifan Yuan, Ganqu Cui, Hanbin Wang, Ning Ding, Xingyao Wang, Jia Deng, Boji Shan 등.
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
  • 예 Tian, Baolin Peng, Linfeng Song, Lifeng Jin, Dian Yu, Haitao Mi 및 Dong Yu.
• AlphaMath Almost Zero: Process Supervision Without Process
  • Guoxin 첸, Minpeng Liao, 청시 Li, 카이 팬.
• ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
  • 다 장, Sining Zhoubian, Yisong Yue, Yuxiao Dong 및 Jie Tang.
• Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search
  • Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi 장, Yibo 왕, Shunyu Liu, Yingjie 왕, Yuxin 노래, Haocheng Feng, Li Shen, Dacheng Tao
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
  • Yuhao Dong, Zhuyan Liu, Hai-Long 일요일, Jingkang 양, Winston Hu, Yongming Rao, Ziwei Liu
• MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time
  • Jikun Kang, Xin Zhe Li, Xi Chen, Amirreza Kazemi, Qianyi Sun, 복싱 첸, Dong Li, Xu He, Quan He, Feng Wen, Jianye Hao, Jun Yao.
• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning
  • Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P. Lillicrap, Kenji Kawaguchi, 마이클 Shieh.
• When is Tree Search Useful for LLM Planning? It Depends on the Discriminator
  • Ziru 첸, 마이클 화이트, 레이몬드 Mooney, 알리페이니, 유수, 화이 선
• Chain of Thought Empowers Transformers to Solve Inherently Serial Problems
  • Zhiyuan Li, 홍콩, Denny Zhou, Tengyu Ma.
• To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
  • Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, 그레그 Durrett
• Do Large Language Models Latently Perform Multi-Hop Reasoning?
  • 소희 양, 엘레나 Gribovskaya, 노라 Kassner, 모 Geva, 세바스티안 Riedel
• Chain-of-Thought Reasoning Without Prompting
  • Xuezhi 왕, 데이니 Zhou
• Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
  • Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna 장, 팬 양, Mao 양
• Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
  • Xuan 장, Chao Du, Tianyu Pang, Qian Liu, Wei Gao, 분 린
• ReFT: Reasoning with Reinforced Fine-Tuning
  • Trung Quoc Luong의 Xinbo 장, Zhanming Jie, Peng 일요일, Xiaoran Jin의 걸림새 Li
• VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment
  • Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, 니콜라스 르 Roux
• Stream of Search (SoS): Learning to Search in Language
  • Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, 노아 D. Goodman
• GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
  • 이만 Mirzadeh, Keivan Alizadeh, 호만 Shahrokhi, 한 번 Tuzel, Samy Bengio, Mehrdad Farajtabar
• Evaluation of OpenAI o1: Opportunities and Challenges of AGI
  • Tianyang Zhong, Zhengliang Liu, 이 판, Yutong 장, Yifan Zhou, Shizhe Liang, Zihao Wu, Yanjun Lyu, Peng Shu, Xiaowei Yu, Chao Cao, Hanqi Jiang, Hanxu Chen, Yiwei Li, Junhao Chen, 등.
• Evaluating LLMs at Detecting Errors in LLM Responses
  • Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Janice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Ranran Haoran Zhang, Sujeeth Reddy Vummanthala, Salika Dave, Shaobo Qin, Arman Cohan, Wenpeng Yin, Rui 장
• On The Planning Abilities of OpenAI’s o1 Models: Feasibility, Optimality, and Generalizability
  • 케빈 왕, Junbo Li, Neel P. Bhatt, Yihan Xi, Qiang Liu, Ufuk Topcu, 장양 왕
• Not All LLM Reasoners Are Created Equal
  • Arian Hosseini, Alessandro Sordoni, 다니엘 토야마, Aaron Courville, Rishabh Agarwal
• LLMs Still Can’t Plan; Can LRMs? A Preliminary Evaluation of OpenAI’s o1 on PlanBench
  • Karthik Valmeekam, Kaya Stechly, Subbarao Kambhampati, 인도네시아
• A Comparative Study on Reasoning Patterns of OpenAI’s o1 Model
  • Siwei Wu, Zhongyuan Peng, Xinrun Du, Tuney Zheng, Minghao Liu, Jialong Wu, Jiachen Ma, Yizhi Li, Jian 양, Wangchunshu Zhou, Qunshu Lin, Junbo Zhao, Zhaoxiang 장, Wenhao Huang, Ge 장, Chenghua Lin, J.H. Liu
• Thinking LLMs: General Instruction Following with Thought Generation
  • Tianhao Wu, Janice Lan, Weizhe Yuan, Jiantao Jiao, Jason Weston, Sainbayar Sukhbaatar
• Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems
  • Jun Zhao, Jingqi Tong, Yurong Mou, Ming 장, Qi 장, Xuanjing Huang
• V-STaR: Training Verifiers for Self-Taught Reasoners
  • Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, Rishabh Agarwal
• CPL: Critical Plan Step Learning Boosts LLM Generalization in Reasoning Tasks
  • Tianlong 왕, Junzhe 첸, Xueting 한, Jing Bai
• RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
  • Tianhao Wu, Janice Lan, Weizhe Yuan, Jiantao Jiao, Jason Weston, Sainbayar Sukhbaatar
• Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
  • Chaojie 왕, Yanchen Deng, Zhiyi Lyu, Liang Zeng, Jujie 그는, Shuicheng Yan, Bo 안
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning
  • Hao Shao, Shengju Qian, 한 Xiao, Guanglu 노래, Zhuofan Zong, Letian 왕, Yu Liu, Hongsheng Li

2023

• Let’s Verify Step by Step
  • Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, 테디 리, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe
• V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
  • Penghao 우, Saining Xie
• Training Chain-of-Thought via Latent-Variable Inference
  • Du Phan, Matthew D. Hoffman, David Dohan, Sholto 더글러스, Tuan Anh Le, Aaron Parisi, Pavel Sountsov, Charles Sutton, Sharad Vikram, Rif A. Saurous
• Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training
  • Xidong Feng, Ziyu Wan, Muning Wen, Stephen Marcus McAleer, Ying Wen, 웨이난 장, Jun 왕
• OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning
  • Fei Yu, Anningzhe Gao, 베니왕
• Reasoning with Language Model is Planning with World Model
  • Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, 젠 왕, 데이지 Zhe 왕, Zhiting Hu
• Don’t throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
  • Liu, Jiacheng, Andrew Cohen, Ramakanth Pasunuru, Yejin Choi, Hannaneh Hajishirzi 및 Asli Celikyilmaz.
• Certified reasoning with language models
  • Gabriel Poesia, Kanishk Gandhi, 에릭 Zelikman, 노아 D. Goodman
• Large Language Models Cannot Self-Correct Reasoning Yet
  • Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu 스티븐 Zheng, Adams Wei Yu, Xinying Song, Denny Zhou

2022

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Jason Wei, Xuezhi 왕, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Xuezhi 왕, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, 데니 Zhou
• Self-critiquing models for assisting human evaluators
  • 윌리엄 Saunders, 캐서린 Yeh, 제프 우, 스티븐 빌, 긴 Ouyang, 조나단 구, Jan Leike
• Chain of Thought Imitation with Procedure Cloning
  • Mengjiao 양, Dale Schuurmans, Pieter Abbeel, Ofir Nachum.
• STaR: Bootstrapping Reasoning With Reasoning
  • 에릭 Zelikman, Yuhuai 우, Jesse Mu, 노아 D. Goodman
• Solving math word problems with processand outcome-based feedback
  • Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, 이리나 Higgins

2021

• Training Verifiers to Solve Math Word Problems
  • Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, John Schulman
• Scalable Online Planning via Reinforcement Learning Fine-Tuning
  • Arnaud Fickinger, Hengyuan Hu, Brandon Amos, 스튜어트 러셀, 노암 브라운.
• Scaling Scaling Laws with Board Games
  • 앤디 L. 존스.
• Show Your Work: Scratchpads for Intermediate Computation with Language Models
  • Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, Augustus Odena

2021년 이전

• Improving Policies via Search in Cooperative Partially Observable Games
  • Adam Lerer, Hengyuan Hu, Jakob Foerster, 노암 브라운.
• Generative Language Modeling for Automated Theorem Proving
  • Stanislas Polu, 일리아 쉬크
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
  • 데이비드 실버, 토마스 허버트, 줄리안 Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis.