LLM Strawberry

Source: https://github.com/hijkzzz/Awesome-LLM-Strawberry at 3f17fc9ee49c
License: Apache-2.0
Read the English original

This edition is an AI-assisted translation.

LLM Strawberry（OpenAI o1）の研究資料

OpenAI Strawberry（o1）と推論に関する研究論文やブログを集めています。 これは、研究論文とブログのコレクションです OpenAI Strawberry(o1)とReasoning. LLM の推論分野の最前線を追うため、リポジトリは継続的に更新されます。 LLM Reasoningのフロンティアを追跡するために、リポジトリは継続的に更新されます。

OpenAI ドキュメント

• https://platform.openai.com/docs/guides/reasoning
• 

ニュース

• [オープンAI] Introducing deep research
• [オープンAI] o3 preview & o3 mini
• [オープンAI] Introducing ChatGPT Pro
• [Googleのディープマインド] Gemini 2.0 Flash Thinking
• [イリアススキバー] AI with reasoning power will be less predictable
• [SemianAlysis] Scaling Laws – O1 Pro Architecture, Reasoning Training Infrastructure, Orion and Claude 3.5 Opus “Failures”
• [DeepSeek] DeepSeek-R1-Lite-Preview is now live: unleashing supercharged reasoning power!
• [ムーンシュート] 数学对标o1系列，搜索再次进化，Kimi 新推理模型与你一起拓展智能边界
• [ムーンシュート] Kimi 发布视觉思考模型 k1，多项理科测试行业领先
• [インターンLM] 强推理模型书生InternThinker开放体验：自主生成高智力密度数据、具备元动作思考能力
• [新智元] 万字独家爆光，首揭o1 pro架构！惊人反转，Claude 3.5 Opus没失败？

ブログ

• [オープンAI] Learning to Reason with LLMs
• [オープンAI] OpenAI o1-mini Advancing cost-efficient reasoning
• [オープンAI] Finding GPT-4’s mistakes with GPT-4
• [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI] - [ARC-AGI - [ARC-AGI] - [ARC-AGI - [ARC-AGI] - [ARC-AGI - [ARC-AGI] - [ARC-AGI] - [ARC-AGI - [ARC-AGI-AGI - [ARC-AGI] - [ARC-AGI - [ARC-AGI-AGI-AGI-AGI] - [ARC-AGI-AGI] - [ARC-AGI - [ARC-AGI-AGI-AGI-AGI - [ARC-AGI-AGI-AGI] - [ARC-AGI-AGI-AGI-AGI-AGI] - [ARC-AGI OpenAI o3 Breakthrough High Score on ARC-AGI-Pub
• [特性] Building effective agents
• [hijkzzz] Stabilizing MoE RL Without Router Replay: The Online IcePop Solution
• [hijkzzz] REINFORCE++-baseline is all you need in RLVR
• [hijkzzz] Exploring OpenAI O1 Model Replication
• [ネイサン・ランバート] OpenAI’s Strawberry, LM self-talk, inference scaling laws, and spending more on inference
• [ネイサン・ランバート] Reverse engineering OpenAI’s o1
• [Andreas Stuhlmüller, jungofthewon] 公式サイト Supervise Process, not Outcomes
• [ノウハ・ディジリ] Have o1 Models Cracked Human Reasoning?
• [リシャブ・アガルヴァル] Improving LLM Reasoning using Self-generated data: RL and Verifiers
• [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen] [Wei Shen [Wei Shen] [Wei Shen [Wei Shen] [Wei Shen [Wei Shen] [Wei Shen [Wei Shen] [Wei Shen [Wei Shen] [Wei Shen [Wei Shen] [Wei Shen [Wei Shen]] [Wei Shen [Wei Shen [Wei Shen [Wei Shen [Wei Shen [Wei Shen [Wei Shen]]]] [Wei Shen [Wei Shen [Wei Shen]] [Wei Shen [Wei Shen [Wei Shen [Wei Shen]]] [Wei Shen [Wei Shen]] [Wei Shen [Wei Shen [Wei Shen Generalization Progress in RLHF: Insights into the Impact of Reward Models and PPO
• [ドミナター069] Codeforces - Analyzing how good O1-Mini actually is
• [ティボ・ブラホ] Summary of what we have learned during AMA hour with the OpenAI o1 team

インタビュー

• 【ノームブラウン】 Parables on the Power of Planning in AI: From Poker to Diplomacy
• 【ノームブラウン】 OpenAI o1 and Teaching LLMs to Reason Better
• [Hyung Won Chung] [Hyung Won Chung] [Hyung Won Chung] [Hyung Won Chung] [Hyung Won Chung] [Hyung Won Chung] [Hyung Won Chung] [Hyung Won Chung [Hyung Won Chung] [Hyung Won Chung [Hyung Won Chung] [Hyung Won Chung [Hyung] [Hyung Won Chung [Hyung] [Hyung [Hyung Won Chung [Hyung] [Hyung] [Hyung] [Hyung [Hyung [Hyung Won [Hyung Won Chung [Hyung] [Hyung] [Hyung] [Hyung] [Hyung] [Hyung Won [Hyung Won Chung [Hyung Won] [Hyung] [Hyung Won Chung [Hyung Won Chung [Hyung] [Hyung] [Hyung] [H Don’t teach. Incentivize.

コース

• [ディープラーニング] AI(人工知能) Reasoning with o1

ツイート

OpenAI開発者

• All the questions addressed by the API team during the December 17, 2024 AMA
• We’re hosting an AMA for developers from 10–11 AM PT today.
• Today we previewed Reinforcement Fine-Tuning
• 

ノアムブラウン

• 
• 
• 
• 

ジェイソン・ワイ

• 
• 
• 

その他

• 
• 
• 

オープンソース

モデル

• [Alibaba Qwenチーム] Qwen3
• [Alibaba Qwenチーム] QwQ
• [Alibaba Qwenチーム] QvQ
• [DeepSeek] DeepSeek R1
• [NVIDIA] Nemotron-Research-Reasoning-Qwen-1.5B
• [スカイワーク] Skywork R1V2
• [rLLM] DeepScaler
• [ノバスカイ] Sky-T1
• 【GAIR-NLP】 O1 Replication Journey: A Strategic Progress Report
• [OpenO1チーム] Open-Source O1
• [テンセント] DRT-o1
• [Alibaba] Marco-o1
• [CUHK-SZ] HuatuoGPT-o1

コードベース

• [OpenRLHF チーム] OpenRLHF
• [OpenRLHF チーム] REINFORCE++ | REINFORCE++-baseline | Code
• [ノバスカイAI] SkyRL
• [RUCAIBOX] STILL: Slow Thinking with LLMs
• [HKUST] Simple Reinforcement Learning for Reasoning
  • DeepSeek-R1-ZeroとDeepSeek-R1の小規模なモデルでのトレーニングの再現
• [ユビカント] Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
• [ステップファン] Open-Reasoner-Zero
• [TideDra] LMM-R1
• [モーダルマインド] MM-EUREKA: Exploring Visual Aha Moment with Rule-based Large-scale Reinforcement Learning
• [R1-Vチーム] R1-V
• [LLaMA-Factoryチーム] EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework
• [Alibaba] ROLL | Paper
• [海AIラボ] Dr. GRPO
• [Berkeley AI 研究] TinyZero
• [Maitrix.org] LLM Reasoners

ペーパー

format:
- [title](paper link) [links]
  - author1, author2, and author3...
  - publisher
  - code
  - experimental environments and datasets

o1モデルに関する技術的なレポート

• INTELLECT-3: Technical Report
  • プライム・インテルlectチーム
• DeepSeek V3.2
  • DeepSeekのAI
• MiMo-V2 Flash
  • 西尾 美
• Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
  • NVIDIAの
• Qwen3 Technical Report
  • Qwenチーム
• Scaling Agents via Continual Pre-training
  • Qwenチーム
• WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents
  • Qwenチーム
• Towards General Agentic Intelligence via Environment Scaling
  • Qwenチーム
• Magistral
  • ミストラルAI
• ERINE-4.5
  • ログイン
• LongCat Flash
  • メイトアン
• GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
  • ジープAI
• Seed Thinking v1.5
  • 耐久性種子
• DeepSeek-V3 Technical Report
  • イマラチオ
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
  • DeepSeekのAI
• DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
  • DeepSeekのAI
• MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
  • ミニマックス
• Kimi k2: Open agentic intelligence
  • ムーンショット
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
  • ムーンショット
• KIMI-VL TECHNICAL REPORT
  • ムーンショット
• Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning
  • ムーンショット&ヌミナ
• Llama-Nemotron: Efficient Reasoning Models
  • NVIDIAの
• Skywork Open Reasoner 1 Technical Report
  • スカイワーク

2025

• Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
  • NVIDIAの
• QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory
  • Qwenチーム
• DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
  • DeepSeek-AIの特長
• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices
  • Qwenチーム
• The Art of Scaling Reinforcement Learning Compute for LLMs
  • Devvrit Khatri、Madaan、Rishabh Tiwari、Rachit Bansal、Sai Surya Duvvuri、Manzil Zaheer、Inderjit S. Dhillon、David Brandfonbrener、Rishabh Agarwal
• BroRL: Scaling Reinforcement Learning via Broadened Exploration
  • Jian Hu, Mingjie Liu, Ximing Lu, Fang Wu, Zaid Harchaoui, Shizhe Diao, Yejin Choi, Pavlo Molchanov, Jun Yang, Jan Kautz, Yi Dong
• Why Language Models Hallucinate
  • オープンAI
• rStar2-Agent: Agentic Reasoning Technical Report
  • Ning Shang、Yifei Liu、Yi Zhu、Li Lyna Zhang、Weijiang Xu、Xinyu Guan、Buze Zhang、Bingcheng Dong、Xudong Zhou、Bowen Zhang、Ying Xin、Ziming Miao、Scarlett Li、ファンYang、Mao Yang
• RL Is Neither a Panacea Nor a Mirage: Understanding Supervised vs. Reinforcement Learning Fine-Tuning for LLMs
  • Hangzhan Jin、Sicheng、Sifan Wu、Mohammad Hamdaqa
• DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization
  • Shuaijie She, Yu Bao, Yu Lu, Lu, Lu Xu, Tao Li, Wenhao Zhu, Shujian Huang, Shanbo Cheng, Lu Lu, Yuxuan Wang, Yuxuan Wang, Yuxuan Wang, Yuxuan Wang, Yuxuan, Yuxuan, Yu, Yuxuan, Yu, Yuxuan, Yu, Yuxuan, Yuxuan, Yu, Yu, Yuxuan,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai,Tai
• Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning
  • Vaishnavi Shrivastava、Ahmed Awadallah、Vidhisha Balachandran、Shivam Garg、Harkirat Behl、Dimitris Papailiopoulos
• ProRL V2 - Prolonged Training Validates RL Scaling Laws
  • Jian Hu, Mingjie Liu, Shizhe Diao, Ximing Lu, Xin Dong, Pavlo Molchanov, Yejin Choi, Jan Kautz, 李東
• Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning
  • Zihe Liu, Jiashun Liu, Yancheng He, Weixun Wang, Jiaheng Liu, Ling Pan, Xinyu Hu, Shaopan Xiong, Ju Huang, Jian Hu, Shengyi Huang, Siran Yang, Jiamang Wang, Wenbo Su, Bo Zheng
• Group Sequence Policy Optimization
  • 問合せ先: Rémi Coulom.
• Your Efficient RL Framework Secretly Brings You Off-Policy RL Training
  • Feng Yao、Liyuan Liu、Dinghuai Zhang、Chengyu Dong、Jingbo Shang、Jianfeng Gao
• Gemini 2.5 Pro Capable of Winning Gold at IMO 2025
  • Yichen Huang, 林 F. ヤン
• DeepSWE: Training a Fully Open-sourced, State-of-the-Art Coding Agent by Scaling RL
  • 一緒に.ai
• OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
  • Zengzhi 王, ファン Zhou, Xuefeng 李, Pengfei Liu
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
  • Mingjie Liu、Shizhe Diao、Ximing Lu、Jian Hu、Xin Dong、Yejin Choi、Jan Kautz、Yi Dong
• REINFORCE++: An Efficient RLHF Algorithm with Robustness to Both Prompt and Reward Models
  • Jian Hu, Jason Klein Liu, ワイ・シェン
  • コード: REINFORCE++-baseline
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
  • Qwenチーム
• The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
  • Xinyu Zhu、Mengzhou Xia、Zhepei Wei、Wei Lin Chen、Danqi Chen、Yu Meng
• Thinking with Generated Images
  • Ethan Chern、Zhijie Deng、Pengfei Liu
• Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning
  • Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, Wenhu Chen, Wenhu Chen, オーストラリア
• DeepEyes: Incentivizing “Thinking with Images” via Reinforcement Learning
  • Ziwei Zheng、Michael Yang、Jack Hong、Chenxiao Zhao、Guohai Xu、Le Yang、Chao Shen、Xing Yu
• Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging
  • 問合せ先: Rémi Coulom.
• QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
  • Fanqi湾、温州の小屋、Shengyi Liao、Yingcheng Shi、Chenliang李、Ziyiヤン、Ji Zhang、Fei Huang、Jingren Zhou、Ming Yan
• Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving
  • 問合せ先: Rémi Coulom.
  • コード: https://github.com/yyht/openrlhf_async_pipline
• A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
  • Wei Xiong、Jiarui Yao、Yuhui Xu、Bo Pang、Lei Wang、Doyen Sahoo、Junnan Li、Nan Jiang、Tong Zhang、Cを向けるXong、Hanze Dong
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example
  • Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, Yelong Shen
• Process Reward Models That Think
  • Muhammad Khalifa、Rishabh Agarwal、Lajanugen Logeswaran、Jekyeom Kim、Hao Peng、Moontae Lee、Honglak Lee、Lu Wang
• M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
  • Junxiong Wang, Wen-Ding Li, Daniele Paliotta, Daniel Ritter, Alexander M. Rush, トライ・ダオ
• A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility
  • Andreas Hochlehnert、Hardik Bhatnagar、Vishaal Udandarao、Samuel Albanie、Ameya Prabhu、Mathias Bethge
• Concise Reasoning via Reinforcement Learning
  • Mehdi Fatemi、Banafsheh Rafiee、Mingjie Tang、Kartik Talamadupula
• VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks
  • YuYue、Yufeng Yuan、Qiying Yu、Xiaochen Zuo、Ruofei Zhu、Wenyuan Xu、Jhengyi Wang、TianTianファン、Zhengyin Du、Xianangpeng Wei、Gaohong Liu、Juncai Liu、Lingjun Liu、Haibin Lin、Zhiqi Lin、Bole Ma、等。
• Inference-Time Scaling for Generalist Reward Modeling
  • Zijun Liu、Peiyi Wang、Runxin Xu、Shirong Ma、Chong Ruan、Peng李、Yang Liu、Yu Wu
• JudgeLRM: Large Reasoning Models as a Judge
  • Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, Bingsheng He, ジンギアン・ホイ
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
  • Qiying Yu、Zheng Zhang、Ruofei Zhu、Yufeng Yuan、Xiaochen Zuo、Yue、Tiantianファン、Gaohong Liu、Lingjun Liu、Xin Liu、Haibin Lin、等。
• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
  • Jonas Geiping、Sean McLeish、Neel Jain、John Kirchenbauer、Siddharth Singh、Bhavya Kailkhura、Abhinav Bhatele、Tom Goldstein
• Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning
  • Yuxiao Qu、マタイY. R.ヤン、Amrith Setlur、Lewis Tunstall、Edward Emanuel Beeching、Ruslan Salakhutdinov、Aviral Kumar
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
  • Huatongソング、Jinhao江、Yingqian Min、Jie Chen、Zhipeng Chen、Wayne Xin Zhao、Lei Fang、Ji-Rong Wen
• Visual-RFT: Visual Reinforcement Fine-Tuning
  • 問合せ先: Rémi Coulom.
• Introducing Visual Perception Token into Multimodal Large Language Model
  • Runpeng Yu、Xinyin Ma、Xinchao 王
• Scaling Test-Time Compute Without Verification or RL is Suboptimal
  • Amrith Setlur、Nived Rajaraman、Sergey Levine、Aviral Kumar
• LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
  • Dacheng Li、Shiyi Cao、Tyler Griggs、Shu Liu、Xiangxi Mo、Shishir G. Patil、Matei Zaharia、Joseph E. Gonzalez、Ion Stoica
• Demystifying Long Chain-of-Thought Reasoning in LLMs
  • Edward Yeo, Yuxuan Tong, Morry Niu, グラハム Neubig, Xiang Yue
• LIMR: Less is More for RL Scaling
  • Xuefeng李、Haoyang Zou、Pengfei Liu
• LIMO: Less is More for Reasoning
  • Yixin Ye、Zhen Huang、Yang Xiao、Ethan Chern、Shijie Xia、Pengfei Liu
• s1: Simple test-time scaling
  • Niklas Muennighoff、Zitong Yang、Weijia Shi、Xiang Lisa Li、Li Fei-Fei、Hannaneh Hajishirzi、Luke Zettlemoyer、Percy Liang、Emanuel Candès、Tatsunori Hashimoto
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
  • Tianzhe Chu、Yexiang Zhai、Jihan Yang、Shengbang Tong、Saining Xie、Dale Schuurman、Quoc V。 Le、Sergey Levine、Yi Ma
• Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  • Zhenyu Hou、Xin Lv、Rui Lu、Jiajie Zhang、Yujiang Li、Zijun Yao、Fuanzi Li、Jie Tang、Yuxiao Dong
• Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
  • Maohao Shen、Guangtao Zeng、Zhenting Qi、Zhang-Wei Hong、Zhenfang Chen、Wei Lu、Gregory Wornell、Subhro Das、David Cox、Chang Gan
• Distillation Quantification for Large Language Models
  • Sunbowen Lee、Junting Zhou、Chao Ao、等。
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
  • Xinyu Guan、Li Lyna Zhang、Yifei Liu、Ning Shang、Yi Zhu、ファンYang、Mao Yang
• Evolving Deeper LLM Thinking
  • Kuang-Huei Lee、Ian Fischer、Yueh-Hua Wu、Dave Marwood、Shumeet Baluja、Dale Schuurman、Xinyun陳
• The Lessons of Developing Process Reward Models in Mathematical Reasoning
  • Zhenru Zhang、Chujie Zheng、Yangzhen Wu、Beichen Zhang、Runji Lin、Bowen Yu、Dayiheng Liu、Jingren Zhou、Juanyang Lin
• Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
  • Violet Xiang、Charlie Snell、Kanishk Gandhi、Alon Albalak、Anikait Singh、Chase Blagden、Duy Phung、Rafael Rafailov、Nathan Lile、Dakota Mahan、Louis Castricato、Jan-Philipp Franken、Nick Haber、Chelsea Finn
• PRMBENCH: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
  • 明陽歌、Zhaochen Su、Xiaaoye Qu、Jiawei Zhou、Yu Cheng
• Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
  • Yifan Du、Zikang Liu、Yifan Li、Wayne Xin Zhao、Yqi Huo、Bingning Wang、Yipeng Chen、Zheng Liu、Zhongyuan Wang、Ji Rong Wen
• Imagine while Reasoning in Space: Multimodal Visualization-of-Thought
  • 成都李、温山呉、Huanyu Zhang、Yan Xia、Shaoguang Mao、Li Dong、Ivan Vulić、Furu Wei
• LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs
  • Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev, Mubarak Shahbaz Khan, サルマン・カーン
• From chaos to order: The atomic reasoner framework for fine-grained reasoning in large language models
  • 問合せ先: Rémi Coulom.

2024

• Deliberative alignment: reasoning enables safer language models
  • オープンAI
• MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
  • ジュン・シェルン・チャン、ニール・コーディレイ、オリヴァー・ジャフ、ジェームズ・アング、ダニー・シェルバーン、エヴァン・メイス、ジリオ・スターエース、ケビン・リュー、レオン・マキン、テジアル・パトワードハン、リリアン・ウェング、アレキサンダー・マツドリー
• From Medprompt to o1: Exploration of Run-Time Strategies for Medical Challenge Problems and Beyond
  • スコット・マッキンニー
• LLM Critics Help Catch LLM Bugs
  • Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Trebacz, 1月 Leike
• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
  • Yuexiang Zhai、Hao Bai、Zipeng Lin、Jiayi Pan、Shengbang Tong、Yifei Zhou、Alane Suhr、Saining Xie、Yan LeCun、Yi Ma、Sergey Levine
• ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
  • Haozhan Shen、Kangjia Zhao、Tiancheng Zhao、Ruochen Xu、Zilun Zhang、Mingweiのチューン、Jianwei Yin
• Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
  • Wenbin Wang、Liang Ding、Minyan Zeng、Xiabin Zhou、Li Shen、Yong Luo、Dacheng Tao
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
  • Charlie Snell, Jaehoon Lee, ケルビンXu, エイビアル・クマール
• An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
  • Yangzhen 呉、Zhiqing 日、山田李、海兵器、Yoming ヤン
• Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling
  • Hritik Bansal、Arian Hosseini、Rishabh Agarwal、Vinh Q. Tran、Mehran Kazemi
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
  • ブラッドリー・ブラウン、ヨルダン・ジュラフスキー、ライアン・ハリッヒ、ローナルド・クラーク、クオック・V・ル、クリストファー・レ、アゾリア・ミロージーニ
• Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems
  • Yingqian Min、Zhipeng Chen、Jie Chen、Jia Deng、Yiwen Hu、Yiru Tang、Jiapeng Wang、Xiaoxue Cheng、Huatong Song、Wayne Xin Zhao、Zheng Liu、Zhongyuan Wang、JiRong Wen
• Training Language Models to Self-Correct via Reinforcement Learning
  • Aviral Kumar、Vincent Zhuang、Rishabh Agarwal、Yi Su、John D Co-Reyes、Avi Singh、Kate Baumli、Shariq Iqbal、Colton Bishop、Rebecca Roelofs、Lei M Zhang、Kay McKinney、Disha Shrivastava、Cosmin Paduraru、George Tucker、Doina Precup、Feryal Behbahani、Alekandra Fauste
• Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
  • Xingyu Chen、Jiaao Xu、Tian Liang、Zhiwei He、Jianhui Pang、Dian Yu、Qiuzhi Liu、Mengfei Zhou、Zhuo Wang、Zhaopeng Tu、Haiitao Mi、Dong Yu
• MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes
  • アスマ・ベン・アバチャ、ウェンワイ・ワイム、ユジュン・フ、ザイ・サン、メリハ・イチスゲン、フェイ・イア、トーマス・リン
• Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
  • ヤン、Beichen Zhang、Binyuan Hui、Bofei Gao、Bowen Yu、Chengpeng Li、Dayiheng Liu、Jianhong Tu、Jingren Zhou、Jinyang、Keming Lu、Mingfeng Xue、Runji Lin、Tianyu Liu、Xingzhang Ren、Zhenru Zhang
• Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
  • Zhenyu Hou、Pengfan Du、Yilin Niu、Zhengxiao Du、Aohan Zeng、Xiao Liu、Minlie Huang、Hongning Wang、Jie Tang、Yuxiao Dong
• Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering
  • 問合せ先: Rémi Coulom.
• Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective
  • ZhiyuanのZeng、Qinyuanのチェン、Zhangyue Yin、Bo Wang、チーミン李、Yunhua Zhou、Qipeng Guo、Xuanjing Huang、Xipeng Qiu
• Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
  • Eric Zelikman、Georges Harik、Yijia Shao、Varuna Jayasiri、Nick Haber、Noah D.Goodman
  • https://github.com/ezelikman/quiet-star
• Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
  • Zhiheng Xi、Dingwen Yang、Jixuan Huang、Jiafu Tang、Guanyu Li、Yiwen Ding、Wei He、Boyang Hong、Shihan Do、Wenyu Zhan、Xiaao Wang、Rui Zheng、Tao Ji、Xiaaowei Shi、Yitao Zhai、Rongxiang Weng、Jingang Wang、Xunliang Cai、Tao Gui、Zuxuan Wu、Qi Zhang、Xihua、Xihua Xiang、Xihua Xiang、Xihua Xiang、Xihua Xiang、Xihua Xiang、Xihua Xiang Xiang Yuang、Yang、Xihua Xiang Xiang、Yang、Yang、Yang Yuang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Yang、Zhang、Zhang、Yang、Zhang、Yang、Yang、Zhang、Yang、Yang
  • https://mathcritique.github.io/
• On Designing Effective RL Reward at Training Time for LLM Reasoning
  • 問合せ先: Rémi Coulom.
• Generative Verifiers: Reward Modeling as Next-Token Prediction
  • Lunjun Zhang、Arian Hosseini、Hritik Bansal、Mehran Kazemi、Aviral Kumar、Rishabh Agarwal
• Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
  • Amrith Setlur、Nagpal、Adam Fisch、Xinyang Geng、ジェイコブEisenstein、Rishabh Agarwal、Alekh Agarwal、Jonathan Berant、Aviral Kumar
• Improve Mathematical Reasoning in Language Models by Automated Process Supervision
  • Liangchen Luo、Yinxiao Liu、Rosanne Liu、Samrat Phatale、Harsh Lara、Yunxuan Li、Lei Shu、Yun Zhu、Lei Meng、Jiao Sun、Abhinav Rastogi
• Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
  • Peiyi Wang、Lei Li、Zhihong Shao、R.X。 Xu、Damiai Dai、Yifei Li、Deli Chen、Y.Wu、Zhifang Sui
• Planning In Natural Language Improves LLM Search For Code Generation
  • Evan Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, Will Song, Vaskar Nath, Ziwen Han, Sean Hendryx, Summer Yue, ヒュー・ザン
• PROCESSBENCH: Identifying Process Errors in Mathematical Reasoning
  • Chujie Zheng、Zhenru Zhang、Beichen Zhang、Runji Lin、Keming Lu、Bowen Yu、Dayiheng Liu、Jingren Zhou、Juanyang Lin
• AFlow: Automating Agentic Workflow Generation
  • Jiayi Zhang、Jinyu Xiang、Zhaoyang Yu、Fengwei Teng、Xiaqi Chen、Mingchen Zhuge、Xin Cheng、Sir Hong、Jinlin Wang、Bingnan Zheng、Bang Liu、Yyu Luo、Chenglin Wu
• Interpretable Contrastive Monte Carlo Tree Search Reasoning
  • Zitian Gao、Boee Niu、Xuzheng He、Haotian Xu、Hongzhang Liu、Aiwei Liu、Xuming Hu、Lijie Wen
• Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
  • Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, Rafael Rafailov
• Mixture-of-Agents Enhances Large Language Model Capabilities
  • ジュンリン・ワン、Jue Wang、Ben Athiwaratkun、Ce Zhang、James Zou
• Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models
  • Zhiyuan Hu、Chumin Liu、Xidong Feng、Yilun Zhao、See-Kiong Ng、Anh Tuan Luu、Junxian He、Pang Wei Koh、Bryan Hooi
• Advancing LLM Reasoning Generalists with Preference Trees
  • Lifan Yuan、Ganqu Cui、Hanbin Wang、Ning Ding、Xingyao Wang、Jia Deng、Boji Shan ら。
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
  • Ye Tian、Baolin Peng、Linfengソング、Lifeng Jin、Dian Yu、Haitao Mi、Dong Yu。
• AlphaMath Almost Zero: Process Supervision Without Process
  • Guoxin Chen、Minpeng Liao、Chengxi李、海ファン。
• ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
  • Dan Zhang、Sining Zhoubian、Yisong Yue、Yuxiao Dong、およびJie Tang。
• Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search
  • Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, Dacheng Tao
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
  • Yuhao Dong、Zuyan Liu、ハイロングサン、Jingkang Yang、ウィンストンハ、Yongming Rao、Ziwei Liu
• MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time
  • Jikun Kang、Xin Zhe Li、Xi Chen、Amirreza Kazemi、Qianyi Sun、Boxing Chen、Dong Li、Xu He、Quan He、Feng Wen、Jianye Hao、Juan Yao。
• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning
  • Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P. Lillicrap, 河口健二, マイケル・シーヘ.
• When is Tree Search Useful for LLM Planning? It Depends on the Discriminator
  • Ziru Chen、マイケル・ホワイト、レイモンド・ムーニー、アリ・ペイニ、ユ・ス、ファン・サン
• Chain of Thought Empowers Transformers to Solve Inherently Serial Problems
  • Zhiyuan李、香港劉、Denny Zhou、Tengyu Ma。
• To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
  • Zayne Sprague、Fangcong Yin、Juan Diego Rodriguez、Dongwei Jiang、Mana Wadhwa、Prasann Singhal、Xinyu Zhao、Xi Ye、Kyle Mahowald、Greg Durrett
• Do Large Language Models Latently Perform Multi-Hop Reasoning?
  • Sohee Yang、Elena Gribovskaya、Nora Kasner、Mor Geva、Sebastian Riedel
• Chain-of-Thought Reasoning Without Prompting
  • Xuezhi 王, デニー Zhou
• Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
  • Zhenting Qi、Mingyuan Ma、Jiahang Xu、Li Lyna Zhang、ファンヤン、マオ・ヤン
• Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
  • Xuan Zhang、Chao Du、Tianyu Pang、Qian Liu、Wei Gao、Min Lin
• ReFT: Reasoning with Reinforced Fine-Tuning
  • Trung Quoc Luong、Xinbo Zhang、Zhanming Jie、Peng Sun、Xiaoran Jin、Hang Li
• VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment
  • Amirhossein Kazemnejad、Milad Aghajohari、Eva Portelance、Alessandro Sordoni、Siva Reddy、Aaron Courville、Nicolas Le Roux
• Stream of Search (SoS): Learning to Search in Language
  • Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, Noah D. グッドマン
• GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
  • Iman Mirzadeh、Kevan Alizadeh、Hooman Shahrokhi、Samy Bengio、Mehrdad Farajtabar
• Evaluation of OpenAI o1: Opportunities and Challenges of AGI
  • Tianyang Zhong、Zhengliang Liu、Yi Pan、Yifan Zhou、Yifan Zhou、Sizhe Liang、Zihao Wu、Yanjun Lyu、Peng Shu、Xiaowei Yu、Chao Cao、Hanqi Jiang、Hanxu Chen、Yiwei Li、Junhao Chen、等。
• Evaluating LLMs at Detecting Errors in LLM Responses
  • Kamoi、Sarkar Snigdha Sarathi Das、Ranran Haoran Zhang、Sujeeth Reddy Vummanthala、Salika Dave、Shaobo Qin、Arman Cohan、Wenpeng Yin、Rui Zhang、Ran Haoran Zhang、Sujeeth Reddy Vummanthala、Salika Dave、Arman Cohan、Wenpeng Yin、Rui Zhang
• On The Planning Abilities of OpenAI’s o1 Models: Feasibility, Optimality, and Generalizability
  • Kevin Wang、Junbo李、Neel P. Bhatt、Yihan Xi、Qiang Liu、Ufuk Topcu、Zhangyang Wang
• Not All LLM Reasoners Are Created Equal
  • Arian Hosseini、Alessandro Sordoni、ダニエル・富山、Aaron Courville、Rishabh Agarwal
• LLMs Still Can’t Plan; Can LRMs? A Preliminary Evaluation of OpenAI’s o1 on PlanBench
  • Karthik Valmeekam、Kaya Stechly、Subbarao Kambhampati
• A Comparative Study on Reasoning Patterns of OpenAI’s o1 Model
  • Siwei Wu、Zhongyuan Peng、Xinrun Du、Tuney Zheng、Minghao Liu、Jialong Wu、Jiachen Ma、Yizhi Li、Jian Yang、Wangchunshu Zhou、Qunshu Lin、Junbo Zhao、Zhaoxiang Zhang、Wenhao Huang、Ge Zhang、Chenghua Lin、J.H. Liu
• Thinking LLMs: General Instruction Following with Thought Generation
  • Tianhao Wu、Janice Lan、Weizhe Yuan、Jiantao Jiao、Jason Weston、Sainbayar Sukhbaatar
• Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems
  • ジュンZhao、Jingqiのトン、Yurong Mou、Ming Zhang、Qi Zhang、Xuanjing Huang
• V-STaR: Training Verifiers for Self-Taught Reasoners
  • Arian Hosseini、Xingdi Yuan、Nikolay Malkin、Aaron Courville、Alessandro Sordoni、Rishabh Agarwal
• CPL: Critical Plan Step Learning Boosts LLM Generalization in Reasoning Tasks
  • Tianlong Wang、Junzhe Chen、Xueting Han、Jing Bai
• RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
  • Tianhao Wu、Janice Lan、Weizhe Yuan、Jiantao Jiao、Jason Weston、Sainbayar Sukhbaatar
• Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
  • Chaojie Wang, Yanchen Deng, Zhiyi Lyu, Liang Zeng, Jujie He, Shuicheng Yan, ボアン
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning
  • Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, Hongshan Liu, Hongshan Liu, 香港, 中国

2023

• Let’s Verify Step by Step
  • ハンター・ライトマン、ビネット・コサラジュ、ユラ・ブルダ、ハリ・エドワーズ、ボウン・ベーカー、テディ・リー、ヤン・ライク、ジョン・シュルマン、イリヤ・スツクヴァー、カール・コブベ
• V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
  • Penghao 呉, サイニング Xie
• Training Chain-of-Thought via Latent-Variable Inference
  • デュ・パイン、マシュー・D・ホフマン、デビッド・ドハン、スホルト・ドグラス、チュン・アン・ル、アロン・パリシ、パベル・ソウンツフ、チャールズ・ストン、シャラド・ヴィクラム、ライフ・ア。サウラス
• Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training
  • Xidong Feng、Ziyu Wan、Muning Wen、Stephen Marcus McAleer、Ying Wen、Weinan Zhang、Jun Wang、Ying Wen
• OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning
  • Fei Yu、Anningzhe Gao、Benyou Wang 氏
• Reasoning with Language Model is Planning with World Model
  • Shibo Hao、Yi Gu、Haodi Ma、Joshua Jiahua Hong、Zhen Wang、Daisy Zhe Wang、Zhiting Hu
• Don’t throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
  • 劉、Jiacheng、Andrew Cohen、Ramakanth Pasunuru、Yejin Choi、Hannaneh Hajishirzi、およびAsli Celikyilmaz。
• Certified reasoning with language models
  • Gabriel Poesia, Kanishk Gandhi, Eric Zelikman, Noah D. グッドマン
• Large Language Models Cannot Self-Correct Reasoning Yet
  • Jie Huang、Xinyun Chen、Swaroop Mishra、Huaixiu Steven Zheng、Adams Wei Yu、Xinying Song、Denny Zhou

2022

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • Jason Wei、Xuezhi Wang、Dale Schuurman、Maarten Bosma、Brian Ichter、Fei Xia、Ed Chi、Quoc Le、Denny Zhou
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • Xuezhi 王, Jason Wei, Dale Schuurman, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, Denny Zhou
• Self-critiquing models for assisting human evaluators
  • ウィリアム・サウンドス、キャサリン・イェ、ジェフ・ウー、スティーブン・ビルス、ロング・オウヤン、ジョナサン・ウォード、ヤン・ライク
• Chain of Thought Imitation with Procedure Cloning
  • Mengjiao Yang、Dale Schuurmans、Piper Abbeel、Onyir Nachum。
• STaR: Bootstrapping Reasoning With Reasoning
  • Eric Zelikman、Yuhuai Wu、Jesse Mu、ノアD。 グッドマン
• Solving math word problems with processand outcome-based feedback
  • Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, イリーナ・ヒギンズ

2021

• Training Verifiers to Solve Math Word Problems
  • カール・コブベ、ヴィネット・コサラジュ、モハンマド・バーリアン、マーク・チェン、ヘウ・ジュン、ルカス・カイザー、マティアス・プラプナート、ジェリー・トゥレク、ジェイコブ・ヒルトン、リチシロ・ナカノ、クリストファー・メッセ、ジョン・シュルマン
• Scalable Online Planning via Reinforcement Learning Fine-Tuning
  • Arnaud Fickinger、Hengyuan Hu、Brandon Amos、Stuart Russell、Noam Brown。
• Scaling Scaling Laws with Board Games
  • Andy L. ジョーンズ.
• Show Your Work: Scratchpads for Intermediate Computation with Language Models
  • Maxwell Nye、Anders Johan Andreassen、Guy Gur-Ari、Henryk Michalewski、ジェイコブ・オースティン、David Bieber、David Dohan、Aitor Lewkowycz、Maarten Bosma、David Luan、Charles Sutton、Augustus Odena

2021年以前

• Improving Policies via Search in Cooperative Partially Observable Games
  • アダム・ラーラー、Hengyuan Hu、Jakob Foerster、Noamブラウン。
• Generative Language Modeling for Automated Theorem Proving
  • Stanislas Polu, イリヤ・ススキバー
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
  • デビッド・シルバー、トーマス・ハバート、ジュリアン・シュリットウィザール、イオアニス・アントヌーグル、マタイ・レイ、アーサー・グエズ、マルク・ランクート、ローラン・シファー、ダルシャン・クマール、Thore Graepel、ティモシー・リリックラップ、カレン・シモンヤン、Demis Hassabis。