LLM Strawberry

Source: https://github.com/hijkzzz/Awesome-LLM-Strawberry at 3f17fc9ee49c
License: Apache-2.0
Read the English original

This edition is an AI-assisted translation.

LLM Strawberry（OpenAI o1）研究精选

这里汇集了关于 OpenAI Strawberry（o1）及推理能力 的研究论文和博客。 这是研究论文和博客的集合 OpenAI 草莓(o1)和原因. 本仓库将持续更新，追踪大语言模型推理领域的前沿进展。 并将不断更新存储器,以跟踪LLM Reasoning的前沿.

OpenAI 文档

• https://platform.openai.com/docs/guides/reasoning
• 

新闻

• [开放AI] Introducing deep research
• [开放AI] o3 preview & o3 mini
• [开放AI] Introducing ChatGPT Pro
• [谷歌深明] Gemini 2.0 Flash Thinking
• [伊利亚·苏茨克维] AI with reasoning power will be less predictable
• [塞米安分析] Scaling Laws – O1 Pro Architecture, Reasoning Training Infrastructure, Orion and Claude 3.5 Opus “Failures”
• [深 DeepSeek-R1-Lite-Preview is now live: unleashing supercharged reasoning power!
• [月 数学对标o1系列，搜索再次进化，Kimi 新推理模型与你一起拓展智能边界
• [月 Kimi 发布视觉思考模型 k1，多项理科测试行业领先
• [实习生] 强推理模型书生InternThinker开放体验：自主生成高智力密度数据、具备元动作思考能力
• [新智元] 万字独家爆光，首揭o1 pro架构！惊人反转，Claude 3.5 Opus没失败？

博客

• [开放AI] Learning to Reason with LLMs
• [开放AI] OpenAI o1-mini Advancing cost-efficient reasoning
• [开放AI] Finding GPT-4’s mistakes with GPT-4
• [ARC-AGI] (英语). OpenAI o3 Breakthrough High Score on ARC-AGI-Pub
• [安 Building effective agents
• [hijkzzz] (英语). Stabilizing MoE RL Without Router Replay: The Online IcePop Solution
• [hijkzzz] (英语). REINFORCE++-baseline is all you need in RLVR
• [hijkzzz] (英语). Exploring OpenAI O1 Model Replication
• [纳坦·兰伯特] OpenAI’s Strawberry, LM self-talk, inference scaling laws, and spending more on inference
• [纳坦·兰伯特] Reverse engineering OpenAI’s o1
• [安德烈斯·斯图尔穆勒, Jungofthewon] Supervise Process, not Outcomes
• [努哈·齐里] Have o1 Models Cracked Human Reasoning?
• [里沙布阿加华] Improving LLM Reasoning using Self-generated data: RL and Verifiers
• [魏沈] Generalization Progress in RLHF: Insights into the Impact of Reward Models and PPO
• [多米尼克069] Codeforces - Analyzing how good O1-Mini actually is
• [提伯布拉荷] Summary of what we have learned during AMA hour with the OpenAI o1 team

会谈

• [诺姆布朗] Parables on the Power of Planning in AI: From Poker to Diplomacy
• [诺姆布朗] OpenAI o1 and Teaching LLMs to Reason Better
• [洪元聪. Don’t teach. Incentivize.

课程

• [深学. AI] (英语). Reasoning with o1

微博

OpenAI 开发者

• All the questions addressed by the API team during the December 17, 2024 AMA
• We’re hosting an AMA for developers from 10–11 AM PT today.
• Today we previewed Reinforcement Fine-Tuning
• 

诺姆·布朗

• 
• 
• 
• 

韦杰森

• 
• 
• 

其他人员

• 
• 
• 

开源

模型

• [阿利巴巴昆 Qwen3
• [阿利巴巴昆 QwQ
• [阿利巴巴昆 QvQ
• [深 DeepSeek R1
• [荷兰] Nemotron-Research-Reasoning-Qwen-1.5B
• [天空工作] Skywork R1V2
• [rLLM] DeepScaler
• [诺瓦斯基] Sky-T1
• [气声] O1 Replication Journey: A Strategic Progress Report
• [OpenO1小组] Open-Source O1
• [十 DRT-o1
• [阿里巴巴] Marco-o1
• [CUHK-SZ] HuatuoGPT-o1

密码库

• [开放高频组] OpenRLHF
• [开放高频组] REINFORCE++ | REINFORCE++-baseline | Code
• [诺瓦斯基-AI] SkyRL
• [RUCAIBOX] STILL: Slow Thinking with LLMs
• [掌声] Simple Reinforcement Learning for Reasoning
  • 这是复制DeepSeek-R1-Zero和DeepSeek-R1关于数据有限的小模型的培训
• [无 Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
• [步风] Open-Reasoner-Zero
• [潮台大 LMM-R1
• [莫达尔明 MM-EUREKA: Exploring Visual Aha Moment with Rule-based Large-scale Reinforcement Learning
• [R1-V小组] R1-V
• [LLAMA-事实组] EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework
• [阿里巴巴] ROLL | Paper
• [海上人工智能实验室] Dr. GRPO
• [伯克利AI研究] TinyZero
• [Maitrix.org] (英语). LLM Reasoners

论文

format:
- [title](paper link) [links]
  - author1, author2, and author3...
  - publisher
  - code
  - experimental environments and datasets

O1型号技术报告

• INTELLECT-3: Technical Report
  • 主智能团队
• DeepSeek V3.2
  • 深搜索 AI
• MiMo-V2 Flash
  • 晓美
• Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
  • 纳米比亚
• Qwen3 Technical Report
  • Quen 团队
• Scaling Agents via Continual Pre-training
  • Quen 团队
• WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents
  • Quen 团队
• Towards General Agentic Intelligence via Environment Scaling
  • Quen 团队
• Magistral
  • 迷雾AI
• ERINE-4.5
  • 贝都
• LongCat Flash
  • 梅东
• GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
  • 大赦国际
• Seed Thinking v1.5
  • 种子
• DeepSeek-V3 Technical Report
  • 深层搜索
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
  • 深搜索 AI
• DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
  • 深搜索 AI
• MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
  • 迷你麦克斯
• Kimi k2: Open agentic intelligence
  • 月光热
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
  • 月光热
• KIMI-VL TECHNICAL REPORT
  • 月光热
• Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning
  • 月光和努米纳
• Llama-Nemotron: Efficient Reasoning Models
  • 纳米比亚
• Skywork Open Reasoner 1 Technical Report
  • 天线工作

2025

• Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
  • 纳米比亚
• QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory
  • Quen 团队
• DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
  • 深层搜索-AI
• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices
  • Quen 团队
• The Art of Scaling Reinforcement Learning Compute for LLMs
  • 德夫里特·卡特里、洛维什·马达安、里沙布·蒂瓦里、拉希特·班萨尔、赛苏里亚·杜夫武里、曼齐尔·扎希尔、因德吉特·德希隆、戴维·布兰德丰布雷纳、里沙布·阿加尔瓦尔
• BroRL: Scaling Reinforcement Learning via Broadened Exploration
  • 胡建,,刘明杰,吕西明,方武,蔡德哈查维,史志卓,叶珍崔,巴甫洛·莫尔恰诺夫,杨俊,扬考茨,易东.
• Why Language Models Hallucinate
  • 开放AI
• rStar2-Agent: Agentic Reasoning Technical Report
  • 宁翔, Yi飞刘, Yi,李林娜张,魏江徐,新玉关,蒲泽张,炳城董,徐东周,鲍文章, Y新,齐明苗,李思嘉,范阳.
• RL Is Neither a Panacea Nor a Mirage: Understanding Supervised vs. Reinforcement Learning Fine-Tuning for LLMs
  • (原始内容存档于2018-09-26). Hangzhan Jin, Sicheng, Sifan Wu, Mohammad Hamdaqa
• DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization
  • 谢少杰,余宝,余鲁,卢旭,陶力,文豪,黄少杰,山波程,卢汝霖,王少豪.
• Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning
  • 瓦伊什纳维·什里瓦斯塔瓦、艾哈迈德·阿瓦达拉、维迪沙·巴拉钱德兰、希瓦姆·加尔格、哈基拉特·贝尔、迪米特里斯·帕皮利奥普洛斯
• ProRL V2 - Prolonged Training Validates RL Scaling Laws
  • 侯建,刘明杰,史志卓,吕西明,新东,巴甫洛·莫尔恰诺夫,叶真崔,扬考茨,易东.
• Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning
  • 齐贺刘,刘家顺,贺彦诚,王伟臣,刘建成,凌潘,胡新玉,肖潘X,朱黄,胡建 Hu,黄盛义,西兰杨,王家康,温波苏,薄 Zheng.
• Group Sequence Policy Optimization
  • 郑楚杰,刘世uan,李明泽,陈兴辉,鲍文瑜,常高,启当,刘玉秋,吕门,安阳,周金格,容阳林
• Your Efficient RL Framework Secretly Brings You Off-Policy RL Training
  • 冯尧,刘丽媛,丁怀章,程玉东,景波翔,建丰高
• Gemini 2.5 Pro Capable of Winning Gold at IMO 2025
  • (原始内容存档于2018-03-09). Yichen Huang, Lin F. Yang.
• DeepSWE: Training a Fully Open-sourced, State-of-the-Art Coding Agent by Scaling RL
  • 一起来
• OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
  • 曾志旺,范 Zhou,薛凤丽,彭非刘
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
  • 刘明杰,史志卓,吕西明,胡建军,新东,叶真崔,杨考茨,易东
• REINFORCE++: An Efficient RLHF Algorithm with Robustness to Both Prompt and Reward Models
  • (原始内容存档于2017-09-21). Jian Hu, Jason Klein Liu, Wei Shen
  • 代码 : REINFORCE++-baseline
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
  • Quen 团队
• The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
  • 新余朱氏,孟州夏氏,浙北卫氏,魏林陈氏,丹齐陈氏,余孟氏.
• Thinking with Generated Images
  • 伊森·切尔恩,朱林胡,施泰菲·切尔恩,西齐·库,贾迪·苏,颜马,邓志杰,彭非刘
• Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning
  • 亚历克斯·苏, 王豪泽, 魏明任, 方正林, 陈文虎.
• DeepEyes: Incentivizing “Thinking with Images” via Reinforcement Learning
  • 齐伟 Zheng,迈克尔·杨,杰克·洪,陈晓昭,郭海旭,黎阳,曹沈,邢玉.
• Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging
  • 陈世齐,张金汉,汤尧 Zhu,魏刘,西阳高,苗X,万灵李,淳化贺
• QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
  • 范齐万,益州沈,辽,盈城史,陈立, Zi阳,吉张,飞黄,金格伦周,明燕.
• Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving
  • 新吉迈,郝天旭,邢W,王维农,张英英,张文强.
  • 代码 : https://github.com/yyht/openrlhf_async_pipline
• A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
  • 魏X,姚家瑞,许玉辉,薄一波,雷旺,多延沙胡, Jun南李,南江,汤章,蔡明X,韩泽东
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example
  • 王义平,杨庆ing,曾志远,黎良任,卢卡斯刘,鲍林鹏,郝程,薛海贺,熊旺,高建丰,陈伟祖,王修章,西蒙绍莱杜,叶龙沈
• Process Reward Models That Think
  • 穆罕默德·哈利法、里沙布·阿加尔瓦尔、拉贾努根·洛热斯瓦兰、杰克尧姆·金、郝鹏、李月娥、李洪拉克、卢旺
• M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
  • 王俊松,李文鼎,丹尼尔·帕里奥塔,丹尼尔·里特,亚历山大·M·拉什,崔道
• A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility
  • 安德烈亚斯·霍赫勒纳特、哈迪克·巴特纳加尔、维沙尔·乌丹达劳、塞缪尔·阿尔巴尼、阿梅亚·普拉布、马蒂亚斯·贝斯盖
• Concise Reasoning via Reinforcement Learning
  • 梅赫迪·法特米、巴纳夫谢·拉菲埃、唐明杰、卡蒂克·塔拉马杜普拉
• VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks
  • 余尧,余丰元,邱英玉,小琴祖,鲁菲 Zhu,文渊徐,陈家泽,程义旺,天天范,郑义仁杜, 鹏伟,高洪刘,刘俊凯,刘凌君,海滨林, Zh志林,宝乐马等.
• Inference-Time Scaling for Generalist Reward Modeling
  • 刘子俊,王佩义,许跑新,白龙马,钟鲁安,彭力,杨刘,于武
• JudgeLRM: Large Reasoning Models as a Judge
  • 牛陈,胡志远,邹庆云,吴嘉英,王Q,布莱恩·胡伊,宾生贺
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
  • 邱英瑜,郑 Zhang,罗菲 Zhu,俞凤元,萧琴祖,余岳,天天范,高鸿刘,凌君刘,新刘,海滨林等.
• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
  • 乔纳斯·盖平(Jonas Geiping),肖恩·麦克莱什(Sean McLeish),尼尔·贾因(Neel Jain),约翰·基兴鲍尔(John Kirchenbauer),西德哈斯·辛格(Siddharth Singh),布莱恩·R·巴托尔森(Brian R. Bartoldson),巴维亚·凯尔库拉(Bhavya Kailkhura),阿比纳夫·巴泰莱(Abhinav Bhatele),汤姆·戈尔德斯坦(Tom Goldstein).
• Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning
  • 尤晓奎,马修·Y·R·杨,阿姆里斯·塞特卢尔,刘易斯·通斯特尔,爱德华·伊曼纽尔·比青,罗斯兰·萨拉胡蒂季诺夫,阿维拉尔·库马尔
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
  • 华东宋,江津豪,英强民,陈杰,齐本陈,韦恩新赵,雷方,智荣文
• Visual-RFT: Visual Reinforcement Fine-Tuning
  • 刘子玉,泽义孙,余杭赞,萧义东,余杭曹,郝东堂,大华林,王佳琪
• Introducing Visual Perception Token into Multimodal Large Language Model
  • 龙鹏羽,马新年,王银曹
• Scaling Test-Time Compute Without Verification or RL is Suboptimal
  • 阿姆里斯·塞特卢尔、尼韦德·拉贾拉曼、谢尔盖·莱文、阿维拉尔·库马尔
• LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
  • 李达诚,曹世义,泰勒·格里格斯,刘少奇,iang西莫,谢希尔·帕蒂尔,马泰·扎哈里亚,约瑟夫·E·冈萨雷斯,伊万·斯托伊卡
• Demystifying Long Chain-of-Thought Reasoning in LLMs
  • (原始内容存档于2018-09-21). Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, Xiang Yue.
• LIMR: Less is More for RL Scaling
  • 李雪峰,邹朝阳,刘鹏非
• LIMO: Less is More for Reasoning
  • 叶一新,黄振,杨晓,伊森·切尔恩,夏世杰,刘鹏非
• s1: Simple test-time scaling
  • Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei, Hanneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, Tatsunori 桥本贤一,
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
  • 朱天泽,余学祥,吉汉杨,盛邦塘,赛宁谢,戴尔·舒尔曼斯,高克五·勒,谢尔盖·莱文,易马.
• Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  • 曾宇侯,新乐,吕i,张家杰,余江李,齐君姚,胡安子李,姬唐,余晓东
• Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
  • 毛豪沈,广涛曾,曾廷琦,张伟宏,陈振芳,魏鲁,格雷戈里·沃内尔,苏波罗·达斯,大卫·考克斯,川甘
• Distillation Quantification for Large Language Models
  • 孙伯文 李, 周俊廷, 曹爱等.
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
  • 关新玉,李莲娜,张一飞,刘宁翔,尤兰孙,易 Zhu,范阳,毛杨
• Evolving Deeper LLM Thinking
  • 李宽惠,伊恩·菲舍尔,岳华武,戴夫·马伍德,舒米特·巴卢贾,戴尔·舒尔曼斯,陈新云
• The Lessons of Developing Process Reward Models in Mathematical Reasoning
  • 曾鲁章,朱杰 Zheng,杨zh吴,北陈章,龙济林,鲍文玉,戴彦刘,龚格伦周, Jun阳林
• Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
  • 维奥莱特·西安,查理·斯内尔,卡尼什克·甘地,阿隆·阿尔巴拉克,阿尼凯特·辛格,蔡斯·布拉格登,杜伊·蓬,拉斐尔·拉法伊洛夫,内森·利勒,达科他·马汉,路易·卡斯特里卡托,扬-菲利普·弗兰肯,尼克·哈伯,切尔西·芬
• PRMBENCH: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
  • 明阳宋氏,昭陈苏氏,小叶邱氏,贾维周氏,余程氏.
• Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
  • 宜凡杜, 齐康刘, 宜凡李, 韦恩新赵, 于奇侯, 炳宁王, 魏鹏陈, 郑刘, 曾荣王, 姬荣文.
• Imagine while Reasoning in Space: Multimodal Visualization-of-Thought
  • 李成祖,吴文山,张万尧,颜霞,毛少光,李东,伊万·武利奇,傅鲁伟
• LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs
  • 穆罕默德·祖姆里、让·拉胡德、拉乌·穆罕默德·安维尔、希沙姆·乔拉卡尔、伊万·拉普特夫、穆巴拉克·沙阿、法哈德·沙巴兹汗、萨勒曼·汗
• From chaos to order: The atomic reasoner framework for fine-grained reasoning in large language models
  • 刘晋义,颜 Zheng,荣成,齐玉武,魏国,飞妮,河滨良,易福元,杭玉毛,傅正章,建义豪.

2024

• Deliberative alignment: reasoning enables safer language models
  • 开放AI
• MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
  • 陈俊,尼尔·乔杜里,奥利弗·贾夫,詹姆斯·昂,达内·谢尔本,埃文·梅斯,朱利奥·斯塔拉斯,凯文·刘,莱昂·马克辛,特雅尔·帕特沃德汉,莉莉安·温,亚历山大·米德里
• From Medprompt to o1: Exploration of Run-Time Strategies for Medical Challenge Problems and Beyond
  • 斯科特·麦金尼
• LLM Critics Help Catch LLM Bugs
  • 纳特·麦克阿莱塞、雷·迈克尔·波科尼、胡安·费利佩·塞隆·乌里韦、埃夫根尼亚·尼蒂申斯卡亚、马贾·特雷巴茨、扬·莱克
• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
  • 余学祥,郝白,林志鹏,贾宜潘,盛邦塘,冀非周,阿兰素尔,赛宁谢,延乐春,易马,谢尔盖·莱文.
• ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
  • 郝山沈,康家赵,天城赵,鲁能徐,齐伦张,明威朱,建威殷.
• Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
  • 王文彬,梁鼎,米彦曾,夏斌周,李沈,龙罗,达诚陶
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
  • 查理·斯内尔 李杰洪 许开文 阿维拉尔·库马尔
• An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
  • 吴阳贞, 孙志清, 李尚达, 肖恩·韦勒克, 杨义明
• Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling
  • 赫里蒂克·班萨尔、阿里安·侯赛尼、里沙布·阿加尔瓦尔、维尼·特兰、梅赫兰·卡泽米
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
  • 布拉德利·布朗,乔丹·尤拉夫斯基,瑞安·埃利希,罗纳德·克拉克,Quoc V. Le,克里斯托弗·雷,阿扎莉娅·米尔霍塞尼
• Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems
  • 英强敏,齐平陈,金豪江,吉陈,贾登,易文虎,义鲁唐,贾鹏王,萧学成,华东宋,韦恩新赵,郑刘, Zh荣王,吉荣文.
• Training Language Models to Self-Correct via Reinforcement Learning
  • 阿维拉尔·库马尔、文森特·卓昂、里沙布·阿加尔瓦尔、易·苏、约翰·D·联合雷耶斯、阿维·辛格、凯特·鲍姆利、沙利克·伊克巴尔、科尔顿主教、丽贝卡·罗洛夫斯、莱姆·张、凯·麦金尼、迪沙·什里瓦斯塔瓦、科斯敏·帕杜拉鲁、乔治·塔克、多伊娜·普雷库普、费拉尔·贝赫巴哈尼、阿莱克桑德拉·浮斯特
• Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
  • 陈兴玉,许嘉豪,田良,贺志伟,彭建辉,丁玉麟,林丰松,刘秋芝,孟非周, Zhu绍生,张瑞旺,赵鹏图,海涛美,董玉.
• MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes
  • 阿斯玛·本·阿巴查,文怀义姆,于俊富,昭义孙,梅利哈·叶提斯根,费霞,托马斯·林
• Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
  • 安杨,张碧琴,宾uan慧,博非高,鲍文玉,程鹏丽,戴彦eng,刘建洪图,龚格伦周,容阳林,陆克明,明丰薛,龙济林,天佑刘,兴章任,振鲁张.
• Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
  • 曾玉侯,彭范杜,依林牛,郑晓杜,奥汉曾,小刘,明莉黄,洪宁王,吉唐,余晓东
• Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering
  • 新燕关,柳江刘,吕新玉,博奇曹,本贺,韩仙培,黎晨,杰娄,鲍文羽,姚杰吕,洪玉林
• Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective
  • 曾志远,秦尧成,张尧殷,薄汪,李世民,云华周,齐鹏郭,黄uan京,西鹏邱
• Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
  • 埃里克·泽利克曼,乔治·哈里克,伊吉亚·肖,瓦鲁纳·贾亚西里,尼克·哈伯,诺亚·D·古德曼
  • https://github.com/ezelikman/quiet-star
• Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
  • 许志恒,丁文杨,黄 J,唐家福,关玉丽,易文鼎,魏贺,博阳宏,石汉道,文渊赞,萧汪,鲁 Zheng,陶池,萧伟世,叶涛泽,罗x文,王 J,薛兰洲,陶桂,朱x武,齐 Zhang,西平邱,黄uan,余广江.
  • https://mathcritique.github.io/
• On Designing Effective RL Reward at Training Time for LLM Reasoning
  • 高家uan,许书生,叶文杰,刘伟林,贺楚义,韦福,梅志玉,王广珠,易武
• Generative Verifiers: Reward Modeling as Next-Token Prediction
  • 张伦君、阿里安·侯赛尼、赫里迪克·班萨尔、梅赫兰·卡泽米、阿维拉尔·库马尔、里沙布·阿加尔瓦尔
• Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
  • 阿姆里斯·塞特卢尔、奇拉格·纳格帕尔、亚当·菲施、新阳耿、雅各布·艾森斯坦、里沙布·阿加尔瓦尔、阿莱赫·阿加尔瓦尔、乔纳森·贝兰特、阿维拉尔·库马尔
• Improve Mathematical Reasoning in Language Models by Automated Process Supervision
  • 罗良琴,刘燕晓,罗珊娜,刘桑拉特·费塔勒,哈什·拉拉,云霄李,雷书,云 Zhu,雷孟,焦晨,阿比西纳夫·拉斯托吉.
• Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
  • 王培义,李立三,薛志红,罗学旭,戴迈戴,李义非,陈德利,叶武,苏志芳
• Planning In Natural Language Improves LLM Search For Code Generation
  • 埃文·王,费德里科·卡萨诺,凯瑟琳·吴,云峰白,威尔·宋,瓦斯卡·纳特,齐文·汉,肖恩·亨德里克斯,夏岳,休·张
• PROCESSBENCH: Identifying Process Errors in Mathematical Reasoning
  • 朱杰郑,曾鲁章,贝琴张,林伦次,吕克明,鲍文羽,戴彦刘,周金格伦, Jun阳林
• AFlow: Automating Agentic Workflow Generation
  • 张家义,龚玉祥,赵阳瑜,冯伟腾,陈ion辉,陈家qi,明诚诸葛亮,新成,石瑞宏,龚林汪,炳南 Zheng,邦刘,饶玉罗,程林吴.
• Interpretable Contrastive Monte Carlo Tree Search Reasoning
  • 齐天高,博义牛,许曾贺,郝天徐,刘鸿章,刘爱伟,许明胡,李杰文
• Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
  • 普拉纳夫·普塔、埃德蒙·米尔斯、纳曼·加格、苏米特·莫特瓦尼、切尔西·芬、迪维扬什·加格、拉斐尔·拉法伊洛夫
• Mixture-of-Agents Enhances Large Language Model Capabilities
  • 王俊林,王俊惠,本·阿提瓦拉特昆,蔡章,詹姆斯·邹
• Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models
  • 胡志远,刘楚民,谢东峰,赵义伦,吴锡谦,安图安路,贺俊祥,庞伟高,布莱恩·胡伊
• Advancing LLM Reasoning Generalists with Preference Trees
  • 李范元,甘泉桂,韩斌汪,宁定,王兴尧,贾登,博济山等.
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
  • 叶天,鲍林鹏,林福松,李生珍,丁玉,海涛美,董玉.
• AlphaMath Almost Zero: Process Supervision Without Process
  • 陈国新,廖敏鹏,程西丽,凯范.
• ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
  • 丹章,西宁周边,宜松岳, Yu晓东,吉唐.
• Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search
  • 万金姚,黄家兴,文豪吴,张静宜,易波汪,刘顺尧,王英杰,宋玉新,郝春峰,李沈,达春涛.
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
  • 余豪东,刘祖燕,海龙孙,杨静康,温斯顿胡,永明饶,刘子伟.
• MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time
  • 吉昆康,新浙李,习陈,阿米尔雷扎·卡泽米,Q义孙,拳陈,东李,徐贺,泉贺,冯文,建义郝,俊姚.
• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning
  • 尤西·谢(Yuxi Xie),阿尼鲁德·戈雅尔(Anirudh Goyal),温尧 Zheng,明-延·坎(Min-Yen Kan),蒂莫西·P·利利格拉普(Timothy P. Lillicrap),川口健二,迈克尔·谢(Michael Shieh).
• When is Tree Search Useful for LLM Planning? It Depends on the Discriminator
  • 陈齐鲁,迈克尔·怀特,雷蒙德·穆尼,阿里·帕亚尼,于秀,万晨
• Chain of Thought Empowers Transformers to Solve Inherently Serial Problems
  • 李志远,洪刘,邓尼周,滕玉马.
• To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
  • 蔡恩·斯普拉格,方康殷,胡安·迭戈·罗德里格斯,江东伟,玛尼亚·瓦德瓦,普拉桑·辛格尔,赵新宇,习艺,凯尔·马豪瓦尔德,格雷格·杜雷特
• Do Large Language Models Latently Perform Multi-Hop Reasoning?
  • 杨素熙,埃琳娜·格里博夫斯卡娅,诺拉·卡斯纳,莫尔·格瓦,塞巴斯蒂安·里德尔
• Chain-of-Thought Reasoning Without Prompting
  • 薛志旺; 周德尼.
• Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
  • 曾廷琦,马敏远,许佳章,李莲娜,张范阳,毛杨
• Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
  • X张, 曹都, 天玉庞, 刘Q, 魏高, 民林
• ReFT: Reasoning with Reinforced Fine-Tuning
  • 曾国藩,罗荣桓,张新波,詹明杰,彭晨,萧兰珍,杭丽.
• VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment
  • 阿米尔霍赛因·卡齐姆内贾德,米拉德·阿加霍哈里,伊娃·波特兰斯,亚历山德罗·索多尼,西瓦·雷迪,亚伦·库维尔,尼古拉·勒鲁
• Stream of Search (SoS): Learning to Search in Language
  • 卡尼什克·甘地,丹妮丝·李,加布里埃尔·格兰德,刘慕新,温森·程,阿奇特·夏尔马,诺亚·D·古德曼
• GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
  • 伊曼·米尔扎德、凯万·阿利扎德、胡曼·沙赫罗基、韦内尔·图泽尔、萨米·本焦、梅赫达德·法拉杰塔巴尔
• Evaluation of OpenAI o1: Opportunities and Challenges of AGI
  • 天阳忠,刘 Zheng,易潘,余通章,易范周,石良,齐豪武,燕君柳,彭舒,小维余,曹操,韩奇江,韩旭陈,易伟李,陈俊豪等.
• Evaluating LLMs at Detecting Errors in LLM Responses
  • 亮·卡莫伊, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Jenice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Ranran Haoran Zhang, Sujeeth Reddy Vummanthala, Salika Dave, Shaobo Qin, Arman Cohan, Wenpeng Yin, Rui Zhang. 互联网档案馆的存檔,存档日期2014-03-02.
• On The Planning Abilities of OpenAI’s o1 Models: Feasibility, Optimality, and Generalizability
  • 王凯文,李俊波,聂尔·P·巴特,易汉西,刘强,乌富克·托普库,张阳王
• Not All LLM Reasoners Are Created Equal
  • 阿利安·侯赛尼,亚历山德罗·索多尼,丹尼尔·托山,亚伦·库维尔,里沙布·阿加尔瓦尔
• LLMs Still Can’t Plan; Can LRMs? A Preliminary Evaluation of OpenAI’s o1 on PlanBench
  • 卡蒂克·瓦尔米卡姆、卡亚·斯特奇利、班巴帕提
• A Comparative Study on Reasoning Patterns of OpenAI’s o1 Model
  • 西魏武则天,ong延鹏,新伦杜氏,图尼 Zheng氏,明高刘氏,贾龙吴氏,贾chen马氏,义志李氏,建阳人,汪川舒周氏,昆舒林氏,淳波赵氏,昭祥张氏,文豪黄氏,革章氏,程华林氏,J·H·刘氏.
• Thinking LLMs: General Instruction Following with Thought Generation
  • 吴天豪,詹妮丝·兰,魏泽元,詹涛·焦,杰森·韦斯顿,赛因巴亚尔·苏赫巴托尔
• Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems
  • 赵俊,字静琪,汤荣茂,明朝政治人物,清朝政治人物.
• V-STaR: Training Verifiers for Self-Taught Reasoners
  • 阿利安·侯赛尼,兴迪·袁,尼古拉·马尔金,亚伦·库维尔,亚历山德罗·索尔多尼,里沙布·阿加尔瓦尔
• CPL: Critical Plan Step Learning Boosts LLM Generalization in Reasoning Tasks
  • 天龙王,陈俊杰,薛廷汉,景白
• RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
  • 吴天豪,詹妮丝·兰,魏泽元,詹涛·焦,杰森·韦斯顿,赛因巴亚尔·苏赫巴托尔
• Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
  • 王朝杰,颜琴登,李志礼,梁曾,贺汝杰,颜水泉,宝安
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning
  • 郝绍o, 沈儒Q, 韩晓, 广鲁宋, 卓芳宗, 列天王, 余刘, 洪生李

2023

• Let’s Verify Step by Step
  • 亨特·莱特曼,维尼特·科萨拉朱,尤拉·布尔达,哈里·爱德华兹,博文·贝克,泰迪·李,扬·莱克,约翰·舒尔曼,伊利亚·苏茨凯弗,卡尔·科布贝
• V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
  • (原始内容存档于2018-10-21). Penghao Wu, Saining Xie.
• Training Chain-of-Thought via Latent-Variable Inference
  • 杜·潘,马修·D·霍夫曼,大卫·多哈,肖尔托·道格拉斯,图安·勒,阿伦·帕里西,帕维尔·索恩佐夫,查尔斯·苏顿,沙拉德·维克拉姆,Rif A. Saurous
• Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training
  • 习东凤,齐玉万,慕宁文,史蒂芬·马库斯·麦凯勒,英文,魏南章,王俊
• OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning
  • 飞羽,高安宁杰,王本尧
• Reasoning with Language Model is Planning with World Model
  • 施宝豪,易国,郝迪马,约书亚·贾华宏,曾王,黛西·王,曾国藩.
• Don’t throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
  • 刘,贾琴,安德鲁·科恩,拉马坎特·帕苏努鲁,叶真崔,汉纳赫·哈吉希尔齐,阿斯利·切利基尔马兹.
• Certified reasoning with language models
  • 加布里埃尔·坡西亚、卡尼什克·甘地、埃里克·泽利克曼、诺亚·D·古德曼
• Large Language Models Cannot Self-Correct Reasoning Yet
  • 黄杰,陈新云,斯瓦罗普·米什拉,怀秀·史蒂文·郑,亚当斯·魏羽,辛英歌,德尼·周

2022

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • 杰森·韦,王雪志,戴尔·舒尔曼斯,马腾·博斯马,布莱恩·伊赫特,费霞,艾德·齐,高克·勒,德尼·周
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • 王雪志, 贾森伟, 戴尔·舒尔曼斯, 高克·勒, 埃德·齐, 夏兰·纳朗, 阿肯沙·周德瑞, 丹尼·周
• Self-critiquing models for assisting human evaluators
  • 威廉·桑德斯,凯瑟琳·叶赫,杰夫·吴,史蒂文·比尔斯,朗欧阳,乔纳森·沃德,扬·莱克
• Chain of Thought Imitation with Procedure Cloning
  • 孟子杨,戴尔·舒尔曼斯,皮耶特·阿比尔,奥菲尔·纳楚姆.
• STaR: Bootstrapping Reasoning With Reasoning
  • 埃里克·泽利克曼,伍玉怀,杰西·穆,诺亚·D·古德曼
• Solving math word problems with processand outcome-based feedback
  • 乔纳森·乌萨托,内特·库什曼,拉玛娜·库马尔,弗朗西斯·宋,诺亚·西格尔,莉萨·王,安东尼娅·克雷斯韦尔,杰弗里·伊尔文,伊琳娜·希金斯

2021

• Training Verifiers to Solve Math Word Problems
  • 卡尔·科布贝,维尼特·科萨拉朱,穆罕默德·巴伐利亚,马克·陈,希宇俊,卢卡斯兹·凯泽,马蒂亚斯·普拉珀特,杰里·特瑞克,雅各布·希尔顿,中野丽二郎,克里斯托弗·黑森,约翰·舒尔曼
• Scalable Online Planning via Reinforcement Learning Fine-Tuning
  • 阿尔诺·菲金格(Arnaud Fickinger),胡恒远(Hengyuan Hu),布兰登·阿莫斯(Brandon Amos),斯图尔特·拉塞尔(Stuart Russell),诺姆·布朗(Noam Brown).
• Scaling Scaling Laws with Board Games
  • 安迪·L·琼斯.
• Show Your Work: Scratchpads for Intermediate Computation with Language Models
  • 马克斯韦尔·尼,安德斯·约翰·安德烈亚斯森,盖伊·古尔-阿里,亨利克·米哈列夫斯基,雅各布·奥斯汀,大卫·比伯,大卫·达恩,艾托尔·柳科维茨,马丁·博斯马,大卫·卢安,查尔斯·苏顿,奥古斯都·奥德纳.

2021年之前

• Improving Policies via Search in Cooperative Partially Observable Games
  • 亚当·勒尔(Adam Lerer),胡恒远(Hengyuan Hu),雅各布·福尔斯特(Jakob Fuerster),诺姆·布朗(Noam Brown).
• Generative Language Modeling for Automated Theorem Proving
  • 斯坦尼斯拉斯·波鲁,伊利亚·苏茨克韦尔
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
  • 大卫·西尔弗,托马斯·休伯特,朱利安·施利特维泽,伊万尼斯·安东诺格卢,马修·莱,亚瑟·盖兹,马克·兰克托,洛朗·西弗尔,达尔尚·库马兰,索雷·格雷佩尔,蒂莫西·利利利亚普,凯伦·西蒙扬,戴米斯·哈萨比斯.