LLM Strawberry

Source: https://github.com/hijkzzz/Awesome-LLM-Strawberry at 3f17fc9ee49c
License: Apache-2.0
Read the English original

This edition is an AI-assisted translation.

LLM Strawberry（OpenAI o1）研究精選

這裡收錄了關於 OpenAI Strawberry（o1）與推理能力 的研究論文及部落格。 這是一份研究文件集和部落格集 OpenAI 草莓( o1) 與原因. 本儲存庫將持續更新，追蹤大型語言模型推理領域的前沿進展。 以追蹤LLM理性的邊界。

OpenAI 文件

• https://platform.openai.com/docs/guides/reasoning
• 

新闻

• [開放] Introducing deep research
• [開放] o3 preview & o3 mini
• [開放] Introducing ChatGPT Pro
• [谷歌深明] Gemini 2.0 Flash Thinking
• [伊利亞·薩特斯科夫] AI with reasoning power will be less predictable
• [塞米安分析] Scaling Laws – O1 Pro Architecture, Reasoning Training Infrastructure, Orion and Claude 3.5 Opus “Failures”
• [深呼吸] DeepSeek-R1-Lite-Preview is now live: unleashing supercharged reasoning power!
• [月光] 数学对标o1系列，搜索再次进化，Kimi 新推理模型与你一起拓展智能边界
• [月光] Kimi 发布视觉思考模型 k1，多项理科测试行业领先
• [实习] 强推理模型书生InternThinker开放体验：自主生成高智力密度数据、具备元动作思考能力
• [新智元] 万字独家爆光，首揭o1 pro架构！惊人反转，Claude 3.5 Opus没失败？

部落格

• [開放] Learning to Reason with LLMs
• [開放] OpenAI o1-mini Advancing cost-efficient reasoning
• [開放] Finding GPT-4’s mistakes with GPT-4
• [ARC-AGI] OpenAI o3 Breakthrough High Score on ARC-AGI-Pub
• [ 雅典 ] Building effective agents
• [hijkzzz] (希克茲語) Stabilizing MoE RL Without Router Replay: The Online IcePop Solution
• [hijkzzz] (希克茲語) REINFORCE++-baseline is all you need in RLVR
• [hijkzzz] (希克茲語) Exploring OpenAI O1 Model Replication
• [納森蘭伯特] OpenAI’s Strawberry, LM self-talk, inference scaling laws, and spending more on inference
• [納森蘭伯特] Reverse engineering OpenAI’s o1
• [安德魯斯·斯圖姆勒, Supervise Process, not Outcomes
• [努哈·迪里] Have o1 Models Cracked Human Reasoning?
• [里沙布阿加瓦] Improving LLM Reasoning using Self-generated data: RL and Verifiers
• [魏沈] Generalization Progress in RLHF: Insights into the Impact of Reward Models and PPO
• [主人069] Codeforces - Analyzing how good O1-Mini actually is
• [提伯布拉荷] Summary of what we have learned during AMA hour with the OpenAI o1 team

談話

• [諾姆布朗] Parables on the Power of Planning in AI: From Poker to Diplomacy
• [諾姆布朗] OpenAI o1 and Teaching LLMs to Reason Better
• [洪元忠] Don’t teach. Incentivize.

课程

• [深入的学习。 阿爾 Reasoning with o1

推特

OpenAI 開發者

• All the questions addressed by the API team during the December 17, 2024 AMA
• We’re hosting an AMA for developers from 10–11 AM PT today.
• Today we previewed Reinforcement Fine-Tuning
• 

諾姆·布朗

• 
• 
• 
• 

魏杰森

• 
• 
• 

其他

• 
• 
• 

開源

模型

• [阿里巴·奎恩隊] Qwen3
• [阿里巴·奎恩隊] QwQ
• [阿里巴·奎恩隊] QvQ
• [深呼吸] DeepSeek R1
• [尼維迪亞] Nemotron-Research-Reasoning-Qwen-1.5B
• [天空工作] Skywork R1V2
• [rLLM] DeepScaler
• [諾瓦斯基] Sky-T1
• 〔气-〕 O1 Replication Journey: A Strategic Progress Report
• [OpenO1隊] Open-Source O1
• [十分] DRT-o1
• [阿里巴巴] Marco-o1
• [CUHK -SZ] HuatuoGPT-o1

碼基

• [OpenRLHF Team] (OpenRLHF Team] (俄語). OpenRLHF
• [OpenRLHF Team] (OpenRLHF Team] (俄語). REINFORCE++ | REINFORCE++-baseline | Code
• [新星 -AI] SkyRL
• 家 STILL: Slow Thinking with LLMs
• [掌聲] Simple Reinforcement Learning for Reasoning
  • 這是DeepSeek-R1-Zero和DeepSeek-R1對數據有限的小模型的訓練
• [無聊] Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
• [斯德芬] Open-Reasoner-Zero
• [TideDra] LMM-R1
• [莫達明斯] MM-EUREKA: Exploring Visual Aha Moment with Rule-based Large-scale Reinforcement Learning
• [R1-V隊] R1-V
• [LLAMA-事實小組] EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework
• [阿里巴巴] ROLL | Paper
• [海AI實驗室] Dr. GRPO
• [伯克利人工智能研究] TinyZero
• [Maitrix.org] LLM Reasoners

文件

format:
- [title](paper link) [links]
  - author1, author2, and author3...
  - publisher
  - code
  - experimental environments and datasets

O1型號的技術報告

• INTELLECT-3: Technical Report
  • 主智能小組
• DeepSeek V3.2
  • 深搜索AI
• MiMo-V2 Flash
  • 小美
• Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
  • 尼維迪亞
• Qwen3 Technical Report
  • Quen 隊伍
• Scaling Agents via Continual Pre-training
  • Quen 隊伍
• WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents
  • Quen 隊伍
• Towards General Agentic Intelligence via Environment Scaling
  • Quen 隊伍
• Magistral
  • 星光AI
• ERINE-4.5
  • 白度
• LongCat Flash
  • 美川
• GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
  • 日普AI
• Seed Thinking v1.5
  • 种子
• DeepSeek-V3 Technical Report
  • 深搜索
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
  • 深搜索AI
• DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
  • 深搜索AI
• MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
  • 迷你Max
• Kimi k2: Open agentic intelligence
  • 月光
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
  • 月光
• KIMI-VL TECHNICAL REPORT
  • 月光
• Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning
  • 月光和努米娜
• Llama-Nemotron: Efficient Reasoning Models
  • 尼維迪亞
• Skywork Open Reasoner 1 Technical Report
  • 天工

2025

• Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
  • 尼維迪亞
• QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory
  • Quen 隊伍
• DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
  • 深度搜索-AI
• Stabilizing Reinforcement Learning with LLMs: Formulation and Practices
  • Quen 隊伍
• The Art of Scaling Reinforcement Learning Compute for LLMs
  • 德夫里特·哈特里、洛維什·馬達安、里沙布·提瓦里、拉奇特·班薩爾、賽·蘇里亚·杜夫武里、曼齊爾·扎赫爾、因德吉特·斯·德希隆、大衛·布蘭登布雷納、里沙布·阿加瓦爾
• BroRL: Scaling Reinforcement Learning via Broadened Exploration
  • 胡建初,明杰劉,,西明路,方武,蔡德哈查維,史哲戴iao,叶真崔,巴甫洛莫爾恰諾夫, Jun陽,楊考茨, Yi東
• Why Language Models Hallucinate
  • OpenAI 檔案
• rStar2-Agent: Agentic Reasoning Technical Report
  • 宁翔,合肥劉,, Yi昭,李林娜張 Zhang,魏江徐,,X玉關,蒲澤章,炳城董,,徐東周,博文張 Zhang, Y新, Zi明苗,史嘉蕾李,,范楊,毛楊
• RL Is Neither a Panacea Nor a Mirage: Understanding Supervised vs. Reinforcement Learning Fine-Tuning for LLMs
  • 杭山金、西琴、西范武、穆罕默德·哈姆達卡
• DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization
  • 薛修杰,俞寶,俞露,盧旭,陶麗,文豪,黃修健,香寶成,盧 Lu,王玉宣
• Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning
  • Vaishnavi Shrivastava、Ahmed Awadallah、Vidhisha Balachandran、Shivam Garg、Harkirat Behl、Dimitris Papiliopoulos
• ProRL V2 - Prolonged Training Validates RL Scaling Laws
  • 胡建初,劉明杰,史志卓,盧西明,辛東,巴甫洛·莫爾恰諾夫,葉真崔,楊考茨,易東
• Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning
  • 子河 劉家春 劉家春 Yan城 贺家春 魏春 王家春 劉家春 凌潘 胡新宇 邵潘 X 朱黃 J J Hu She She She She
• Group Sequence Policy Optimization
  • 朱杰 Zheng,劉世uan,李明澤,X惠陳,博文玉,常高,启登,劉玉秋,雷門,安陽,周金格倫, Jun陽林
• Your Efficient RL Framework Secretly Brings You Off-Policy RL Training
  • 馮姚 劉良 張定怀 程玉東 鄭波翔 建丰高
• Gemini 2.5 Pro Capable of Winning Gold at IMO 2025
  • 黃一琴 林F楊
• DeepSWE: Training a Fully Open-sourced, State-of-the-Art Coding Agent by Scaling RL
  • 一起
• OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
  • 曾志旺,范 Zhou,薛芬李,彭非劉
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
  • 劉明杰,史志卓,盧西明,胡建,,辛東,葉真崔,楊考茨,易東
• REINFORCE++: An Efficient RLHF Algorithm with Robustness to Both Prompt and Reward Models
  • 胡建, 傑森·克林劉, 魏沈
  • 密碼 : REINFORCE++-baseline
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
  • Quen 隊伍
• The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
  • 新余朱氏,孟州夏氏,浙北魏氏,魏林陈氏,丹qi陈氏,于孟氏
• Thinking with Generated Images
  • 伊森·切恩、胡卓林、史黛菲·切恩、西奇·庫、賈迪·蘇、楊馬、邓志杰、彭菲
• Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning
  • 亞歷克斯·蘇、王豪芝、溫明、林方正、陳文虎
• DeepEyes: Incentivizing “Thinking with Images” via Reinforcement Learning
  • 西魏 Zheng,麥可楊,傑克洪,陳小昭,郭海徐,黎陽,曹沈,邢瑜
• Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging
  • 陳志魁、張金漢、董家 Zhu、魏劉、西洋高、苗雄、李文玲、何俊仙
• QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
  • 范琪婉,益州沈氏,沈氏,廖氏,英城史氏,陈廉李氏,子yi楊氏,姬章氏,飛黃氏, Jin氏,明 Yan氏.
• Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving
  • 新吉梅, 郝天旭, 邢W, 王維寧, 張英英, 張文強
  • 密碼 : https://github.com/yyht/openrlhf_async_pipline
• A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
  • 魏X,姚家瑞,徐玉辉,薄一波,雷汪,多延沙虎, Jun南李,南江,唐章,蔡明X,韓澤東
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example
  • 王怡平, 青陽, 曾志遠, 黎良任, 盧卡斯, 劉 Lucas, 包林鹏, 郝程, 薛海河, 熊汪, 高建峰, 陳維祖, 邵杭旺, 西蒙·肖萊迪, 耶龍沈
• Process Reward Models That Think
  • Muhammad Khalifa、Rishah Agarwal、Lajanugen Logeswaran、Jaekyeom Kim、郝鹏、Mountae Lee、Honglak Lee、Lu Wang
• M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
  • 王俊雄、李文鼎、丹妮爾·帕廖塔、丹尼爾·里特、亞歷山大·M·拉什、崔道
• A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility
  • 安德列亞斯·霍赫勒納特、哈迪克·巴特納加、維沙爾·烏丹達羅、塞缪爾·奧爾巴尼、阿梅亞·普拉布、馬提亚斯·貝斯格
• Concise Reasoning via Reinforcement Learning
  • Mehdi Fatemi、Banafsheh Rafiee、唐明杰、Kartik Talamadupula
• VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks
  • 余岳,余丰元,邱英瑜,肖琴祖,茹菲 Zhu,文云徐,陈家澤,程yi汪,天天范,曾仁都, 彭威,高洪劉,劉俊凱,凌君劉,海斌林, Zh林,博乐馬等.
• Inference-Time Scaling for Generalist Reward Modeling
  • 劉子俊、王佩宜、徐跑、白龍馬、冠魯安、彭力、楊劉、于武
• JudgeLRM: Large Reasoning Models as a Judge
  • 努o,胡志远,邹青云, Ji京武,Q汪,布萊恩胡伊,炳生贺
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
  • 余季英,郑章,茹菲 Zhu,袁玉芬,朱小琴,余岳,天天范,高洪劉,凌君劉,新劉,海斌林等.
• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
  • Jonas Geiping、Sean McLeish、Neel Jain、John Kirchenbauer、Siddharth Singh、Brian R.Bartoldson、Bhavya Kailkhura、Abhinav Bhatele、Tom Goldstein
• Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning
  • 尤克肖·奎、馬修·Y·R·楊、阿姆里斯·塞特魯爾、路易斯·通斯特爾、愛德華·伊曼紐爾·比青、拉斯蘭·薩拉胡迪诺夫、阿維拉爾·庫馬
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
  • 華通宋,金豪江,英琴敏,吉陳,芝本陳,韋恩新趙,雷方,智榮文
• Visual-RFT: Visual Reinforcement Fine-Tuning
  • 劉子玉, 澤淮孫, 余杭赞, 董小易, 余杭曹, 郝東都, 大華林, 王佳琪
• Introducing Visual Perception Token into Multimodal Large Language Model
  • 魯彭玉,馬新年,王新川
• Scaling Test-Time Compute Without Verification or RL is Suboptimal
  • 阿姆里斯·塞特魯爾、尼夫德·拉吉拉曼、谢尔蓋·列文、阿維拉爾·庫馬
• LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
  • Dacheng Li、Shiyi Cao、Tyler Griggs、Shu Liu、Xiangxi Mo、Shishir G. Patil、Matei Zaharia、Joseph E. Gonzalez、Ion Stoica
• Demystifying Long Chain-of-Thought Reasoning in LLMs
  • 葉愛德華、湯雄、莫瑞妮、格雷厄姆·諾比、項羽
• LIMR: Less is More for RL Scaling
  • 李雪峰、邹朝陽、劉彭非
• LIMO: Less is More for Reasoning
  • Yixin Ye、Zhen Huang、Yang Xia、Ethan Chern、Shijie Xia、Pengfei Liu
• s1: Simple test-time scaling
  • Niklas Muennighoff、Zitong Yang、Weijia Shi、Xiang Lisa Li、Li Fei、Hanneh Hajishirzi、Luke Zettlemoyer、Percy Liang、Emmanuel Candès、Tatsunori 桥本
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
  • 朱天zh,俞希祥,柴建洋,盛邦塘,賽宁西,戴爾·舒爾曼斯, Quoc V. Le, Sergey Levine, Yi馬
• Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  • 曾宇侯,新黎夫,陆瑞,張家杰,余江李,子俊姚,胡安子李,姬唐,余小東
• Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search
  • 沈茂豪, 耿陶曾, 曾廷琦, 張威宏, 陳振芳, 魏路, 格雷戈里·沃內爾, 蘇伯赫羅·達斯, 大衛·考克斯, 川甘
• Distillation Quantification for Large Language Models
  • 孙博文 李,, 周俊廷, 曹昂等.
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
  • 關新宇 張麗娜 劉一菲 寧翔 尤蘭 孫一 Zhu 范洋 毛洋
• Evolving Deeper LLM Thinking
  • 李光惠、伊恩·菲舍爾、岳華武、戴夫·馬伍德、舒米特·巴魯雅、戴爾·舒爾曼斯、陳新云
• The Lessons of Developing Process Reward Models in Mathematical Reasoning
  • 張振茹 朱杰 Zheng Yang Yang Yang
• Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
  • 維奧萊特·西安、查理·斯內爾、卡尼什克·甘地、阿隆·阿巴拉克、阿尼卡特·辛格、蔡斯·布拉登、杜伊·普洪、拉斐爾·拉菲洛夫、內森·利爾、達科塔·馬罕、路易·卡斯特里卡托、扬-菲利普·弗蘭肯、尼克·哈伯、切爾西·芬
• PRMBENCH: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
  • 明陽宋,昭衡蘇,小耶邱,周家威,余成
• Virgo: A Preliminary Exploration on Reproducing o1-like MLLM
  • 宜凡杜, Zi康劉, 宜凡李, 韋恩新昭, 于秋侯, 炳宁汪, 魏彭陳, Zheng劉, Zh永汪, 姬榮文
• Imagine while Reasoning in Space: Multimodal Visualization-of-Thought
  • 李成祖,文山武,張萬秋,燕霞,毛少光,李東,伊万·武利奇,傅魯魏
• LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs
  • 穆巴拉克·沙哈、法哈德·沙巴茲汗、薩勒曼汗
• From chaos to order: The atomic reasoner framework for fine-grained reasoning in large language models
  • 劉真宜,颜 Zheng,榮成,邱玉武,魏國人,飛妮,河斌良,冀福元,杭玉茂,傅正章,建义豪

2024

• Deliberative alignment: reasoning enables safer language models
  • OpenAI 檔案
• MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
  • 尼爾·喬杜里、奧利弗·賈夫、詹姆斯·昂、丹妮·謝本、埃文·梅斯、朱利奥·斯塔拉斯、凱文·劉、萊昂·馬克辛、泰雅爾·帕特沃德漢、莉莉安·溫格、阿萊克桑德·米德里
• From Medprompt to o1: Exploration of Run-Time Strategies for Medical Challenge Problems and Beyond
  • 斯科特·麥金尼
• LLM Critics Help Catch LLM Bugs
  • 納特·麥卡萊塞、雷·邁克爾·波科尼、胡安·菲利佩·塞隆·烏里韦、埃夫根尼亞·尼蒂申斯卡婭、瑪雅·特雷巴茨、楊·萊克
• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
  • 俞希iang,郝白, Zip林, Ji井潘,盛邦塘,冀非周,艾倫蘇爾,賽寧西,延勒昆, Yi馬,謝爾蓋·萊文
• ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration
  • 郝山沈,康家昭,天城昭,徐若衡, Zi, Zi,明威昭,建威殷.
• Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
  • 汪文斌,梁鼎,曾敏良,周霞斌,李沈,龍羅,達成陶
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
  • 查理·斯內爾、李傑洪、徐開文、阿維拉爾·庫馬爾
• An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
  • 吳靖正、孫志清、李珊達、韋克、楊怡明
• Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling
  • Hritik Bansal、Arian Hosseini、Rishah Agarwal、Vinh Q.Tran、Mehran Kazemi
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
  • 布拉德利·布朗、喬丹·尤拉夫斯基、瑞安·埃利希、羅納德·克拉克、Quoc V. Le、克里斯托弗·雷、阿扎莉亞·米爾霍塞尼
• Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems
  • 英琴敏,志鹏陳,金豪江,吉陳,佳登,宜文胡,宜魯唐,佳彭汪,小徐程,華通宋,韋恩新昭, Zheng劉, Zh川汪,智榮文
• Training Language Models to Self-Correct via Reinforcement Learning
  • 阿維拉爾·庫馬爾、文森特·卓昂、里沙布·阿加瓦爾、伊蘇、約翰·D·科雷耶斯、阿維·辛格、凱特·鮑姆利、沙利克·伊克巴爾、科爾頓主教、麗卡·羅夫斯、雷姆·張、凱·麥金尼、迪沙·什里瓦斯塔瓦、科斯敏·帕杜拉魯、喬治·塔克、多伊娜·普雷古普、費爾·貝赫巴哈尼、阿列克桑德拉·福斯特
• Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
  • 陳星玉,徐家豪,田良,何志蔚,潘建辉,丁玉,林峰宋,邱志劉,孟非周,卓生章,瑞旺,昭坪土,海涛米,董玉.
• MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes
  • 阿斯瑪·本·阿巴查、溫懷·依姆、尤柱·傅、昭义·孫、梅莉哈·葉蒂斯根、費霞、湯瑪斯·林
• Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
  • 安陽、張碧琴、惠賓、博非高、鮑文玉、程鹏利、劉大ih、 J洪圖、金格倫周、 Jun陽林、盧克明、薛明甫、林跑吉、劉天宇、邢宗任、曾魯章
• Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
  • 曾宇豪,彭范杜,依林倪,曾iao杜,奧漢曾,小劉,明莉黃,洪寧汪,姬唐,于小東
• Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering
  • 新燕關,柳江,盧新玉,拳曹,本河,咸北漢,黎晨,姬盧,鮑文羽,姚杰盧,洪玉林
• Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective
  • 曾志g,秦y程,章尧殷,薄汪,石敏李,云華周,祁彭郭,X京黃,西彭邱
• Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
  • Eric Zelikman、Georges Harik、Yijia Shao、Varuna Jayasiri、Nick Haber、Noah D. Goodman
  • https://github.com/ezelikman/quiet-star
• Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
  • 薛志恒,丁文亮,黃家福,唐家福,關羽,李 Guan,易文鼎,魏河,博陽宏,石漢道,文武詹,小汪,瑞 Zheng,陶池,小偉史,易陶 Z,林x文, J江, J江,X江,X江,X江,un江,X江
  • https://mathcritique.github.io/
• On Designing Effective RL Reward at Training Time for LLM Reasoning
  • 高家ax,徐述生,葉文杰,劉維林,何楚宜,魏福,梅志玉,汪光珠,易武
• Generative Verifiers: Reward Modeling as Next-Token Prediction
  • 張倫君、阿里安·侯賽尼、赫里迪克·班薩爾、梅赫蘭·卡齊米、阿維拉爾·庫馬、里沙布·阿加瓦
• Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
  • 阿姆里斯·塞特魯爾、奇拉格·納格帕爾、亞當·菲施、X陽耿、雅各布·艾森斯坦、里沙布·阿加瓦爾、阿雷赫·阿加瓦爾、喬納森·貝蘭特、阿維拉爾·庫馬爾
• Improve Mathematical Reasoning in Language Models by Automated Process Supervision
  • 梁琴 羅o 殷小劉 羅珊 劉珊拉特 費塔爾 哈爾什 拉拉 雲仙麗 萊修 雲 Zhu 萊孟 廖晨 阿布希納夫 拉斯托吉
• Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
  • 裴yi 王, 李 Li 李 Zh 薛志宏 R. X. X. X. 徐, Damai Dai, Yifei Li, Deli Chen, Y. Wu, Zhifang Sui
• Planning In Natural Language Improves LLM Search For Code Generation
  • Evan Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, Will Song, Vaskar Nath, Ziwen Han, Sean Hendryx, Summer Yue, Hugh Zhang, 夏雨,
• PROCESSBENCH: Identifying Process Errors in Mathematical Reasoning
  • 朱杰 Zheng Zh Zh Zh Zh Zh Zh
• AFlow: Automating Agentic Workflow Generation
  • 張家义,金玉祥,紫陽玉,凤威滕g,陳ion辉,陳家qi,明琴卓格,新成,瑞紅,金林汪,炳南 Zheng,邦劉,玉玉羅,成林武
• Interpretable Contrastive Monte Carlo Tree Search Reasoning
  • 齊天高,博耶尼烏,許正河,郝天徐,洪章劉,艾薇劉,舒明胡,李杰文
• Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
  • 普拉納夫·普塔、埃德蒙·米爾斯、納曼·加格、蘇米特·莫特瓦尼、切爾西·芬、迪維扬什·加格、拉斐爾·拉法伊洛夫
• Mixture-of-Agents Enhances Large Language Model Capabilities
  • 王俊林、王俊惠、本·阿提瓦拉特昆、蔡章、詹姆斯·祖
• Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models
  • 胡志远,朱民劉,西東峰,冀倫昭,西京恩,安屯路,河俊祥,潘威高,布萊恩胡伊
• Advancing LLM Reasoning Generalists with Preference Trees
  • 黎范元,甘泉桂,韩斌汪,宁定,王兴尧,贾登,博济山等.
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
  • 叶天,包林彭,林丰松,李生珍,丁玉,海涛美,董玉.
• AlphaMath Almost Zero: Process Supervision Without Process
  • 陳国新,廖民鹏,程西李,启范.
• ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
  • 丹章,西宁周比安,宜松于岳,余萧東,吉唐.
• Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search
  • 姚萬金,黃家盛,文豪武,張景義,易波王,劉書友,汪英杰,宋玉新,郝春峰,李沈,大成陶
• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
  • 余豪東, 祖燕劉, 海龍孫, J康楊, 溫斯頓胡, 永明勞, Zi
• MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time
  • 吉昆姜,新浙李,西陳,阿米雷扎·喀泽米,Q日晨,拳陈,董力,徐河,泉河,冯文,建义郝,俊姚.
• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning
  • 尤西·謝 阿尼魯德·戈雅爾 溫尤·鄭 明仁·坎 蒂莫西·P·利利格拉普 川口健二 邁克爾·希赫
• When is Tree Search Useful for LLM Planning? It Depends on the Discriminator
  • 陳齊魯 麥可懷特 雷蒙德穆尼 阿里帕亞尼 于秀 萬晨
• Chain of Thought Empowers Transformers to Solve Inherently Serial Problems
  • 李志y,洪劉,周德尼,滕玉母.
• To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
  • Zayne Sprague、Fangcong Yin、Juan Diego Rodriguez、Dongwei Jiang、Manya Wadhwa、Prasann Singhal、Xinyu Zha、Xi Ye、Kyle Mahowald、Greg Durret,
• Do Large Language Models Latently Perform Multi-Hop Reasoning?
  • 索希·楊、埃琳娜·格里博夫斯卡婭、諾拉·卡斯納、莫爾·格瓦、塞巴斯蒂安·里德爾
• Chain-of-Thought Reasoning Without Prompting
  • 王雪芝,周丹妮
• Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers
  • 曾廷琦,馬民久,徐家庄,李林娜 張,,范洋,毛洋
• Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
  • X章,朝都,天玉潘, Liu劉,魏高,民林
• ReFT: Reasoning with Reinforced Fine-Tuning
  • 川Q 盧ong 張新波 詹明杰 彭晨 萧蘭金 杭莉
• VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment
  • 阿米霍賽因·卡齊姆內賈德、米拉德·阿加霍哈里、伊娃·波特蘭斯、亞歷山德羅·索多尼、西瓦·雷迪、亞倫·庫維爾、尼古拉斯·勒魯
• Stream of Search (SoS): Learning to Search in Language
  • 甘地 丹妮絲 李 加布里爾·格蘭 劉慕辛 溫森·程 阿奇特·夏爾馬 諾亞·D·古德曼
• GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
  • 伊曼·米爾扎德、基万·阿利扎德、胡曼·沙赫羅基、韋瑟爾·圖策爾、薩米·本吉奧、梅赫達德·法拉杰塔巴
• Evaluation of OpenAI o1: Opportunities and Challenges of AGI
  • 天陽中,正廉劉,,易潘,,余通章,易范周,石良,,子豪武,燕君柳,彭舒,小威余,曹ao,韩奇江,韓旭陈,易魏李,俊豪陈等.
• Evaluating LLMs at Detecting Errors in LLM Responses
  • Ryo Kamoi、Sarkar Snigdha Sarathi Das、Renze Lou、Jihyun Jennis Ahn、Yilun Zhao、Xiaoxin Lu、Nan Zhang、Yusen Zhang、Ranran Haoran Zhang、Sujeeth Redy Vummanthala、Salika Dave、Shaobo Qin、Arman Cohan、Wenpeng Yin、Rui Zhang
• On The Planning Abilities of OpenAI’s o1 Models: Feasibility, Optimality, and Generalizability
  • 凱文·王、李俊波、尼爾·P·巴特、伊漢·西、劉強、烏福克·托普庫、張陽王
• Not All LLM Reasoners Are Created Equal
  • 阿利安·霍賽尼、亞歷山德羅·索多尼、丹尼爾·托山、亞倫·庫維爾、里沙布·阿加瓦爾
• LLMs Still Can’t Plan; Can LRMs? A Preliminary Evaluation of OpenAI’s o1 on PlanBench
  • 卡蒂克·瓦米卡姆、卡雅·斯特奇利、
• A Comparative Study on Reasoning Patterns of OpenAI’s o1 Model
  • 西魏武帝, Zh州彭城人,X州刺史,X州刺史, 州刺史, 州刺史, Tu州刺史, Tu州刺史, Tu州刺史, 州刺史, 州刺史, 州刺史, 州刺史, 州刺史,n州刺史,n州刺史,n州刺史,n州刺史,n州刺史.
• Thinking LLMs: General Instruction Following with Thought Generation
  • 天豪 伍天豪 珍妮絲 蘭恩 魏芝 袁建涛 賈森·維斯顿 森巴雅爾·蘇赫巴托
• Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems
  • 趙俊, J秋塘,玉容穆,明章宗,齐章宗,uan京黃
• V-STaR: Training Verifiers for Self-Taught Reasoners
  • 阿利安·侯賽尼、辛迪·袁、尼古拉·馬爾金、亞倫·庫維爾、亞歷山德羅·索多尼、里沙布·阿加瓦
• CPL: Critical Plan Step Learning Boosts LLM Generalization in Reasoning Tasks
  • 天龍王 陳俊哲 薛廷漢 景白
• RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
  • 天豪 伍天豪 珍妮絲 蘭恩 魏芝 袁建涛 賈森·維斯顿 森巴雅爾·蘇赫巴托
• Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
  • 王朝杰 楊琴登 柳志毅 梁曾 何菊杰 楊水泉 寶安
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning
  • 郝绍,,盛州Q人,漢孝,光路宋氏,卓芳宗,列天汪氏,于劉氏,洪生李氏

2023

• Let’s Verify Step by Step
  • 亨特·萊特曼 維尼特·科薩拉朱 尤拉·伯達 哈里·愛德華斯 鮑恩·貝克 泰迪·李 楊·萊克 約翰·舒爾曼 伊利亞·蘇特斯卡維 卡爾·科布
• V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
  • 彭高武,西寧西
• Training Chain-of-Thought via Latent-Variable Inference
  • 杜潘、馬修·D·霍夫曼、戴維·達漢、肖托·道格拉斯、圖安·勒、阿倫·帕里西、帕維爾·索恩佐夫、夏爾·蘇頓、沙拉德·維克拉姆、里夫·索魯斯
• Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training
  • 薛東峰 齊玉婉 慕宁文 史蒂芬·馬庫斯·麥卡勒 殷文 張維南 汪俊
• OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning
  • 飛羽,高安寧,王本汝
• Reasoning with Language Model is Planning with World Model
  • 石波豪,易古,郝迪馬,約書亞·佳華洪,振汪,黛西·王,志廷胡
• Don’t throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
  • 劉 、 賈 琴 、 安德 羅 科 、 拉 瑪 肯 特 巴 蘇 努 魯 、 耶 金 崔 、 漢 拿 尼 、 哈 吉 希 爾 西 、 亞 斯 利 塞 利 基 瑪 斯
• Certified reasoning with language models
  • 卡尼什克·甘地 埃里克·澤利克曼 諾亞·D·古德曼
• Large Language Models Cannot Self-Correct Reasoning Yet
  • 黃杰 陳欣雲 斯瓦羅普·米沙拉 懷克修·史蒂文·鄭 亞當斯·魏羽 辛英歌 丁尼·周

2022

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • 杰森·魏、薛志旺、戴爾·舒爾曼斯、瑪丹·博斯瑪、布賴恩·伊克特、費霞、艾德·奇、高克、周丹妮
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
  • 王雪芝、魏杰森、戴爾·舒爾曼斯、 郭克·勒、艾德·奇、沙蘭·納朗、阿肯沙·周德瑞、丹尼·周
• Self-critiquing models for assisting human evaluators
  • 威廉·桑德斯、凱瑟琳·葉、杰夫·伍、史蒂文·比爾斯、龍歐陽、喬納森·沃德、楊·萊克
• Chain of Thought Imitation with Procedure Cloning
  • 孟 家 楊 、 戴 理 舒 曼 斯 、 彼 特 亞 比 、 俄 比 勒 拿 空
• STaR: Bootstrapping Reasoning With Reasoning
  • Eric Zelikman, Yuhuai Wu, Jesse Mu, Noah D. 古德曼
• Solving math word problems with processand outcome-based feedback
  • 喬納森·烏薩托、內特·庫什曼、拉瑪娜·庫馬爾、弗朗西斯·宋、諾亞·西格爾、麗莎·汪、安東妮亞·克雷斯威爾、杰弗里·艾文、伊琳娜·希金斯

2021

• Training Verifiers to Solve Math Word Problems
  • 卡爾·科布、維尼特·科薩拉朱、穆罕默德·巴瓦里安、馬克·陳、希宇俊、盧卡斯茲·凱瑟、馬提亚斯·普拉珀特、杰瑞·特瑞克、雅各布·希爾頓、中野利二郎、克里斯托弗·黑森、約翰·舒爾曼
• Scalable Online Planning via Reinforcement Learning Fine-Tuning
  • 阿諾德·菲金格 胡恒久 布蘭登·阿莫斯 斯圖爾特·羅素 諾姆·布朗
• Scaling Scaling Laws with Board Games
  • 安迪·L·瓊斯
• Show Your Work: Scratchpads for Intermediate Computation with Language Models
  • 麥斯韋爾·尼、安德斯·約翰·安德列森、蓋伊·古爾-阿里、亨利克·米哈勒斯基、雅各布·奧斯汀、大衛·比伯、大衛·達恩、艾托爾·盧考維茨、瑪丹·博斯瑪、大衛·盧安、查爾斯·蘇頓、奧古斯都·奧德納

2021年之前

• Improving Policies via Search in Cooperative Partially Observable Games
  • 亞當·萊爾 亨吉安·胡 雅各布·福斯特 諾姆·布朗
• Generative Language Modeling for Automated Theorem Proving
  • 斯坦尼斯拉斯·波魯,伊利亞·蘇茨克沃
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
  • 大衛·西爾弗 托馬斯·休伯特 朱利安·施利特維瑟 伊万尼斯·安東諾格魯 馬修·萊 阿瑟·古茲 馬克·蘭克托 洛朗·西弗爾 達尚·庫馬蘭 索雷·格雷佩爾 蒂莫西·利利拉普 卡倫·西蒙尼安 戴米斯·哈薩比斯