Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโ€™ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

๐Ÿ› fix: fix reasoning output for OpenRouter reasoning models like deepseek-r1 #5903

Merged

Conversation

deephbz
Copy link
Contributor

@deephbz deephbz commented Feb 8, 2025

๐Ÿ’ป ๅ˜ๆ›ด็ฑปๅž‹ | Change Type

  • [x ] โœจ feat

๐Ÿ”€ ๅ˜ๆ›ด่ฏดๆ˜Ž | Description of Change

Business Description

Get reasoning (i.e. deep thinking) outputs for all OpenRouter reasoning models, and display them in lobe-chat UI.

Fully tested: unittests as well as integration testing.

Technical Description

Unlike most other DeepSeek-R1 providers which simply return <think> ... </think> inside reply messages. OpenRouter use a separate "reasoning" key to return reasoning outputs. Besides it requires add include_reasoning=True in payload. See their docs.

Therefore, to support it we need two changes:

  1. Set include_reasoning=True by using handlePayload callback. This does not affect non-reasoning models at all. Verification:
    -you could see reasoning outputs in response:
    image
  2. Transform the response stream from
{'role': 'assistant', 'content': None, 'reasoning': 'Okay'}
{'role': 'assistant', 'content': 'start-content', 'reasoning': ''}
{'role': 'assistant', 'content': 'foo bar', 'reasoning': ''}

to

{'role': 'assistant', 'content':  '<think>Okay'}
{'role': 'assistant', 'content': '</think>start-content'}
{'role': 'assistant', 'content': 'foo bar'}

This transformed content is consistent with other platforms - a single 'content' stream of thinking then normal output tokens. Therefore, the part gets automatically picked up and displayed.

Verification:
image

๐Ÿ“ ่กฅๅ……ไฟกๆฏ | Additional Information

Solves #5766 , related #5860

Copy link

vercel bot commented Feb 8, 2025

@deephbz is attempting to deploy a commit to the LobeHub Team on Vercel.

A member of the Team first needs to authorize it.

@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Feb 8, 2025
@lobehubbot
Copy link
Member

๐Ÿ‘ @deephbz

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
้žๅธธๆ„Ÿ่ฐขๆ‚จๆๅ‡บๆ‹‰ๅ–่ฏทๆฑ‚ๅนถไธบๆˆ‘ไปฌ็š„็คพๅŒบๅšๅ‡บ่ดก็Œฎ๏ผŒ่ฏท็กฎไฟๆ‚จๅทฒ็ป้ตๅพชไบ†ๆˆ‘ไปฌ็š„่ดก็ŒฎๆŒ‡ๅ—๏ผŒๆˆ‘ไปฌไผšๅฐฝๅฟซๅฎกๆŸฅๅฎƒใ€‚
ๅฆ‚ๆžœๆ‚จ้‡ๅˆฐไปปไฝ•้—ฎ้ข˜๏ผŒ่ฏท้šๆ—ถไธŽๆˆ‘ไปฌ่”็ณปใ€‚

@dosubot dosubot bot added the ๐ŸŒ  Feature Request New feature or request | ็‰นๆ€งไธŽๅปบ่ฎฎ label Feb 8, 2025
Copy link
Contributor

gru-agent bot commented Feb 8, 2025

TestGru Assignment

Summary

Link CommitId Status Reason
Detail 7c7b5ba ๐Ÿšซ Skipped No files need to be tested {"src/config/modelProviders/openrouter.ts":"The code does not contain any functions or classes.","src/libs/agent-runtime/openrouter/index.ts":"no test value"}

Tip

You can @gru-agent and leave your feedback. TestGru will make adjustments based on your input

@deephbz deephbz changed the title โœจ feat: Support include_reasoning for OpenRouter provider's models. โœจ feat: Support reasoning output for OpenRouter reasoning models (like deepseek-r1). | ๆ”ฏๆŒ่ฏปๅ–ๅ’Œๅฑ•็คบOpenRouterๆทฑๅบฆๆ€่€ƒ่พ“ๅ‡บ Feb 9, 2025
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Feb 9, 2025
@deephbz deephbz force-pushed the feature/suppport-openrouter-reasoning-tokens branch 2 times, most recently from ca6d009 to d579c59 Compare February 9, 2025 04:41
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Feb 11, 2025
@deephbz deephbz force-pushed the feature/suppport-openrouter-reasoning-tokens branch 2 times, most recently from bb8bcfa to e471961 Compare February 11, 2025 09:18
@arvinxx arvinxx changed the title โœจ feat: Support reasoning output for OpenRouter reasoning models (like deepseek-r1). | ๆ”ฏๆŒ่ฏปๅ–ๅ’Œๅฑ•็คบOpenRouterๆทฑๅบฆๆ€่€ƒ่พ“ๅ‡บ โœจ feat: Support reasoning output for OpenRouter reasoning models like deepseek-r1 Feb 11, 2025
@deephbz
Copy link
Contributor Author

deephbz commented Feb 12, 2025

build passed:
image

@lobehubbot
Copy link
Member

Bot detected the issue body's language is not English, translate it automatically. ๐Ÿ‘ฏ๐Ÿ‘ญ๐Ÿป๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘๐Ÿ‘ซ๐Ÿง‘๐Ÿฟโ€๐Ÿคโ€๐Ÿง‘๐Ÿป๐Ÿ‘ฉ๐Ÿพโ€๐Ÿคโ€๐Ÿ‘จ๐Ÿฟ๐Ÿ‘ฌ๐Ÿฟ


build passed:
image

its "reasoning" to be consistent with most other platforms: reasoning outputs wrapped by <think> XML tag.
@deephbz deephbz force-pushed the feature/suppport-openrouter-reasoning-tokens branch from 1f04a01 to 6e943d9 Compare February 12, 2025 02:03
@deephbz deephbz force-pushed the feature/suppport-openrouter-reasoning-tokens branch from 6e943d9 to d014d34 Compare February 12, 2025 02:20
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Feb 12, 2025
Copy link

vercel bot commented Feb 12, 2025

The latest updates on your projects. Learn more about Vercel for Git โ†—๏ธŽ

Name Status Preview Comments Updated (UTC)
lobe-chat-preview โœ… Ready (Inspect) Visit Preview ๐Ÿ’ฌ Add feedback Feb 12, 2025 3:33am

@deephbz
Copy link
Contributor Author

deephbz commented Feb 12, 2025

The latest updates on your projects. Learn more about Vercel for Git โ†—๏ธŽ

Name Status Preview Comments Updated (UTC)
lobe-chat-preview โœ… Ready (Inspect) Visit Preview ๐Ÿ’ฌ Add feedback Feb 12, 2025 3:33am

Integration test looks good:
image

Copy link
Contributor

@arvinxx arvinxx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now. Thanks for your great job!

@arvinxx arvinxx changed the title โœจ feat: Support reasoning output for OpenRouter reasoning models like deepseek-r1 ๐Ÿ’„ style: Support reasoning output for OpenRouter reasoning models like deepseek-r1 Feb 12, 2025
@arvinxx arvinxx changed the title ๐Ÿ’„ style: Support reasoning output for OpenRouter reasoning models like deepseek-r1 ๐Ÿ› fix: fix reasoning output for OpenRouter reasoning models like deepseek-r1 Feb 12, 2025
@arvinxx arvinxx merged commit bfd9317 into lobehub:main Feb 12, 2025
4 of 5 checks passed
@lobehubbot
Copy link
Member

โค๏ธ Great PR @deephbz โค๏ธ

The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world.
้กน็›ฎ็š„ๆˆ้•ฟ็ฆปไธๅผ€็”จๆˆทๅ้ฆˆๅ’Œ่ดก็Œฎ๏ผŒๆ„Ÿ่ฐขๆ‚จ็š„่ดก็Œฎ! ๅฆ‚ๆžœๆ‚จๅฏน LobeHub ๅผ€ๅ‘่€…็คพๅŒบๆ„Ÿๅ…ด่ถฃ๏ผŒ่ฏทๅŠ ๅ…ฅๆˆ‘ไปฌ็š„ discord๏ผŒ็„ถๅŽ็งไฟก @arvinxx ๆˆ– @canisminor1990ใ€‚ไป–ไปฌไผš้‚€่ฏทๆ‚จๅŠ ๅ…ฅๆˆ‘ไปฌ็š„็งๅฏ†ๅผ€ๅ‘่€…้ข‘้“ใ€‚ๆˆ‘ไปฌๅฐ†ไผš่ฎจ่ฎบๅ…ณไบŽ Lobe Chat ็š„ๅผ€ๅ‘๏ผŒๅˆ†ไบซๅ’Œ่ฎจ่ฎบๅ…จ็ƒ่Œƒๅ›ดๅ†…็š„ AI ๆถˆๆฏใ€‚

github-actions bot pushed a commit that referenced this pull request Feb 12, 2025
### [Version&nbsp;1.53.1](v1.53.0...v1.53.1)
<sup>Released on **2025-02-12**</sup>

#### ๐Ÿ› Bug Fixes

- **misc**: Fix reasoning output for OpenRouter reasoning models like deepseek-r1.

<br/>

<details>
<summary><kbd>Improvements and Fixes</kbd></summary>

#### What's fixed

* **misc**: Fix reasoning output for OpenRouter reasoning models like deepseek-r1, closes [#5903](#5903) [#5766](#5766) ([bfd9317](bfd9317))

</details>

<div align="right">

[![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top)

</div>
@lobehubbot
Copy link
Member

๐ŸŽ‰ This PR is included in version 1.53.1 ๐ŸŽ‰

The release is available on:

Your semantic-release bot ๐Ÿ“ฆ๐Ÿš€

@deephbz deephbz deleted the feature/suppport-openrouter-reasoning-tokens branch February 12, 2025 05:06
github-actions bot pushed a commit to bentwnghk/lobe-chat that referenced this pull request Feb 12, 2025
### [Version&nbsp;1.95.1](v1.95.0...v1.95.1)
<sup>Released on **2025-02-12**</sup>

#### ๐Ÿ› Bug Fixes

- **misc**: Disable openrouter client fetch, fix reasoning output for OpenRouter reasoning models like deepseek-r1.

<br/>

<details>
<summary><kbd>Improvements and Fixes</kbd></summary>

#### What's fixed

* **misc**: Disable openrouter client fetch, closes [lobehub#6043](https://github.com/bentwnghk/lobe-chat/issues/6043) ([63b22ce](63b22ce))
* **misc**: Fix reasoning output for OpenRouter reasoning models like deepseek-r1, closes [lobehub#5903](https://github.com/bentwnghk/lobe-chat/issues/5903) [lobehub#5766](https://github.com/bentwnghk/lobe-chat/issues/5766) ([bfd9317](bfd9317))

</details>

<div align="right">

[![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top)

</div>
AirboZH pushed a commit to yuanze-dev/lobe-chat that referenced this pull request Feb 14, 2025
โ€ฆpseek-r1 (lobehub#5903)

* โœจ feat: Support include_reasoning for OpenRouter provider's models. Solves lobehub#5766

* โœจ feat: Support display OpenRouter's "reasoning" output by transforming
its "reasoning" to be consistent with most other platforms: reasoning outputs wrapped by <think> XML tag.

* Revert openrouter stream transformer: Capture reasoning content inside openai stream processor

* Address PR comments: r1 does not support functional calling

* add unittest for openrouter reasoning stream handling
AirboZH pushed a commit to yuanze-dev/lobe-chat that referenced this pull request Feb 14, 2025
### [Version&nbsp;1.53.1](lobehub/lobe-chat@v1.53.0...v1.53.1)
<sup>Released on **2025-02-12**</sup>

#### ๐Ÿ› Bug Fixes

- **misc**: Fix reasoning output for OpenRouter reasoning models like deepseek-r1.

<br/>

<details>
<summary><kbd>Improvements and Fixes</kbd></summary>

#### What's fixed

* **misc**: Fix reasoning output for OpenRouter reasoning models like deepseek-r1, closes [lobehub#5903](lobehub#5903) [lobehub#5766](lobehub#5766) ([bfd9317](lobehub@bfd9317))

</details>

<div align="right">

[![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top)

</div>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
๐ŸŒ  Feature Request New feature or request | ็‰นๆ€งไธŽๅปบ่ฎฎ released size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants