Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Obs AI Assistant] Use hardware optimised version of Elser for Linux x86-64 #205852

Closed
sorenlouv opened this issue Jan 8, 2025 · 1 comment · Fixed by #205851
Closed

[Obs AI Assistant] Use hardware optimised version of Elser for Linux x86-64 #205852

sorenlouv opened this issue Jan 8, 2025 · 1 comment · Fixed by #205851
Assignees
@sorenlouv
Labels
bug Fixes for quality problems that affect the customer experience Team:Obs AI Assistant Observability AI Assistant

Comments

@sorenlouv
Copy link
Member

sorenlouv commented Jan 8, 2025 •
edited
Loading

Regression introduced in 8.17

When creating the Obs knowledge base an inference endpoint with the Elser model .elser_model_2 is created. While this works it is suboptimal in Linux x86-64 because an optimised version of Elser exists elser_model_2_linux-x86_64 and we should use that instead when applicable.

kibana/x-pack/platform/plugins/shared/observability_solution/observability_ai_assistant/server/service/inference_endpoint.ts

Lines 16 to 19 in 06cf554

export async function createInferenceEndpoint({
esClient,
logger,
modelId = '.elser_model_2',

Impact

Users mauy end up with two versions of Elser: .elser_model_2 for the internal knowledge base and .elser_model_2_linux-x86_64 for search connectors. In this case users will simultaneously be running two models. This may cause the number of ML nodes to double.
@sorenlouv sorenlouv added the Team:Obs AI Assistant Observability AI Assistant label Jan 8, 2025
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

@sorenlouv sorenlouv mentioned this issue Jan 8, 2025
Merged
@sorenlouv sorenlouv self-assigned this Jan 8, 2025
@sorenlouv sorenlouv added the bug Fixes for quality problems that affect the customer experience label Jan 8, 2025
sorenlouv added a commit that referenced this issue Jan 8, 2025
@sorenlouv
[Obs AI Assistant] Use architecture-specific elser model (#205851)
ad3b988 
Closes #205852

When installing the Obs knowledge base it will always install the model
`.elser_model_2`.
For Linux with an x86-64 CPU an optimised version of Elser exists
(`elser_model_2_linux-x86_64`). We should use that when possible.

After this change the inference endpoint will use
`.elser_model_2_linux-x86_64` on supported hardware:

![image](https://github.com/user-attachments/assets/fedc6700-877a-47ab-a3b8-055db53407d0)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jan 8, 2025
@sorenlouv
[Obs AI Assistant] Use architecture-specific elser model (elastic#205851
3a9810b 
)

Closes elastic#205852

When installing the Obs knowledge base it will always install the model
`.elser_model_2`.
For Linux with an x86-64 CPU an optimised version of Elser exists
(`elser_model_2_linux-x86_64`). We should use that when possible.

After this change the inference endpoint will use
`.elser_model_2_linux-x86_64` on supported hardware:

![image](https://github.com/user-attachments/assets/fedc6700-877a-47ab-a3b8-055db53407d0)

(cherry picked from commit ad3b988)
sorenlouv added a commit to sorenlouv/kibana that referenced this issue Jan 8, 2025
@sorenlouv
[Obs AI Assistant] Use architecture-specific elser model (elastic#205851
dcde5fd 
)

Closes elastic#205852

When installing the Obs knowledge base it will always install the model
`.elser_model_2`.
For Linux with an x86-64 CPU an optimised version of Elser exists
(`elser_model_2_linux-x86_64`). We should use that when possible.

After this change the inference endpoint will use
`.elser_model_2_linux-x86_64` on supported hardware:

![image](https://github.com/user-attachments/assets/fedc6700-877a-47ab-a3b8-055db53407d0)

(cherry picked from commit ad3b988)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sorenlouv sorenlouv

Labels
bug Fixes for quality problems that affect the customer experience Team:Obs AI Assistant Observability AI Assistant
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Obs AI Assistant] Use architecture-specific elser model sorenlouv/kibana
2 participants
@sorenlouv @elasticmachine