Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[LOGS]: Issue in pagination due to second presion of ksuid. #6365

Open
nityanandagohain opened this issue Nov 5, 2024 · 3 comments · May be fixed by SigNoz/signoz-otel-collector#489
Open

[LOGS]: Issue in pagination due to second presion of ksuid. #6365

nityanandagohain opened this issue Nov 5, 2024 · 3 comments · May be fixed by SigNoz/signoz-otel-collector#489
Assignees
@nityanandagohain
Labels
bug Something isn't working

Comments

@nityanandagohain
Copy link
Member

nityanandagohain commented Nov 5, 2024 •
edited
Loading

KSUID seems to have second precision and not ms/ns precision because of which id generation is shuffled for the same minute.

TS : 2024-11-04T18:02:07.026Z - id : 2oOayA9mfiFdFG4M1brPu3rfO0X
TS: 2024-11-04T18:02:07.025Z. - id: 2oOayC2SOU84wN95sZOIfMDMLDJ

ksuid -f inspect 2oOayA9mfiFdFG4M1brPu3rfO0X

REPRESENTATION:

  String: 2oOayA9mfiFdFG4M1brPu3rfO0X
     Raw: 13B6BE1F69D0C929FFA4932C944BA846902809A9

COMPONENTS:

       Time: 2024-11-04 23:32:07 +0530 IST
  Timestamp: 330743327
    Payload: 69D0C929FFA4932C944BA846902809A9

ksuid -f inspect 2oOayC2SOU84wN95sZOIfMDMLDJ

REPRESENTATION:

  String: 2oOayC2SOU84wN95sZOIfMDMLDJ
     Raw: 13B6BE1FA7A57487AAADD3F10E826FDD31B5B12D

COMPONENTS:

       Time: 2024-11-04 23:32:07 +0530 IST
  Timestamp: 330743327
    Payload: A7A57487AAADD3F10E826FDD31B5B12D

The way to solve this will be ms/ns level presion in ksuid.

cc @srikanthccv
@nityanandagohain nityanandagohain self-assigned this Nov 5, 2024
@nityanandagohain nityanandagohain added the bug Something isn't working label Nov 5, 2024
@nityanandagohain
Copy link
Member Author

If we add millisecond precision to ksuid then we will add to create a for and the size will increase to 29 bytes

The other option is we move to something like ULID .

In the previous issue related to ksuid we just solved that the current timestamp should be considered, but we ignored the precsion of the timestamp considerd

@srikanthccv
Copy link
Member

Is it possible to add nanos precision to ksuid?

@nityanandagohain
Copy link
Member Author

Yes, we can do that the size will be 32 bytes for that.

Do you think if we should add ns precision and also play around with the Random payload of 128 bits to reduce it down as we already have ns precision we just need something for randomness, or 32 bytes should be good ?

@nityanandagohain nityanandagohain mentioned this issue Nov 15, 2024
Open
@nityanandagohain nityanandagohain mentioned this issue Dec 3, 2024
Merged
@nityanandagohain nityanandagohain linked a pull request Dec 16, 2024 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nityanandagohain nityanandagohain

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: use signoz ksuid for epoch nano in log id's SigNoz/signoz-otel-collector
2 participants
@srikanthccv @nityanandagohain