Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added series aggregation for group by with value type panel | 5949 #6744

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

aniketio-ctrl
Copy link
Contributor

@aniketio-ctrl aniketio-ctrl commented Jan 3, 2025

Summary

Group by was not possible with value type panel, due to result containing multiple series, we have introduced seriesAggregation for aggregating multiple series into one

Related Issues / PR's

#5949

Affected Areas and Manually Tested Areas

Caching keys in value type panel, and group by with value type panel for metric builder queries.


Important

Adds series aggregation for value type panels with group by in metrics queries, updating query preparation and cache key generation.

  • Behavior:
    • Adds AddSecondaryAggregation() in series_agg_helper.go to aggregate multiple series into one for value type panels.
    • Updates PrepareMetricQuery() in query_builder.go to apply series aggregation when PanelTypeValue and GroupBy are used.
  • Models:
    • Introduces SeriesAggregation type and constants in v3.go for sum, avg, min, and max operations.
    • Adds SeriesAggregation field to BuilderQuery in v3.go.
  • Tests:
    • Adds TestPrepareMetricQueryValueTypePanelWithGroupBY() in query_builder_test.go to test new series aggregation functionality.
    • Updates TestGenerateCacheKeysMetricsBuilder() in query_builder_test.go to include series aggregation in cache keys.

This description was created by Ellipsis for fdcf3f9. It will automatically update as commits are pushed.

@github-actions github-actions bot added enhancement New feature or request docs required labels Jan 3, 2025
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 3d21b6a in 1 minute and 28 seconds

More details
  • Looked at 299 lines of code in 6 files
  • Skipped 0 files when reviewing.
  • Skipped posting 10 drafted comments based on config settings.
1. pkg/query-service/app/metrics/v4/helpers/series_agg_helper.go:28
  • Draft comment:
    Handle the case where seriesAggregator is SeriesAggregationUnspecified to avoid unexpected behavior.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment raises a valid point about missing case handling. However, returning the unmodified query when no aggregation is specified could be the intended behavior. Without seeing the SeriesAggregation enum definition or understanding the broader system design, we can't be certain that this is actually a problem. The comment is somewhat speculative.
    We don't know if returning the unmodified query for unspecified aggregation is actually a bug - it could be the desired behavior. We also don't have context about how this function is called.
    The comment is speculative and asks for defensive programming without clear evidence that it's needed. Following our rules, we should avoid speculative "what if" comments.
    Delete the comment as it's speculative and we don't have strong evidence that handling the unspecified case differently is actually required.
2. pkg/query-service/app/metrics/v4/query_builder.go:93
  • Draft comment:
    Ensure that AddSeriesAggregation handles all possible values of SeriesAggregation to prevent unexpected behavior.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The AddSeriesAggregation function is being used in PrepareMetricQuery to modify the query when the panel type is PanelTypeValue and there is a group by clause. This is consistent with the PR description and the intended functionality.
3. pkg/query-service/app/metrics/v4/query_builder_test.go:618
  • Draft comment:
    The test cases for PrepareMetricQuery are comprehensive and cover the new series aggregation feature effectively.
  • Reason this comment was not posted:
    Confidence changes required: 20%
    The test cases for PrepareMetricQuery cover various scenarios including the new series aggregation feature. This ensures that the new functionality is well-tested.
4. pkg/query-service/model/v3/v3.go:715
  • Draft comment:
    The SeriesAggregation type and its validation method are well-defined, ensuring only valid values are used.
  • Reason this comment was not posted:
    Confidence changes required: 10%
    The SeriesAggregation type and its constants are defined correctly, and the Validate method ensures that only valid values are used. This aligns with the new feature implementation.
5. pkg/query-service/app/metrics/v4/helpers/series_agg_helper.go:1
  • Draft comment:
    Avoid using the component/index.tsx file structure approach, as it makes it difficult to debug and find components using global search tools like VS Code.
  • Reason this comment was not posted:
    Comment was on unchanged code.
6. pkg/query-service/app/metrics/v4/query_builder.go:89
  • Draft comment:
    Avoid using the component/index.tsx file structure approach, as it makes it difficult to debug and find components using global search tools like VS Code.
  • Reason this comment was not posted:
    Marked as duplicate.
7. pkg/query-service/app/metrics/v4/query_builder_test.go:614
  • Draft comment:
    Avoid using the component/index.tsx file structure approach, as it makes it difficult to debug and find components using global search tools like VS Code.
  • Reason this comment was not posted:
    Marked as duplicate.
8. pkg/query-service/app/queryBuilder/query_builder.go:425
  • Draft comment:
    Avoid using the component/index.tsx file structure approach, as it makes it difficult to debug and find components using global search tools like VS Code.
  • Reason this comment was not posted:
    Marked as duplicate.
9. pkg/query-service/app/queryBuilder/query_builder_test.go:1304
  • Draft comment:
    Avoid using the component/index.tsx file structure approach, as it makes it difficult to debug and find components using global search tools like VS Code.
  • Reason this comment was not posted:
    Marked as duplicate.
10. pkg/query-service/model/v3/v3.go:712
  • Draft comment:
    Avoid using the component/index.tsx file structure approach, as it makes it difficult to debug and find components using global search tools like VS Code.
  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_t8xPXQ8UHha4Q4T4


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@srikanthccv
Copy link
Member

@aniketio-ctrl Let's call it secondaryAggregation (because we will use it beyond the value panel or metrics) and make it optional till we integrate the frontend.

@aniketio-ctrl
Copy link
Contributor Author

@aniketio-ctrl Let's call it secondaryAggregation (because we will use it beyond the value panel or metrics) and make it optional till we integrate the frontend.

done bro

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 421a6c2 in 1 minute and 44 seconds

More details
  • Looked at 233 lines of code in 6 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. pkg/query-service/app/metrics/v4/helpers/series_agg_helper.go:28
  • Draft comment:
    Consider adding a default case to handle SecondaryAggregationUnspecified to avoid returning an unmodified query when the aggregation type is unspecified.
  • Reason this comment was not posted:
    Comment was on unchanged code.
2. pkg/query-service/app/metrics/v4/query_builder.go:92
  • Draft comment:
    Before calling AddSecondaryAggregation, check if mq.SecondaryAggregation is not SecondaryAggregationUnspecified to avoid unnecessary query modifications.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment suggests adding defensive programming, but without seeing the implementation of AddSecondaryAggregation, we don't know if this check is necessary. The function might already handle unspecified aggregations internally. Also, this seems like a speculative suggestion rather than a clear issue.
    I might be wrong about the internal implementation of AddSecondaryAggregation - maybe it really does need this check. Also, defensive programming can prevent bugs.
    While defensive programming is good, without seeing the AddSecondaryAggregation implementation, this is a speculative comment. The function should handle invalid inputs internally.
    This comment should be deleted as it's speculative and we don't have enough context to know if the check is actually needed.

Workflow ID: wflow_2aaiQ56qmmU8AWzw


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Comment on lines 984 to 988
if panelType == PanelTypeValue {
if err := b.SecondaryAggregation.Validate(); err != nil {
return fmt.Errorf("series aggregation is required for value type panel with group by: %w", err)
}
}
Copy link
Member

@srikanthccv srikanthccv Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The panels are going to throw an error because you are making it mandatory to have a valid aggregation input for secondary aggregation for value panel types, please make it optional

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but its inside if b.GroupBy != nil {} so only if groupBY is not nil and value type is panel, then only it will validate

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grouby can be a non-nil but empty list.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added the panelType == PanelTypeValue && len(b.GroupBy) > 0 check

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on fdcf3f9 in 47 seconds

More details
  • Looked at 13 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. pkg/query-service/model/v3/v3.go:986
  • Draft comment:
    The error message should refer to 'secondary aggregation' instead of 'series aggregation' for clarity.
				return fmt.Errorf("secondary aggregation is required for value type panel with group by: %w", err)
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_m980SwpDf5tYr3kh


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs required enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Spatial group by a field is not possible today in value panel type
2 participants