-
Notifications
You must be signed in to change notification settings - Fork 30
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Support monolithic deployment mode (#722)
* Support monolithic deployment mode Signed-off-by: Andreas Gerstmayr <[email protected]> * Add changelog Signed-off-by: Andreas Gerstmayr <[email protected]> * update mutate_test.go Signed-off-by: Andreas Gerstmayr <[email protected]> * add webhook tests Signed-off-by: Andreas Gerstmayr <[email protected]> * add generated files Signed-off-by: Andreas Gerstmayr <[email protected]> * support pruning unmanaged objects Signed-off-by: Andreas Gerstmayr <[email protected]> * use common reconcile/prune function Signed-off-by: Andreas Gerstmayr <[email protected]> * fix linter Signed-off-by: Andreas Gerstmayr <[email protected]> * add sts tests Signed-off-by: Andreas Gerstmayr <[email protected]> * add service tests Signed-off-by: Andreas Gerstmayr <[email protected]> * add configmap tests Signed-off-by: Andreas Gerstmayr <[email protected]> * use a single service, because the monolithic deployment will use a single pod Signed-off-by: Andreas Gerstmayr <[email protected]> * drop component from labels Signed-off-by: Andreas Gerstmayr <[email protected]> * fix configmap Signed-off-by: Andreas Gerstmayr <[email protected]> * TestBuildAll() Signed-off-by: Andreas Gerstmayr <[email protected]> * add e2e test Signed-off-by: Andreas Gerstmayr <[email protected]> * drop defaulter webhook for TempoMonolithic Signed-off-by: Andreas Gerstmayr <[email protected]> * delete operator namespace and webhooks in make run Signed-off-by: Andreas Gerstmayr <[email protected]> * change the way to include .env file Signed-off-by: Andreas Gerstmayr <[email protected]> * rebuild bundle, fix linter Signed-off-by: Andreas Gerstmayr <[email protected]> * fix typo in jaeger-grpc port name Signed-off-by: Andreas Gerstmayr <[email protected]> * add warning when overriding tempo config Signed-off-by: Andreas Gerstmayr <[email protected]> * specify default value of storage.traces.backend Signed-off-by: Andreas Gerstmayr <[email protected]> * update storage docs Signed-off-by: Andreas Gerstmayr <[email protected]> * drop .env file Signed-off-by: Andreas Gerstmayr <[email protected]> * reconcile: retry on conflict Signed-off-by: Andreas Gerstmayr <[email protected]> * update changelog, set size defaults, add api docs Signed-off-by: Andreas Gerstmayr <[email protected]> * update changelog Signed-off-by: Andreas Gerstmayr <[email protected]> * fix test Signed-off-by: Andreas Gerstmayr <[email protected]> * add controller test Signed-off-by: Andreas Gerstmayr <[email protected]> * enable OTLP/HTTP by default Signed-off-by: Andreas Gerstmayr <[email protected]> * drop TLS struct (will reuse TLS struct from TempoStack in follow-up PR) Signed-off-by: Andreas Gerstmayr <[email protected]> * add comment to default function Signed-off-by: Andreas Gerstmayr <[email protected]> * show diff using cmp.Diff() in logs when immutable field is changed Signed-off-by: Andreas Gerstmayr <[email protected]> * re-use ExtraConfig from TempoStack Signed-off-by: Andreas Gerstmayr <[email protected]> * add comment on exported function Signed-off-by: Andreas Gerstmayr <[email protected]> --------- Signed-off-by: Andreas Gerstmayr <[email protected]>
- Loading branch information
1 parent
2d4c168
commit d8afdec
Showing
68 changed files
with
5,713 additions
and
759 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' | ||
| change_type: enhancement | ||
|
|
||
| # The name of the component, or a single word describing the area of concern, (e.g. operator, github action) | ||
| component: operator | ||
|
|
||
| # A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). | ||
| note: Support monolithic deployment mode | ||
|
|
||
| # One or more tracking issues related to the change | ||
| issues: [710] | ||
|
|
||
| # (Optional) One or more lines of additional information to render under the primary note. | ||
| # These lines will be padded with 2 spaces and then inserted directly into the document. | ||
| # Use pipe (|) for multiline entries. | ||
| subtext: | | ||
| The operator exposes a new CRD `TempoMonolithic`, which manages a Tempo instance in monolithic mode. | ||
| The monolithic mode supports the following additional storage backends: in-memory and file system (persistent volume). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,236 @@ | ||
| package v1alpha1 | ||
|
|
||
| import ( | ||
| "k8s.io/apimachinery/pkg/api/resource" | ||
| metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||
| ) | ||
|
|
||
| // TempoMonolithicSpec defines the desired state of TempoMonolithic. | ||
| type TempoMonolithicSpec struct { | ||
| // Storage defines the backend storage configuration | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| Storage *MonolithicStorageSpec `json:"storage,omitempty"` | ||
|
|
||
| // Ingestion defines the trace ingestion configuration | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| Ingestion *MonolithicIngestionSpec `json:"ingestion,omitempty"` | ||
|
|
||
| // JaegerUI defines the Jaeger UI configuration | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| JaegerUI *MonolithicJaegerUISpec `json:"jaegerui,omitempty"` | ||
|
|
||
| // ManagementState defines whether this instance is managed by the operator or self-managed | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| Management ManagementStateType `json:"management,omitempty"` | ||
|
|
||
| // Observability defines observability configuration for the Tempo deployment | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| Observability *MonolithicObservabilitySpec `json:"observability,omitempty"` | ||
|
|
||
| // ExtraConfig defines any extra (overlay) configuration for components | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| ExtraConfig *ExtraConfigSpec `json:"extraConfig,omitempty"` | ||
| } | ||
|
|
||
| // MonolithicStorageSpec defines the storage for the Tempo deployment. | ||
| type MonolithicStorageSpec struct { | ||
| // Traces defines the backend storage configuration for traces | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| Traces MonolithicTracesStorageSpec `json:"traces"` | ||
| } | ||
|
|
||
| // MonolithicTracesStorageSpec defines the traces storage for the Tempo deployment. | ||
| type MonolithicTracesStorageSpec struct { | ||
| // Backend defines the backend for storing traces. Default: memory | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| // +kubebuilder:default=memory | ||
| Backend MonolithicTracesStorageBackend `json:"backend"` | ||
|
|
||
| // WAL defines the write-ahead logging (WAL) configuration | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| WAL *MonolithicTracesStorageWALSpec `json:"wal,omitempty"` | ||
|
|
||
| // PV defines the Persistent Volume configuration | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| PV *MonolithicTracesStoragePVSpec `json:"pv,omitempty"` | ||
| } | ||
|
|
||
| // MonolithicTracesStorageBackend defines the backend storage for traces. | ||
| // | ||
| // +kubebuilder:validation:Enum=memory;pv | ||
| type MonolithicTracesStorageBackend string | ||
|
|
||
| const ( | ||
| // MonolithicTracesStorageBackendMemory defines storing traces in a tmpfs (in-memory filesystem). | ||
| MonolithicTracesStorageBackendMemory MonolithicTracesStorageBackend = "memory" | ||
| // MonolithicTracesStorageBackendPV defines storing traces in a Persistent Volume. | ||
| MonolithicTracesStorageBackendPV MonolithicTracesStorageBackend = "pv" | ||
| ) | ||
|
|
||
| // MonolithicTracesStorageWALSpec defines the write-ahead logging (WAL) configuration. | ||
| type MonolithicTracesStorageWALSpec struct { | ||
| // Size defines the size of the Persistent Volume for storing the WAL. Defaults to 10Gi. | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| // +kubebuilder:default="10Gi" | ||
| Size resource.Quantity `json:"size"` | ||
| } | ||
|
|
||
| // MonolithicTracesStoragePVSpec defines the Persistent Volume configuration. | ||
| type MonolithicTracesStoragePVSpec struct { | ||
| // Size defines the size of the Persistent Volume for storing the traces. Defaults to 10Gi. | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| // +kubebuilder:default="10Gi" | ||
| Size resource.Quantity `json:"size"` | ||
| } | ||
|
|
||
| // MonolithicIngestionSpec defines the ingestion settings. | ||
| type MonolithicIngestionSpec struct { | ||
| // OTLP defines the ingestion configuration for OTLP | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| OTLP *MonolithicIngestionOTLPSpec `json:"otlp,omitempty"` | ||
| } | ||
|
|
||
| // MonolithicIngestionOTLPSpec defines the settings for OTLP ingestion. | ||
| type MonolithicIngestionOTLPSpec struct { | ||
| // GRPC defines the OTLP/gRPC configuration | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| GRPC *MonolithicIngestionOTLPProtocolsGRPCSpec `json:"grpc,omitempty"` | ||
|
|
||
| // HTTP defines the OTLP/HTTP configuration | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| HTTP *MonolithicIngestionOTLPProtocolsHTTPSpec `json:"http,omitempty"` | ||
| } | ||
|
|
||
| // MonolithicIngestionOTLPProtocolsGRPCSpec defines the settings for OTLP ingestion over GRPC. | ||
| type MonolithicIngestionOTLPProtocolsGRPCSpec struct { | ||
| // Enabled defines if OTLP over gRPC is enabled | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| // +kubebuilder:default=true | ||
| Enabled bool `json:"enabled"` | ||
| } | ||
|
|
||
| // MonolithicIngestionOTLPProtocolsHTTPSpec defines the settings for OTLP ingestion over HTTP. | ||
| type MonolithicIngestionOTLPProtocolsHTTPSpec struct { | ||
| // Enabled defines if OTLP over HTTP is enabled | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| Enabled bool `json:"enabled"` | ||
| } | ||
|
|
||
| // MonolithicJaegerUISpec defines the settings for the Jaeger UI. | ||
| type MonolithicJaegerUISpec struct { | ||
| // Enabled defines if the Jaeger UI should be enabled | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| Enabled bool `json:"enabled"` | ||
|
|
||
| // Ingress defines the ingress configuration for Jaeger UI | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| Ingress *MonolithicJaegerUIIngressSpec `json:"ingress,omitempty"` | ||
|
|
||
| // Route defines the route configuration for Jaeger UI | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| Route *MonolithicJaegerUIRouteSpec `json:"route,omitempty"` | ||
| } | ||
|
|
||
| // MonolithicJaegerUIIngressSpec defines the settings for the Jaeger UI ingress. | ||
| type MonolithicJaegerUIIngressSpec struct { | ||
| // Enabled defines if an Ingress object should be created for Jaeger UI | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| Enabled bool `json:"enabled"` | ||
| } | ||
|
|
||
| // MonolithicJaegerUIRouteSpec defines the settings for the Jaeger UI route. | ||
| type MonolithicJaegerUIRouteSpec struct { | ||
| // Enabled defines if a Route object should be created for Jaeger UI | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| Enabled bool `json:"enabled"` | ||
| } | ||
|
|
||
| // MonolithicObservabilitySpec defines the observability settings of the Tempo deployment. | ||
| type MonolithicObservabilitySpec struct { | ||
| // Metrics defines the metrics configuration of the Tempo deployment | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| Metrics *MonolithicObservabilityMetricsSpec `json:"metrics,omitempty"` | ||
| } | ||
|
|
||
| // MonolithicObservabilityMetricsSpec defines the metrics settings of the Tempo deployment. | ||
| type MonolithicObservabilityMetricsSpec struct { | ||
| // ServiceMonitors defines the ServiceMonitor configuration | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| ServiceMonitors *MonolithicObservabilityMetricsServiceMonitorsSpec `json:"serviceMonitors,omitempty"` | ||
|
|
||
| // ServiceMonitors defines the PrometheusRule configuration | ||
| // | ||
| // +kubebuilder:validation:Optional | ||
| PrometheusRules *MonolithicObservabilityMetricsPrometheusRulesSpec `json:"prometheusRules,omitempty"` | ||
| } | ||
|
|
||
| // MonolithicObservabilityMetricsServiceMonitorsSpec defines the ServiceMonitor settings. | ||
| type MonolithicObservabilityMetricsServiceMonitorsSpec struct { | ||
| // Enabled defines if the operator should create ServiceMonitors for this Tempo deployment | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| Enabled bool `json:"enabled"` | ||
| } | ||
|
|
||
| // MonolithicObservabilityMetricsPrometheusRulesSpec defines the PrometheusRules settings. | ||
| type MonolithicObservabilityMetricsPrometheusRulesSpec struct { | ||
| // Enabled defines if the operator should create PrometheusRules for this Tempo deployment | ||
| // | ||
| // +kubebuilder:validation:Required | ||
| Enabled bool `json:"enabled"` | ||
| } | ||
|
|
||
| // TempoMonolithicStatus defines the observed state of TempoMonolithic. | ||
| type TempoMonolithicStatus struct { | ||
| // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster | ||
| // Important: Run "make" to regenerate code after modifying this file | ||
| } | ||
|
|
||
| //+kubebuilder:object:root=true | ||
| //+kubebuilder:subresource:status | ||
|
|
||
| // TempoMonolithic is the Schema for the tempomonolithics API. | ||
| type TempoMonolithic struct { | ||
| metav1.TypeMeta `json:",inline"` | ||
| metav1.ObjectMeta `json:"metadata,omitempty"` | ||
|
|
||
| Spec TempoMonolithicSpec `json:"spec,omitempty"` | ||
| Status TempoMonolithicStatus `json:"status,omitempty"` | ||
| } | ||
|
|
||
| //+kubebuilder:object:root=true | ||
|
|
||
| // TempoMonolithicList contains a list of TempoMonolithic. | ||
| type TempoMonolithicList struct { | ||
| metav1.TypeMeta `json:",inline"` | ||
| metav1.ListMeta `json:"metadata,omitempty"` | ||
| Items []TempoMonolithic `json:"items"` | ||
| } | ||
|
|
||
| func init() { | ||
| SchemeBuilder.Register(&TempoMonolithic{}, &TempoMonolithicList{}) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| package v1alpha1 | ||
|
|
||
| import ( | ||
| apierrors "k8s.io/apimachinery/pkg/api/errors" | ||
| "k8s.io/apimachinery/pkg/runtime" | ||
| "k8s.io/apimachinery/pkg/util/validation/field" | ||
| ctrl "sigs.k8s.io/controller-runtime" | ||
| "sigs.k8s.io/controller-runtime/pkg/webhook" | ||
| "sigs.k8s.io/controller-runtime/pkg/webhook/admission" | ||
| ) | ||
|
|
||
| // SetupWebhookWithManager will setup the manager to manage the webhooks. | ||
| func (r *TempoMonolithic) SetupWebhookWithManager(mgr ctrl.Manager) error { | ||
| return ctrl.NewWebhookManagedBy(mgr). | ||
| For(r). | ||
| Complete() | ||
| } | ||
|
|
||
| // Default sets all default values in a central place, instead of setting it at every place where the value is accessed. | ||
| // NOTE: This function is called inside the Reconcile loop, NOT in the webhook. | ||
| // We want to keep the CR as minimal as the user configures it, and not modify it in any way (except for upgrades). | ||
| func (r *TempoMonolithic) Default() { | ||
| if r.Spec.Storage == nil { | ||
| r.Spec.Storage = &MonolithicStorageSpec{} | ||
| } | ||
|
|
||
| if r.Spec.Storage.Traces.Backend == "" { | ||
| r.Spec.Storage.Traces.Backend = MonolithicTracesStorageBackendMemory | ||
| } | ||
|
|
||
| if r.Spec.Storage.Traces.Backend != MonolithicTracesStorageBackendMemory && r.Spec.Storage.Traces.WAL == nil { | ||
| r.Spec.Storage.Traces.WAL = &MonolithicTracesStorageWALSpec{ | ||
| Size: tenGBQuantity, | ||
| } | ||
| } | ||
|
|
||
| if r.Spec.Storage.Traces.Backend == MonolithicTracesStorageBackendPV && r.Spec.Storage.Traces.PV == nil { | ||
| r.Spec.Storage.Traces.PV = &MonolithicTracesStoragePVSpec{ | ||
| Size: tenGBQuantity, | ||
| } | ||
| } | ||
|
|
||
| if r.Spec.Ingestion == nil { | ||
| r.Spec.Ingestion = &MonolithicIngestionSpec{} | ||
| } | ||
| if r.Spec.Ingestion.OTLP == nil { | ||
| r.Spec.Ingestion.OTLP = &MonolithicIngestionOTLPSpec{} | ||
| } | ||
| if r.Spec.Ingestion.OTLP.GRPC == nil { | ||
| r.Spec.Ingestion.OTLP.GRPC = &MonolithicIngestionOTLPProtocolsGRPCSpec{ | ||
| Enabled: true, | ||
| } | ||
| } | ||
| if r.Spec.Ingestion.OTLP.HTTP == nil { | ||
| r.Spec.Ingestion.OTLP.HTTP = &MonolithicIngestionOTLPProtocolsHTTPSpec{ | ||
| Enabled: true, | ||
| } | ||
| } | ||
| } | ||
|
|
||
| //+kubebuilder:webhook:path=/validate-tempo-grafana-com-v1alpha1-tempomonolithic,mutating=false,failurePolicy=fail,sideEffects=None,groups=tempo.grafana.com,resources=tempomonolithics,verbs=create;update,versions=v1alpha1,name=vtempomonolithic.kb.io,admissionReviewVersions=v1 | ||
|
|
||
| var _ webhook.Validator = &TempoMonolithic{} | ||
|
|
||
| // ValidateCreate implements webhook.Validator so a webhook will be registered for the type. | ||
| func (r *TempoMonolithic) ValidateCreate() (admission.Warnings, error) { | ||
| return r.validate() | ||
| } | ||
|
|
||
| // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type. | ||
| func (r *TempoMonolithic) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { | ||
| return r.validate() | ||
| } | ||
|
|
||
| // ValidateDelete implements webhook.Validator so a webhook will be registered for the type. | ||
| func (r *TempoMonolithic) ValidateDelete() (admission.Warnings, error) { | ||
| // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. | ||
| return r.validate() | ||
| } | ||
|
|
||
| func (tempo *TempoMonolithic) validate() (admission.Warnings, error) { | ||
| log := ctrl.Log.WithName("tempomonolithic-webhook") | ||
| log.V(1).Info("running validating webhook", "name", tempo.Name) | ||
|
|
||
| allWarnings := admission.Warnings{} | ||
| allErrors := field.ErrorList{} | ||
|
|
||
| if tempo.Spec.ExtraConfig != nil && len(tempo.Spec.ExtraConfig.Tempo.Raw) > 0 { | ||
| allWarnings = append(allWarnings, "overriding Tempo configuration could potentially break the deployment, use it carefully") | ||
| } | ||
|
|
||
| if len(allErrors) == 0 { | ||
| return allWarnings, nil | ||
| } | ||
| return allWarnings, apierrors.NewInvalid(tempo.GroupVersionKind().GroupKind(), tempo.Name, allErrors) | ||
| } |
Oops, something went wrong.