How to confirm MetricsEndpoints (port and path) for Karmada components with ServiceMonitor or PodMonitor?

[weidalin]

Please provide an in-depth description of the question you have:
According to the documentation (Karmada Metrics Reference), Karmada provides many useful metrics.

1. I am trying to use PodMonitor to scrape metrics exposed by the karmada-controller-manager and karmada-scheduler components in my Karmada environment. However, I am unsure about the correct port and path values for the podMetricsEndpoints field.

Here is an example of my PodMonitor configuration:

apiVersion: monitoring.coreos.com/v1  
kind: PodMonitor  
metadata:  
  name: karmada-controller-manager-monitor  
  namespace: monitoring  
spec:  
  selector:  
    matchLabels:  
      app: karmada-controller-manager  
  namespaceSelector:  
    matchNames:  
      - karmada-system  
  podMetricsEndpoints:  
    - port: metrics  
      path: /metrics  
      interval: 30s  

I have referred to the Karmada Metrics documentation, but the specific port and path settings are not clearly mentioned. Could you confirm the correct values for these fields?

2. Besides configuring ServiceMonitor or PodMonitor, is there any additional configuration required to scrape metrics from Karmada?

What do you think about this question?
Clear documentation or guidelines on the correct port and path for metrics scraping would be very helpful, especially for integrating with Prometheus.

Environment:

Karmada version: 1.10.4
Kubernetes version: v1.27.x
Others: The Karmada cluster already has Prometheus installed and running. @zhzhuang-zju

[zhzhuang-zju]

@weidalin In release v1.11.0, Karmada defined metrices port name in container field of related karmada components, so that the port name can be specified in PodMonitor to form prothumes metrics dashboard. More info can refer to #5166

[zhzhuang-zju]

Are you using Karmada version v1.10.4?

[weidalin]

Are you using Karmada version v1.10.4?

Yes. Do I have to upgrade to v1.11.0 to use PodMonitor?
If I don't use PodMonitor, is there any other way to import data into Prometheus? @zhzhuang-zju

[zhzhuang-zju]

Yes. Do I have to upgrade to v1.11.0 to use PodMonitor?

If you do not wish to upgrade, you can also define the metrics name in the manifest of the related components according to #5166.

If I don't use PodMonitor, is there any other way to import data into Prometheus? @zhzhuang-zju

How about extraScrapeConfigs? It can add additional scrape configurations to the Prometheus instance.

[LavredisG]

I for one use this values.yaml file to scrape karmada-controller-manager for example:

prometheus:
  prometheusSpec:
    additionalScrapeConfigs:
      - job_name: 'karmada_metrics'
        scrape_interval: 30s
        static_configs:
          - targets:
            - '<karmada-controller-manager-IP>:8080'

Replace with your controller IP. I use the prometheus helm chart, so when installing it you have to apply this as:

helm install prometheus prometheus-community/kube-prometheus-stack -f values.yaml

Hope it helps!

[zhzhuang-zju]

@weidalin Hello, has your problem been resolved? If so, could you share your solution?