System Monitoring with Prometheus and Grafana

This guide covers the complete setup of a monitoring system using Prometheus, Grafana, and Node Exporter.

Table of Contents

1. Install Prometheus
2. Install Grafana
3. Install and Configure Node Exporter
4. Creating a Custom Dashboard

INSTALLATION

1. Install Prometheus

1.1 Download and Install Prometheus

wget https://github.com/prometheus/prometheus/releases/download/v2.37.0/prometheus-2.37.0.linux-amd64.tar.gz
tar xvfz prometheus-*.tar.gz
cd prometheus-*/
sudo mv prometheus /usr/local/bin/
sudo mv promtool /usr/local/bin/
sudo mkdir /etc/prometheus
sudo mv prometheus.yml /etc/prometheus/
cd ..
rm -rf prometheus-*

Create Prometheus systemd Service

sudo tee /etc/systemd/system/prometheus.service <<EOF
[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=default.target
EOF

NOTE:

The command: bash --storage.tsdb.path /var/lib/prometheus/ is responsible for setting up the storage and data retention

Create Prometheus User and Directories

sudo useradd --no-create-home --shell /bin/false prometheus
sudo mkdir -p /var/lib/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus

Start Prometheus Service

sudo systemctl daemon-reload
sudo systemctl start prometheus
sudo systemctl enable prometheus

2. Install Grafana

2.1 Add Grafana Repository and Install

wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
sudo apt-get update
sudo apt-get install -y grafana

2.2 Start Grafana Service

sudo systemctl start grafana-server
sudo systemctl enable grafana-server

3. Install and Configure Node Exporter

3.1 Download and Install Node Exporter

wget https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz
tar xvfz node_exporter-1.3.1.linux-amd64.tar.gz
sudo mv node_exporter-1.3.1.linux-amd64/node_exporter /usr/local/bin/
rm -rf node_exporter-1.3.1.linux-amd64*

3.2 Create Node Exporter systemd Service

sudo tee /etc/systemd/system/node_exporter.service <<EOF
[Unit]
Description=Node Exporter
After=network.target

[Service]
User=node_exporter
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target
EOF

3.3 Create Node Exporter User

sudo useradd -rs /bin/false node_exporter

3.4 Start Node Exporter Service

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl enable node_exporter

3.5 Configure Prometheus to Scrape Node Exporter

Edit bash /etc/prometheus/prometheus.yml and add:

scrape_configs:
  - job_name: 'node'
    static_configs:
      - targets: ['localhost:9100']

Restart Prometheus:

sudo systemctl restart prometheus

DASHBOARD CONFIG

• Open Grafana on port :3000
• Navigate to dashboards and view dashboard matrics with node explorer

ACCESS CONTROL

• A guest user account is set up to give access to guests, with permission only to view the dashboard and other matrics.

login