📊 Monitoring Stack Setup with Prometheus, Loki, and Grafana on Kubernetes

This guide walks through the process of deploying a full monitoring stack (Prometheus, Loki, and Grafana) into a Kubernetes cluster using Helm.

🧾 Prerequisites

• A running Kubernetes cluster
• kubectl and helm installed and configured
• Cluster access with sufficient privileges

🛠️ Step 1: Prepare Namespace and Repositories

Create a namespace for the monitoring stack and add the required Helm repositories:

kubectl create namespace monitoring
 
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

🔍 Step 2: Explore Loki Chart and Customize Values

Search for Loki in the Helm repo:

helm search repo loki

Export default values for customization:

helm show values grafana/loki-stack > loki.yaml

NOTE: In loki.yaml, modify the grafana section:

grafana:
  enabled: true
  image:
    tag: latest

This enables Grafana and optionally pulls the latest image tag.

🚀 Step 3: Deploy Loki Stack with Helm

Install or upgrade the Loki stack using the customized values:

helm upgrade --install --values loki.yaml loki grafana/loki-stack -n monitoring --create-namespace

Loki UI will be available at the following internal URL:

http://loki.monitoring.svc.cluster.local:3100

📈 Step 4: Install Prometheus

helm install prometheus prometheus-community/prometheus -n monitoring

Prometheus UI will be available at the following internal URL:

http://prometheus-server.monitoring.svc.cluster.local

📊 Step 5: Install Grafana

helm install grafana grafana/grafana -n monitoring

Retrieve Grafana admin password:

kubectl get secret --namespace monitoring grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Expose Grafana locally via port-forward:

kubectl port-forward service/grafana --namespace monitoring 3000:80

Alternatively, expose via NodePort (not recommended for production):

kubectl patch svc grafana -n monitoring -p '{"spec": {"type": "NodePort"}}'

Access Grafana at http://<NodeIP>:<NodePort>

📉 Recommended Grafana Dashboards

You can import the following dashboards from Grafana Labs using their Dashboard IDs:

Argo Pipelines :

prometheus

project: default
source:
  repoURL: https://prometheus-community.github.io/helm-charts
  targetRevision: 58.5.0
  helm:
    parameters:
      - name: grafana.enabled
        value: 'false'
    releaseName: prometheus
    values: |
      grafana.enabled: false
  chart: kube-prometheus-stack
destination:
  server: https://kubernetes.default.svc
  namespace: monitoring
syncPolicy:
  syncOptions:
    - CreateNamespace=true