How to Use Grafana to Visualize Kubernetes Metrics

Grafana is an open-source monitoring tool that allows users to create custom dashboards and visualization by analyzing metrics and logs in real-time. Monitoring the Kubernetes cluster is essential to ensure that the application running is in a healthy state and has a good performance. So this makes Grafana an ideal monitoring tool to observe Kubernetes cluster.

Why Monitoring Important?

• To ensure high availability: Kubernetes main function is to provide high availability, but to make sure it works properly careful monitoring and management are required. Monitoring the cluster and setting up alerts ensures that applications remain available even in the event of failures or unexpected events.
• Identify Issues and Troubleshoot: By monitoring you can detect when there is any application failure or any network problems. With real-time monitoring, you can troubleshoot the issues before they impact your users.
• Optimize performance and capacity: Monitoring enables tracking the performance of applications and infrastructure over time and identifying opportunities to optimize performance and capacity.
• Security: By monitoring the Kubernetes cluster you can get details of potential security risks. You can set policies and regulations to counter these security risks.

Pre-requisite

The following installation should be done before moving to the next section :

• Docker
• Kind
• Kubernetes CLI(kubectl)

How to use Grafana to Visualize Kuberbernetes Metrics

Step 1: Create Kubernetes Cluster

The first step is to create a Kubernetes cluster . kind is recommended for creating a Kubernetes cluster because it supports multiple nodes on local machine. Start docker and use the configuration file provided below to create the cluster.

1. Configure YAML

config.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 32215
    hostPort: 80
- role: worker
- role: worker

Create the kubernetes cluster using the below command .

kind create cluster --config config.yaml

2 Deploy and autoscale an application on the Kubernetes cluster

After creating the cluster follow the below yaml files to deploy and autoscale an application on the Kubernetes cluster.

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: drupal
  labels:
    app: drupal
spec:
  replicas: 1
  selector:
    matchLabels:
      app: drupal
  template:
    metadata:
      labels:
        app: drupal
    spec:
      containers:
      - name: drupal
        image: drupal:latest
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 500m
          requests:
            cpu: 200m

kubectl apply -f deployment.yaml

3) Download the metrics server

Download the metrics server yaml file

https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Then after downloading the components.yaml , under the section container add this command part (line number 140 in components.yaml):

        command:
        - /metrics-server
        - --kubelet-insecure-tls
        - --kubelet-preferred-address-types=InternalIP

Create the metric server using the command below

kubectl apply -f components.yaml

4) Create a service to connect the deployment

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  type: NodePort
  selector:
    app: drupal
  ports:
    - port: 80
      targetPort: 80
      nodePort: 32215

Create a service to connect the deployment from the localhost.

kubectl apply -f service.yaml

After service is created you can access the website at http://localhost:80 .

5) Create a horizontal pod autoscaler

hpa.yaml

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
 name: drupal
spec:
 scaleTargetRef:
   apiVersion: apps/v1
   kind: Deployment
   name: drupal
 minReplicas: 1
 maxReplicas: 10
 targetCPUUtilizationPercentage: 50

Create a horizontal pod autoscaler (this will scale up or scale down the number of pods on the user traffic)

kubectl apply -f hpa.yaml

You can see the number of pods running using this command : kubectl get pods

You can check the services running using this command : kubectl get svc

You can check the whether metric-server is running or not using this command : kubectl get pods -n kube-system

You can check hpa using this command : kubectl get hpa

Step 2: Setup Prometheus and Grafana on the cluster

The second step is to setup Prometheus and Grafana on the cluster .

The following commands will setup Prometheus server on the cluster.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install prometheus prometheus-community/prometheus

The following commands will setup Grafana server on the cluster.

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm install grafana grafana/grafana  

After installing Grafana, a command will be available in the terminal to obtain the Grafana server password.

kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Step 3: Connect the Prometheus server and Grafana server

The third step is to connect the Prometheus server and Grafana server . Use different terminals to write the below commands .Start the Prometheus server at port 9090 in localhost.

kubectl port-forward service/prometheus-server 9090:80

Start the Grafana server at port 3000 in localhost.(to login into your Grafana you will require the password obtained in step 2)

kubectl port-forward service/grafana 3000:80

Access the Grafana server at http://localhost:3000

Step 4: Integrate the Prometheus server and Grafana server

The fourth step is to integrate the Prometheus server and Grafana server. Begin by logging into the Grafana server. You can then edit your username . Then, create a new data source, Prometheus.

Since both Prometheus and Grafana are running on Docker, use http://host.docker.internal:9090 instead of http://localhost:9090 for the connection. You can save and test connection .

Step 5: Observe the Kubernetes cluster.The fifth step is to observe the Kubernetes cluster using Grafana dashboards . Navigate to the left section, choose the “Dashboard” option,

then select “New” and then select “Import”.

Now here, input IDs in the grafana.com and then press load .

Then attach Prometheus data source to observe some default Grafana dashboard the cluster.

You can apply more load on the pods and observe the changes in Grafana . To apply more load on the Kubernetes cluster use this command : kubectl run -i –tty load-generator –image=busybox — sh -c “while true; do wget -O – http://my-service:80; done”

Dashboard ID for pod overview

Here you can observe the number of pods running in the default namespace and also find out the number of nodes running in the cluster.

To observe the nodes in the Kubernetes cluster you can use this id : 15759 .To observe more pod details use this dashboard id : 15760

Conclusion

So you have now successfully learned why monitoring Kubernetes cluster is required . Also we have created a sample application and , deployed and scaled it as load increases on the Kubernetes cluster . We also learned how to use Prometheus as a data source in Grafana to get the Kubernetes metrics . At last we also visualized some of the dashboards that describes pods and Kubernetes cluster in Grafana .

FAQs On How to Use Grafana to Visualize Kubernetes Metrics

1. How Grafana is used in troubleshooting issues within the Kubernetes cluster?

Grafana troubleshoots Kubernetes cluster issues by visualizing Kubernetes cluster metrics , identifying performance bottlenecks and setting up alerts on defined thresholds to ensure immediate notification when issue occurs.

2. What are the different Kubernetes metrics which can be monitored using Grafana?

Grafana can collect specific metrics from Kubernetes cluster such as pod health , performance of nodes , resource utilization ,network details and other Kubernetes cluster details .

3. What are the different practices used for setting up Grafana to monitor a Kubernetes cluster?

Configuring the data source , creating dashboards and setting up alerts for any specific Kubernetes metrics .

4. Name some of the alternative tools used in place of Grafana for monitoring Kubernetes cluster .

Some of alternative tools are Kubernetes Dashboard (official) , Kibana , Prometheus , Dynatrace , etc.

5. What are the pros and cons of using prebuilt dashboards for monitoring Kubernetes cluster in Grafana?

Prebuilt dashboards for Kubernetes in Grafana can save time and effort by providing a starting point for visualizing your cluster but prebuilt dashboards does not include all of the metrics that you want to visualize .