K8s Infra Metrics

Overview

To export K8s metrics, you can enable different receivers in OpenTelemetry collector which will send metrics about your Kubernetes infrastructure to SigNoz. These OpenTelemetry collectors will act as agents which send metrics about K8s to SigNoz.

Based on where you are running SigNoz ( e.g. in an independent VM or K8s cluster), you have to provide the address to send data from the above receivers.

Steps to export K8s metrics to SigNoz

Clone Otel collector repo

git clone https://github.com/SigNoz/otel-collector-k8s.git && cd otel-collector-k8s

Set up the address to SigNoz in your OTel collectors
You need to set up the address to SigNoz in your OTel collector which is collecting the K8s metrics.
a. If you are running SigNoz in an independent VM, you need to change the placeholder IPs in the following files with the IP of machine where you are hosting signoz.
- agent/infra-metrics.yaml
- deployment/all-in-one.yaml
You need to update the below section.
```
exporters:
   otlp:
     endpoint: "<IP of machine hosting SigNoz>:4317"
     tls:
       insecure: true
```
b. If you are running SigNoz in the same Kubernetes cluster where your applications are, you have to replace the above endpoint in agent/infra-metrics.yaml and deployment/all-in-one.yaml by
```
<helm_release_name>-signoz-otel-collector.<namespace_hosting_signoz>.svc.cluster.local:4317
```

Install OTel collectors and enable specific receivers to send metrics to SigNoz

To access metrics from kubeletstats receivers you have to:

kubectl create ns signoz-infra-metrics
kubectl -n signoz-infra-metrics apply -Rf agent
kubectl -n signoz-infra-metrics apply -Rf deployment
kubectl -n signoz-infra-metrics get pods

List of metrics from Kubernetes receiver.

Plot Metrics in SigNoz UI
To plot these metrics, follow the instructions given in the docs here.
Import Dashboard with CPU and Memory Metrics
You can import basic dashboard with CPU and Memory metrics of K8s cluster containers from here.
After importing, you can include more widgets using other metrics from the list below.

Generate and Import Dashboard with Node Metrics

To generate node metrics dashboards for each nodes of K8s cluster:

for node in $(kubectl get nodes -o name | sed -e "s/^node\///");
do
curl -sL https://github.com/SigNoz/benchmark/raw/main/dashboards/hostmetrics/hostmetrics-import.sh \
   | HOSTNAME="$node" DASHBOARD_TITLE="Node Metrics Dashboard for $node" bash
done

After importing the generated dashboards, you can include more widgets as per you need using metrics from the list below.

List of metrics

Kubernetes Metrics - kubeletstats and k8s_cluster

container_cpu_time
container_cpu_utilization
container_filesystem_available
container_filesystem_capacity
container_filesystem_usage
container_memory_available
container_memory_major_page_faults
container_memory_page_faults
container_memory_rss
container_memory_usage
container_memory_working_set
k8s_container_cpu_limit
k8s_container_cpu_request
k8s_container_ephemeral_storage_request
k8s_container_memory_limit
k8s_container_memory_request
k8s_container_ready
k8s_container_restarts
k8s_daemonset_current_scheduled_nodes
k8s_daemonset_desired_scheduled_nodes
k8s_daemonset_misscheduled_nodes
k8s_daemonset_ready_nodes
k8s_deployment_available
k8s_deployment_desired
k8s_namespace_phase
k8s_node_condition_memory_pressure
k8s_node_condition_ready
k8s_node_cpu_time
k8s_node_cpu_utilization
k8s_node_filesystem_available
k8s_node_filesystem_capacity
k8s_node_filesystem_usage
k8s_node_memory_available
k8s_node_memory_major_page_faults
k8s_node_memory_page_faults
k8s_node_memory_rss
k8s_node_memory_usage
k8s_node_memory_working_set
k8s_node_network_errors
k8s_node_network_io
k8s_pod_cpu_time
k8s_pod_cpu_utilization
k8s_pod_filesystem_available
k8s_pod_filesystem_capacity
k8s_pod_filesystem_usage
k8s_pod_memory_available
k8s_pod_memory_major_page_faults
k8s_pod_memory_page_faults
k8s_pod_memory_rss
k8s_pod_memory_usage
k8s_pod_memory_working_set
k8s_pod_phase
k8s_replicaset_available
k8s_replicaset_desired
k8s_statefulset_current_pods
k8s_statefulset_desired_pods
k8s_statefulset_ready_pods
k8s_statefulset_updated_pods

Node Metrics - hostmetrics

system_network_connections
system_disk_weighted_io_time
system_disk_merged
system_disk_operation_time
system_disk_pending_operations
system_disk_io_time
system_disk_operations
system_disk_io
system_filesystem_inodes_usage
system_filesystem_usage
system_cpu_time
system_memory_usage
system_network_packets
system_network_dropped
system_network_io
system_network_errors
system_cpu_load_average_5m
system_cpu_load_average_15m
system_cpu_load_average_1m

Overview​

Steps to export K8s metrics to SigNoz​

List of metrics​

Overview

Steps to export K8s metrics to SigNoz

List of metrics