1 # Prometheus Statistics
3 Each contiv-agent exposes statistics in Prometheus format at port `9999` by default.
4 Exposed data is split into two groups:
5 - `/stats` provides statistics for VPP interfaces managed by contiv-agent
6 Prometheus data is a set of counters with labels. For each interface,
7 the following counters are exposed:
20 Labels let you add additional information to a counter. The *interfaceName* and *node*
21 labels are specified for all counters. If an interface is associated with a particular
22 pod, then the *podName* and *podNamespace* labels are also specified for its counters;
23 otherwise, a placeholder value (`--`) is used (for example, for node interconnect
25 - `/metrics` provides general go runtime statistics
27 To access Prometheus stats of a node you can use `curl localhost:9999/stats` from the node. The output of contiv-agent running at k8s master node looks similar to the following:
30 $ curl localhost:9999/stats
31 # HELP dropPackets Number of dropped packets for interface
32 # TYPE dropPackets gauge
33 dropPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
34 dropPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 52
35 dropPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 9
36 dropPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 12
37 dropPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
38 # HELP inBytes Number of received bytes for interface
40 inBytes{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
41 inBytes{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 24716
42 inBytes{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 726
43 inBytes{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 6113
44 inBytes{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
45 # HELP inErrorPackets Number of received packets with error for interface
46 # TYPE inErrorPackets gauge
47 inErrorPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
48 inErrorPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
49 inErrorPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
50 inErrorPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
51 inErrorPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
52 # HELP inMissPackets Number of missed packets for interface
53 # TYPE inMissPackets gauge
54 inMissPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
55 inMissPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
56 inMissPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
57 inMissPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
58 inMissPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
59 # HELP inNobufPackets Number of received packets ??? for interface
60 # TYPE inNobufPackets gauge
61 inNobufPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
62 inNobufPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
63 inNobufPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
64 inNobufPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
65 inNobufPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
66 # HELP inPackets Number of received packets for interface
67 # TYPE inPackets gauge
68 inPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
69 inPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 97
70 inPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 9
71 inPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 60
72 inPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
73 # HELP ipv4Packets Number of ipv4 packets for interface
74 # TYPE ipv4Packets gauge
75 ipv4Packets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
76 ipv4Packets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 68
77 ipv4Packets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
78 ipv4Packets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 52
79 ipv4Packets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
80 # HELP ipv6Packets Number of ipv6 packets for interface
81 # TYPE ipv6Packets gauge
82 ipv6Packets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
83 ipv6Packets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 26
84 ipv6Packets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 9
85 ipv6Packets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 8
86 ipv6Packets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
87 # HELP outBytes Number of transmitted bytes for interface
89 outBytes{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
90 outBytes{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 5203
91 outBytes{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
92 outBytes{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 17504
93 outBytes{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
94 # HELP outErrorPackets Number of transmitted packets with error for interface
95 # TYPE outErrorPackets gauge
96 outErrorPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
97 outErrorPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
98 outErrorPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
99 outErrorPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
100 outErrorPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
101 # HELP outPackets Number of transmitted packets for interface
102 # TYPE outPackets gauge
103 outPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
104 outPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 49
105 outPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
106 outPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 45
107 outPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
108 # HELP puntPackets Number of punt packets for interface
109 # TYPE puntPackets gauge
110 puntPackets{interfaceName="GigabitEthernet0/9/0",node="dev",podName="--",podNamespace="--"} 0
111 puntPackets{interfaceName="tap-vpp2",node="dev",podName="--",podNamespace="--"} 0
112 puntPackets{interfaceName="tap0e6439a7a934336",node="dev",podName="web-667bdcb4d8-pxkfs",podNamespace="default"} 0
113 puntPackets{interfaceName="tap5338a3285ad6bd7",node="dev",podName="kube-dns-6f4fd4bdf-rsz9b",podNamespace="kube-system"} 0
114 puntPackets{interfaceName="vxlanBVI",node="dev",podName="--",podNamespace="--"} 0
119 In order to browse stats in web UI Prometheus, it must be started locally by following the information in
120 the [Prometheus Getting Started Guide](https://prometheus.io/docs/prometheus/latest/getting_started/).
122 If you start Prometheus on a node, the following sample config can be used:
128 - job_name: 'contiv_stats'
129 metrics_path: '/stats'
131 - targets: ['localhost:9999']
132 - job_name: 'contiv_agent'
133 # metrics_path defaults to '/metrics'
135 - targets: ['localhost:9999']
138 Once Prometheus is started with the specified config, you should be able access its web UI at
141 tester@dev:~/Downloads/prometheus-2.1.0.linux-amd64$ ./prometheus --config.file=config.yml
144 If security features are enabled for the HTTP endpoint, then the config must be adjusted:
146 - job_name: 'contiv_secured'
154 insecure_skip_verify: true
155 # CA certificate to validate API server certificate with.
156 #[ ca_file: <filename> ]
158 - targets: ['localhost:9191']