126. Prometheus-operator v0.79.0 和 v0.79.1 对 Istio 无法正常工作的监控
Situation 地理位置Many Istio dashboards in Grafana show no data, displaying only N/A.Grafana 中许多 Istio 仪表盘没有显示任何数据只显示 N/A。The istio_requests_total metric and other expected Istio metrics series are not being collected by Prometheus.Prometheus 并未收集“istio_requests_total”指标及其他预期的 Istio 指标系列。The graphs on the Kiali UI are also lacking information.Kiali 界面上的图表也缺乏信息。Resolution 结局The permanent fix is to upgrade prometheus-operator to version v0.79.2 or higher. For rancher-monitoring this is achieved by upgrading the chart to version 106.1.0up69.8.2-rancher.4 or higher, in Rancher v2.11.永久修复方法是将 prometheus operator 升级到 v0.79.2 或更高版本。对于牧场主监控这通过将图表升级到 106.1.0up69.8.2-rancher.4 或更高版本牧场者 v2.11实现。In affected versions, the workaround is to edit the envoy service monitor (kubectl -n istio-system edit servicemonitor envoy-stats-monitor) and update the operator to ! instead of DoesNotExist:在受影响的版本中解决方法是编辑使者服务监视器kubectl -n istio-system edit servicemonitor envoy-stats-monitor并将操作员更新为“”而不是 DoesNotExistspan stylecolor:#000000span stylebackground-color:#ffffffspan stylebackground-color:#efefefcode[...] selector: matchExpressions: - key: istio-prometheus-ignore operator: !/code/span/span/spanCause 病因This results from the upstream prometheus-operator issue https://github.com/prometheus-operator/prometheus-operator/issues/7214 . The ServiceMonitor for Istio used matchExpressions with operator: DoesNotExist. In prometheus-operator versions v0.79.0 and v0.79.1, this operator was not recognized, leading to validation errors (unsupported value: doesnotexist) and rejection of the ServiceMonitor. As a result, Prometheus did not scrape Envoy metrics. For more information you can refer to https://github.com/rancher/rancher/issues/51982这是由于上游的普罗米修斯-算子问题 https://github.com/prometheus-operator/prometheus-operator/issues/7214。Istio 的 ServiceMonitor 使用了带有“operator DoesNotExist” 的 matchExpressions。在 prometheus 操作符版本 v0.79.0 和 v0.79.1 中该操作符未被识别导致验证错误不支持的值“doesnnotexist”并拒绝 ServiceMonitor。因此普罗米修斯没有抓到 Envoy 的指标。更多信息可参考 https://github.com/rancher/rancher/issues/51982访问Rancher-K8S解决方案博主企业合作伙伴 https://blog.csdn.net/lidw2009