trace query on tempo {resource.service.name="publisher" && name="publisher.generate_and_publish"} {resource.service.name="publisher" && name="publisher.generate_and_publish"} _______________________________AI incidents______________________________________________ {job="docker"} |= "recommendation" | json | line_format "{{.log}}" | json | line_format " ???????????????????????????????????????????????????? DEVICE: {{.device}} __________________________ SEVERITY: {{.severity}} Z-SCORE: {{.zscore}} CONFIDENCE: {{.confidence}} __________________________ AI ANALYSIS: {{.explanation}} __________________________ RECOMMENDATION: {{.recommendation}} ???????????????????????????????????????????????????? " _________________________________AI Recommendation____________________________________________ {job="docker"} |= "recommendation" | json | line_format "{{.log}}" | json | line_format "DEVICE: {{.device}} Z-SCORE: {{.zscore}} CONFIDENCE: {{.confidence}} AI ANALYSIS: {{.explanation}} RECOMMENDATION: {{.recommendation}}" _____________________________CPU usage Edge02_b____________________________ process_cpu_seconds_total{instance="100.68.152.61:2113"} _______________________________cpu uasage add 2,3,4_________________________________________ rate(container_cpu_usage_seconds_total{name="arm64-publisher_edge01-1"}[1m]) _________________________ai_last_anomaly_zscore_____________________- ai_last_anomaly_zscore #promethous query __________________ai anomaly edge13________________________ ai_anomalies_total{device="edge13"} ___________________________________CAdvicer cpu usage______________________________________ sum by (container_label_com_docker_compose_service) ( rate(container_cpu_usage_seconds_total{ container_label_com_docker_compose_project="arm64", container_label_com_docker_compose_service!="", image!="", container_label_com_docker_compose_service!~"promtail|cadvisor" }[1m]) ) ________________________________________________________container memory usage sum by (container_label_com_docker_compose_service) ( container_memory_usage_bytes{ container_label_com_docker_compose_project="arm64", container_label_com_docker_compose_service!="", image!="", container_label_com_docker_compose_service!~"promtail|cadvisor" } ) ___________________________________edegs UP or Down do 4 2.3.4. min(up{instance="opi-nvme:2112"}) min(up{instance="uefi-arm64:2112"}) min(up{instance="uefi-arm64:2112"}) _________________________________edge temp _________________________________ publisher_last_temperature_celsius{instance="100.68.152.61:2112"} _____________________________metrics query sum(count_over_time({job="docker"} [$__auto])) by (detected_level) _______________________________open telemetry min(up{instance="100.68.152.61:9464"}) ______________________________rate per device rate(publisher_messages_published_total[1m]) _________________________publishers temp publisher_last_temperature_celsius _______________________Average Temperature Across Publishers (°C) avg(publisher_last_temperature_celsius{ }) #####Loki ____________________Docker Jobs__________________________ {job="docker", host="orange-pi"} |= `edge01` | json _________________________temp log {job="docker"} |= `msg` | json | line_format `{{.log}}` |= `publish ok` _______________________logs edge04 {job="docker"} |= `edge04`