Fix false-positive slow alerts: require 3 consecutive checks + use P95 baseline

Previously a single check above 2× average response time fired an alert.
This caused false positives from normal network variance.

- scheduler.js: track consecutive slow checks per resource; only alert after
  3 in a row (resets to 0 when response time normalises)
- monitorService.js: calculateResponseTimeBaseline now returns P95 instead
  of mean, making 2× baseline a much more realistic slow threshold

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
nmemmertandClaude Sonnet 4.6 committed 2026-08-06 16:50:57 -04:00
1 parent 98e0949d08
commit dd85a79d55
2 files changed
+28 -19

No files matched your search

+4 -3
View File
@@ -765,9 +765,10 @@ class MonitorService {
return null;
}
// Return average of response times
const sum = checks.reduce((acc, c) => acc + (c.response_time || 0), 0);
return Math.round(sum / checks.length);
// Use P95 so the baseline reflects realistic worst-case, not the average.
// Threshold is 2× P95, making it very unlikely to trigger on normal variance.
const p95Index = Math.min(Math.floor(checks.length * 0.95), checks.length - 1);
return Math.round(checks[p95Index].response_time);
}
// Check if response time is anomalous