Alerting configured for anomalous patterns (failed logins, privilege escalation, unexpected connections)

Infrastructure and application metrics tell you CPU is high — security alerting tells you someone is trying to escalate privileges. Without alerts on failed authentication bursts, RBAC denials, or unexpected egress connections, active attacks proceed silently through your cluster until data has already left (NIST 800-53 SI-4, AU-6). CIS Kubernetes 3.2.1 requires audit logging and alert routing. A Prometheus instance with only CPU/memory/disk alerts is operationally useful but security-blind — the two alert categories are not interchangeable.

Add security-focused PrometheusRule resources alongside your infrastructure alerts. In k8s/monitoring/alerts.yaml:

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: security-alerts
  namespace: monitoring
spec:
  groups:
    - name: security
      rules:
        - alert: BruteForceAuthAttempts
          expr: rate(http_requests_total{status="401"}[5m]) > 0.5
          for: 2m
          labels:
            severity: warning
          annotations:
            summary: "High 401 rate — possible brute force on {{ $labels.job }}"
        - alert: RBACDenialSpike
          expr: increase(apiserver_audit_event_total{verb="create",objectRef_resource="clusterrolebindings"}[10m]) > 0
          for: 1m
          labels:
            severity: critical
          annotations:
            summary: "ClusterRoleBinding creation detected"

Route critical alerts to PagerDuty or a dedicated security Slack channel, separate from on-call infrastructure alerts.