Cloud Monitoring
Cloud Monitoring is the metrics, dashboards, alerting, and SLO layer of Google Cloud Observability. It collects metrics from GCP services automatically (CPU, memory, request rate, latency, error rate), accepts custom metrics from applications, and through the Ops Agent collects system metrics from VMs and on-prem. Dashboards visualize; alerting policies fire notifications when thresholds or SLO budgets are violated; uptime checks probe public endpoints from multiple Google POPs.
The most consequential 2026 change: PromQL is now the recommended query language. The older MQL (Monitoring Query Language) is deprecated for new work — Google publishes migration guidance, but new dashboards, alerts, and queries should use PromQL from the start. Industry alignment with Prometheus is the practical benefit; the team's skills carry across clouds.
Metrics Collection and Types
Three sources of metrics flow into Cloud Monitoring:
- Built-in GCP metrics — every managed service emits metrics automatically. Cloud Run request rate, GKE pod CPU, Cloud SQL query latency, Pub/Sub backlog. No configuration required.
- Custom metrics — applications push application-specific values through the Monitoring API or via OpenTelemetry. Business metrics (orders per minute), feature flags, queue depths.
- System metrics from VMs and on-prem — the Ops Agent collects host-level metrics (disk, memory, network) from any Linux/Windows machine running the agent.
All three streams share the same query and alerting layer. From the dashboard's perspective, a built-in metric and a custom metric look identical.
Dashboards (Built-in vs Custom)
Google ships per-service dashboards: open the Cloud Run console and see request rate, latency, and error rate without configuration. The built-ins are a good starting point for operational visibility — they show what most teams need most of the time. Custom dashboards are necessary for cross-service views: a dashboard that overlays application request rate, downstream Pub/Sub depth, and database CPU to show end-to-end health. Define custom dashboards in code (Terraform's google_monitoring_dashboard resource) so they live in version control and review.
Alerting Policies and Notification Channels
An alerting policy defines a condition (PromQL or MQL expression with a threshold) and the notification channels to alert. Notification channels route to email, PagerDuty, Slack, Pub/Sub, webhook URLs, and Cloud Mobile App push. The right setup combines several: critical alerts page on-call via PagerDuty, all alerts also write to a Pub/Sub topic that feeds a SIEM, and a Slack channel mirrors the firing alerts so the rest of the team sees what is happening.
SLOs, SLIs, and Error Budgets
Cloud Monitoring has first-class support for Service Level Objectives. The pattern is the same SRE practice many teams already use: define a Service Level Indicator (e.g., "the fraction of HTTP requests that returned 2xx in under 500ms"), commit to an objective (99.9% of requests over a 30-day window), derive an error budget (0.1% of requests are allowed to fail), and alert on budget burn rate rather than raw threshold.
Burn-rate alerting is dramatically better than threshold alerting for production systems. A static "alert if error rate above 1%" pages when 1% has been bad for 5 minutes, then again at 6 minutes, then at 7 minutes — same incident, repeated noise. Burn-rate alerting pages once when the budget is burning fast enough to exhaust the period, and again only if the burn rate is so high the budget will exhaust before on-call can intervene. This is what SLO-based alerting buys you.
Uptime Checks (Synthetic Monitoring)
Uptime checks probe public endpoints from multiple Google POPs around the world and alert on failure. The simplest synthetic monitor — "is the homepage returning 200" — replaces the need to wait for a real customer to report an outage. Configure them on every public-facing endpoint, alert on failures with a sensible threshold (multiple consecutive failures from multiple POPs), and route to the same on-call rotation as other production alerts.
MQL vs PromQL
For years Google's native query language was MQL — a SQL-like syntax specific to Cloud Monitoring. As of 2026, PromQL is the recommended path forward. New dashboards, alerts, and queries should use PromQL; existing MQL queries continue to work but should be migrated when teams touch them anyway. The reason is alignment: Prometheus is the de facto standard for cloud-native metrics, and PromQL skills carry across cloud providers and on-prem deployments. MQL was an isolated dialect; PromQL is the lingua franca.
- Alerting on every metric Cloud Monitoring exposes. The team gets paged 50 times a day on different signals; the next real alert is dismissed as more noise.
- No SLO definition. Monitoring is reactive only — the team finds out about issues when customers complain. There is no proactive baseline of "what we promise" against which to detect drift.
- Single notification channel (email only). Critical alerts arrive in the same inbox as marketing newsletters; the team learns to ignore both.
- Dashboard sprawl without owners. A long-tenured team has 200 dashboards, 195 are stale, no one knows which 5 are authoritative, every incident starts with "do we have a dashboard for this?"
- Custom metrics with high-cardinality labels (user IDs, request paths, free-form strings). Metric series count grows linearly with cardinality; cost grows with series count.
- Uptime checks configured but no alert when they fail. The dashboard shows the failure; nobody notices.
- MQL used for new work in 2026 when PromQL is the recommended path. The migration burden grows with every new MQL query added.
- SLO-based alerting (error budget burn rate) for production services. Threshold-based alerts only for clear binary conditions (process down, disk 100% full).
- Multi-channel notification with severity-aware routing. P0 pages via PagerDuty; P1 also writes to Slack; everything also writes to a Pub/Sub topic for SIEM correlation.
- Dashboards defined in code (Terraform). Hand-built dashboards drift, have no review, and have no version history.
- PromQL for all new queries. Migrate MQL when the team next touches the surrounding alert or dashboard.
- Uptime checks for every public endpoint, alerts on failure from multiple POPs.
- Custom metric cardinality budget. Label dimensions reviewed before adding; user IDs and request bodies are off-limits as labels.
- Integration with Cloud Logging via log-based metrics for application-specific signals not directly available as metrics.
Knowledge Check
Why is burn-rate alerting on SLO error budgets dramatically better than raw threshold alerting?
- Burn-rate alerts cost less per fire than ordinary threshold alerts, because Cloud Monitoring rate-limits how often each burn-rate policy is permitted to evaluate, fire, and bill against the account
- A burn-rate alert fires once when the budget is burning fast enough to matter, instead of firing every minute the threshold stays exceeded — so one incident does not produce many duplicate pages
- Burn-rate alerts automatically suppress all notifications during configured business hours, so on-call engineers are only ever paged outside of working hours
- Threshold alerts have been deprecated, so as of 2026 only burn-rate alert policies can still integrate with PagerDuty in production
What is the recommended query language for new Cloud Monitoring queries in 2026?
- MQL, the Monitoring Query Language — Google's native metric dialect, which remains the recommended default for any newly authored Cloud Monitoring query
- SQL run through Log Analytics, which is the supported way to query Cloud Monitoring metric time series in 2026
- PromQL — the de facto industry standard, aligned with Prometheus, with skills that carry across cloud providers and on-prem deployments
- A Python-based metric DSL that is embedded directly inside each dashboard's JSON definition and evaluated server-side
What is the principal failure mode of alerting on every available metric?
- Cloud Monitoring rate-limits and hard-throttles any account that exceeds 100 simultaneously active alerting policies within a single project
- Alert fatigue — the team gets paged many times a day on different signals, so the next genuinely real alert is dismissed along with all the noise
- The recurring cost of continuously evaluating all of those alerting policies eventually grows to exceed the cost of the underlying compute fleet in most workloads
- Cloud Monitoring deduplicates alerts only across one policy at a time, which causes downstream PagerDuty to misroute incidents
Why do high-cardinality labels on custom metrics produce an outsized Cloud Monitoring bill?
- Cloud Monitoring charges a flat fee per label name regardless of its values, so simply adding more labels raises the per-metric base price
- Series count grows with the number of distinct label-value combinations; user IDs or full URLs as labels can produce millions of series per metric, and cost tracks series count
- High-cardinality labels silently disable Cloud Monitoring's dashboard caching layer, which then forces every chart on the page to fully recompute its underlying data from scratch on each load
- PromQL refuses to query any label that carries more than 1000 distinct values, so high cardinality breaks the affected queries entirely
What is the role of uptime checks in Cloud Monitoring?
- They confirm whether VMs are reachable on their internal IP addresses from inside the VPC, supplementing the load balancer's own health checks
- Synthetic monitoring — probing public endpoints from multiple Google POPs worldwide and alerting on failure, so outages surface before a customer reports them
- They verify that the Ops Agent installed on each VM is healthy and still shipping its metrics, and they automatically restart that agent process whenever it stops responding
- Scheduled automated penetration testing of public-facing applications, scanning each endpoint for OWASP-class security vulnerabilities
You got correct