Cloud Logging

Cloud Logging is the central log aggregator on Google Cloud. Every GCP service emits logs to it automatically — Cloud Run request logs, GKE pod stdout/stderr, Compute Engine serial console, IAM audit, every API call across the platform. Workloads outside GCP can write through the Cloud Logging API or via the Ops Agent on VMs. The output is a unified queryable store of everything that happened on the platform, in one place, with a single query language.

The non-obvious thing about Cloud Logging is its economics. The service itself is cheap; the analytic queries and the long-term retention are not. A production org without thought-out routing and retention strategy can run a six-figure annual Cloud Logging bill while still missing the data they actually need during an incident. The architectural decisions in this chapter — which logs go where, how long they live, what gets routed to BigQuery, what audit categories are even enabled — matter more than the API surface.

Centralized Log Collection

GCP services emit logs without configuration — that part is free, in the sense that you do not enable it. What gets collected by default: HTTP request logs from Cloud Run, App Engine, and load balancers; container stdout/stderr from GKE; serial console output from GCE; Cloud Audit Logs (Admin Activity always; others on demand); structured platform logs from every other managed service. For workloads on VMs or off GCP, the Ops Agent (the modern replacement for the older Cloud Logging Agent and Fluentd) collects system logs, application logs, and metrics in one process.

Sinks and Log Routing

→ BigQueryanalytics

Route logs to BigQuery to query them with SQL — usage analysis, security investigations, dashboards.

→ Cloud Storagearchive

Route to a bucket for cheap long-term retention and compliance archives.

→ Pub/SubSIEM

Stream to Pub/Sub to feed an external SIEM or a real-time processing pipeline.

A sink is just a filter plus a destination. The same log can fan out to several sinks at once.

A sink is a routing rule: log entries matching a filter are exported to a destination. Three common destinations:

• BigQuery — for SQL-based analytics, long-term audit, and joining with business data. The right place for any logs you want to keep beyond the default retention.
• Cloud Storage — for cold archival at the lowest cost. Logs land as files; analytics requires loading back into BigQuery.
• Pub/Sub — for real-time forwarding to external SIEM (Chronicle / Google Security Operations, Splunk, Sentinel) or to custom pipelines.

Sinks are the single most important Cloud Logging configuration. Without them, every log entry is stuck in the default _Default bucket with the default 30-day retention. With them, the team controls cost, retention, and downstream analytics deliberately.

Log Explorer and Filtering

The Log Explorer in the GCP console runs queries against logging buckets using Cloud Logging's filter syntax. For "show me errors from this service in the last hour" the UI is fine. For complex analytics — "which user agents account for the 5xx error rate, broken down by region and hour" — switch to Log Analytics, which exposes logs as BigQuery-style SQL. The two share data; the UI choice is about query complexity, not access.

Cloud Audit Logs — Special Status

Cloud Audit Logs are a special category that records every API call against GCP resources. Four sub-types:

• Admin Activity — administrative API calls. Always on, free, retained 400 days by default. Never disable; you will need them in an incident review.
• Data Access — data plane reads and writes (Cloud Storage object reads, Spanner row reads). Off by default for a reason: enabling everywhere at full detail produces volumes that dwarf application logs and runs an enormous bill. Enable selectively on sensitive resources. The one exception is BigQuery — its Data Access logs are always on and cannot be disabled.
• System Event — Google-initiated actions on your resources (live migration, automatic security patches). Always on, free, useful in audits.
• Policy Denied — IAM denials, VPC Service Controls denials. Generated by default, but unlike Admin Activity these are billed, kept 30 days in the _Default bucket, and can be excluded with a filter. Critical for debugging "why can't this work" and for catching attempted unauthorized access.

Log-Based Metrics

Cloud Logging can derive metrics from log content. A counter that increments when a log line matches a filter (every 5xx response from this service); a distribution that captures a numeric field from logs (latency_ms extracted from request logs). These metrics flow into Cloud Monitoring and feed dashboards and alerts. The pattern is powerful but cardinality is the trap: label dimensions with unbounded values (user IDs, request paths) explode metric series count and cost.

Retention, Storage Cost, and Log Analytics

Logging buckets each have a retention policy. The defaults: _Default bucket keeps 30 days; _Required bucket (Admin Activity audit, etc.) keeps 400 days; custom buckets can be configured up to 3650 days. Longer retention costs more linearly. For long-term audit beyond what custom buckets practically support, route to BigQuery.

Log Analytics is the BigQuery-backed analytics layer on top of logging buckets — without copying data, you query logs in SQL. The right tool for complex queries the Log Explorer does not handle well: aggregations, joins, regex extraction at scale.