Service 24

Cloud Tasks

QueueAsync

Cloud Tasks is the managed task queue for deferred, rate-limited, retry-aware HTTP work directed at a specific endpoint. Engineers reach for it after they have tried to make Pub/Sub do this job — usually unhappily. Pub/Sub is fan-out (one publish, many consumers, broadcast); Cloud Tasks is the opposite (one task, one endpoint, controlled dispatch rate, per-task retry policy).

The right mental model: Cloud Tasks is a queue of HTTP calls you want to make later, with deadlines and retries. Pub/Sub is a stream of events you want others to react to. Different shapes, different tools.

Tasks vs Pub/Sub

The most common architectural mistake in this chapter is choosing the wrong one of these two. A few decision rules:

One specific endpoint should be called with retry on transient failure → Cloud Tasks.
Many independent consumers should learn about an event → Pub/Sub.
External rate limit (third-party API allows 10 calls/sec) needs to be enforced on outbound calls → Cloud Tasks, with dispatch rate set.
Order does not matter, only that someone reacts → Pub/Sub.

Cloud Tasks vs Pub/Sub

Cloud Tasks — one task targets one HTTP endpoint with explicit retry policy and dispatch rate limits. Use when you need to call a specific URL later, rate-limited, with retry on transient failure.

Pub/Sub — one publish reaches every subscription on the topic. Use for broadcasting an event so multiple components can react independently, with no per-consumer rate control.

HTTP Target Queues

An HTTP target queue dispatches each task as an HTTPS request to the URL you specified when enqueuing. The body, headers, and method are stored with the task. Cloud Tasks waits for the dispatched endpoint to return 2xx and acks the task; non-2xx triggers retry per the queue policy. For private endpoints (Cloud Run, internal load balancers), attach an OIDC token to the task so the request authenticates correctly.

Rate Limits and Retry Configuration

Two layers of control: queue-level rate limits (max dispatches per second, max concurrent dispatches) and per-queue retry policy (max attempts, min/max backoff, max retry duration). Set these explicitly on every production queue. The default retry policy is generous — 100 attempts with exponential backoff, and setting max_attempts to -1 removes the cap entirely — so a permanently failing endpoint keeps retrying for a long time before giving up.

Scheduled Execution

A task can be enqueued with a schedule_time in the future — Cloud Tasks holds it until then, then dispatches. This makes Cloud Tasks the right tool for one-off deferred work ("send this reminder email in 24 hours"). Cloud Scheduler is for recurring cron-style schedules; Cloud Tasks is for individual deferred dispatches.

Authentication for HTTP Tasks

For any non-public endpoint, configure the task with an OIDC token tied to a service account that the destination authorizes. Cloud Tasks mints the token at dispatch time and attaches it as an Authorization header. Without this, calls to Cloud Run with --no-allow-unauthenticated, internal load balancers, or any IAM-gated endpoint return 401 and stack up in retries.

Common Mistakes

Using Pub/Sub for what is really rate-limited HTTP work to a single endpoint with retries. Pub/Sub has no built-in rate control; the workload ends up reimplementing it badly.
Using Cloud Tasks for broadcast scenarios where multiple components should each react independently — one task can only target one endpoint.
Leaving the default retry policy on the queue. A permanently failing endpoint burns through 100 attempts of backoff-and-retry, consuming budget before anyone notices.
HTTP tasks to private endpoints without OIDC tokens. Every dispatch returns 401; the queue depth grows.
Cloud Scheduler used for a single deferred dispatch ("send this in 24 hours") when Cloud Tasks with schedule_time is the cleaner fit.
Non-idempotent task handlers. Retries are guaranteed under transient failure; double-processing breaks correctness.

Best Practices

Use Cloud Tasks for retry-able rate-limited HTTP work to a specific endpoint. Use Pub/Sub for broadcast events.
Set explicit max_attempts, min_backoff, max_backoff, and a max retry duration on every production queue.
OIDC tokens on tasks targeting private endpoints. Verify auth works once before enqueuing at scale.
Idempotent task handlers. Treat retries as the normal case, not the exception.
Monitor queue depth and dispatch error rate with Cloud Monitoring alerts.
Match queue dispatch rate to the destination's actual capacity — overflowing a downstream service does not produce more throughput, just more 5xx.

Comparable services AWS SQS (with delay queues) Azure Storage Queue + Service Bus

Knowledge Check

When should you pick Cloud Tasks over Pub/Sub?

When you need to deliver one event to many independent consumers that each react to it in parallel
When you need to call one specific HTTP endpoint with retry and a controlled dispatch rate
When messages must be delivered exactly once with no duplicates possible
When messages must be retained for replay over multiple weeks

A Cloud Tasks queue dispatches HTTP tasks to a private Cloud Run service. Every dispatch returns 401. Most likely cause?

The queue's max dispatches per second exceeds the Cloud Run instance concurrency limit, so excess requests are turned away
Tasks are dispatched without an OIDC token, so requests hit the private endpoint unauthenticated
Cloud Tasks cannot target Cloud Run services regardless of authentication
The queue's retry policy is set to drop messages on first failure

What is the practical risk of leaving the retry policy at its defaults on a production Cloud Tasks queue?

Tasks fail silently because the queue does not retry by default
A permanently failing endpoint burns through 100 retry attempts, consuming budget and obscuring genuine error signals
The queue refuses new enqueues until existing tasks complete
Cloud Tasks silently switches the queue from pull mode to push mode and starts rejecting any task created by a manual dispatch call

Which scenario is Cloud Tasks the right fit for over Cloud Scheduler?

A recurring nightly ETL job that runs at 03:00 UTC
Triggering a function every hour to refresh a cache
A one-off deferred dispatch — send a reminder email to this specific user in 24 hours
Scheduling a weekly report to be generated every Monday at 08:00 and emailed to the whole team

Why do Cloud Tasks handlers need to be idempotent?

Cloud Tasks lacks transactional dispatch and the same task is always delivered twice for redundancy
Retries on transient failure are part of normal operation — a non-idempotent handler will double-process some tasks
Idempotency is required for Cloud Tasks to accept enqueue requests at all; the API probes the target and rejects any non-idempotent endpoint
Cloud Tasks deduplicates only by task name, never by content, so duplicates by intent are common

You got correct