Overview of the Google Cloud Platform (GCP) Cloud Pub/Sub Product

Below is an overview of the Google Cloud Platform (GCP) Cloud Pub/Sub product.

*Double-tap to expand/collapse an item. Left-tap & drag to move. Mouse-wheel/pinch to zoom.
Knowledge Graph |  Text |  Top
Google Cloud Platform (GCP) Cloud Pub/Sub Product Overview Google Cloud Platform (GCP) A global, multi-tenanted, managed, distributed, real-time messaging service Use with Dataflow in order to get exactly-once, ordered processing Concepts Topic Subscription Publisher Subscriber - can be many subscribers on the same subscription IoT Data Ingestion Challenges Handle data streaming from various processes or devices Distributing event notifications for multiple systems Scale to handle volume Reliable delivery of messages, and without duplicates Features At-least-once delivery Protocol Subscriber ACKs each message Message is resent if subscriber takes longer than the 'ackDeadline' setting to respond - so subscriber COULD get a DUPLICATE message A subscriber can extend the deadline per message - to the reduce chance of missing the deadline and getting a duplicate message No provisioning, auto-everything Open APIs - all libraries are just doing REST calls Global Topic/subscription namespaces are globally unique Messages are stored in the region closet to the publisher (and duplicated in multiple availability zones) A subscription collates topic messages stored in different regions Subscribers can be anywhere End-to-end encryption Messages are durable and persist for 7 days Unordered message delivery Push (webhook, low-latency) and pull (for large number of subscribers) delivery flows Fast - order of hundreds of milliseconds Client libraries in Java, Python, C#, Ruby, PHP, Node.js, and auto-generated in 10 gRPC languages Fan In - multiple subscribers listening to a single topic Fan Out - multiple publishers publishing to a single topic How To: Topics Create a topic: gcloud pubsub topics create mytopicname Create a topic and publish a message: gcloud pubsub topics create mytopicname "my message" Create a topic in Python: from google.cloud import pubsub ; publisher = pubsub.PublisherClient() ; event_type = publisher.topic_path(project, "mytopicname") ; publisher.create_topic(event_type) ; publisher.publish("mytopicname", b'my message', extra='extra attribute', myid="1234") Batch messages in a single request in Python: with publisher.batch() as batch: ; batch.publish(payload1) ; batch.publish(payload2, extra='attribute', myid="1234) Subscriptions gcloud pubsub subscriptions create --topic mytopicname mysubname gcloud pubsub subsriptions pull --auto-ack mysubname sub = topic.subscription(mysubname) ; sub.create() ; results = sub.pull(return_immediately=True) ; if results: ; subscription.acknowledge([ack_id for ack_id, message in results]) gcloud pubsub subscriptions delete mysubname