How to Use Cloud Pub/Sub for Event-driven Architecture on GCP?

Pre-requisites: Google Cloud Services

Cloud Pub/Sub is a fully managed, scalable messaging system offered by Google Cloud Platform (GCP) that enables asynchronous communication between applications. It is a simple and reliable way to exchange data between services and applications in real time. With its event-driven architecture, Cloud Pub/Sub provides a flexible way to build scalable, reliable, and loosely-coupled applications that can handle large-scale data processing.

In this article, we will explore how to use Cloud Pub/Sub for event-driven architecture on GCP. We will go through the steps required to set up a Cloud Pub/Sub topic and subscription, publish and consume messages, and demonstrate how to integrate Cloud Pub/Sub with other GCP services.

Key Terminologies

• Topic: A named resource to which messages can be sent by publishers.
• Subscription: A named resource representing the stream of messages from a single, specific topic, to be delivered to the subscribing application.
• Publisher: An application or service that sends messages to a topic.
• Subscriber: An application or service that receives messages from a subscription.
• Message: The unit of data exchanged between publishers and subscribers.

Let’s say you are building a simple application that sends notifications to users when a new post is added to a forum. You can use Cloud Pub/Sub to build an event-driven architecture where the application publishes a message to a Cloud Pub/Sub topic whenever a new post is added, and subscribers consume these messages to send notifications to users.

Steps to use Cloud Pub/Sub for event-driven architecture:

Step 1: Sign up for a Google Cloud Platform (GCP) account if you don’t already have one. You can sign up for a free trial that includes $300 in credit to use on GCP services.

Step 2: Enable Cloud Pub/Sub API: In the Cloud Console, navigate to APIs & Services > Dashboard, click Enable APIs and Services, search for Cloud Pub/Sub, and enable the API.

Step 3: Create a Topic: In the Cloud Console, navigate to Pub/Sub > Topics.

Step 4: After Clicking on ‘Topics’ , Click on Create Topic.

Step 5: Give a name to your Pub/Sub topic such as “new-post-topic” and keep all the things as default and Click on ‘CREATE’.

Step 6: Create a Subscription: In the Cloud Console, navigate to your topic, click Create Subscription, give it a name such as “new-post-subscription”, and choose the delivery type as “Pull”. 

Step 7: Leave all other fields as it is and Click on ‘Create’.

Now you have done with creating Cloud Pub/Sub for event-driven architecture. 

Step 8: Publish a Message 

In your application code, whenever a new post is added, publish a message to the “new-post-topic” with the post information as the payload. 

Here’s an example code snippet in Python:

Python3

from google.cloud import pubsub_v1

publisher = pubsub_v1.PublisherClient()
topic_path = "projects/<project-id>/topics/new-post-topic"

def publish_message(post):
    data = post.encode("utf-8")
    future = publisher.publish(topic_path, data=data)
    print(future.result())

Step 9: Consume Messages

In your notification service code, set up a subscription to the “new-post-subscription” and consume messages whenever they are available. 

Here’s an example code snippet in Python:

Python3

from google.cloud import pubsub_v1

subscriber = pubsub_v1.SubscriberClient()
subscription_path = "projects/<project-id>/subscriptions/new-post-subscription"

def handle_message(message):
    print(f"Received message: {message.data}")
    # Send notification to users
    message.ack()

subscriber.subscribe(subscription_path, callback=handle_message)

while True:
    # Wait for messages
    time.sleep(1)

With these steps, your application can now use Cloud Pub/Sub to publish and consume messages and build a scalable and reliable notification system for new posts in the forum.