Install and configure Apache Pulsar implementation in Docker

1. Install Docker and Docker Compose

Make sure Docker and Docker Compose are installed on your system. If you have not installed it, you can refer to the following link to install it:

Docker installation guide
Docker Compose Installation Guide

2. Pull the Pulsar image

Pulsar's official Docker image can be obtained through Docker Hub. First, pull the latest Pulsar image:

docker pull apache/pulsar

3. Create Pulsar's Docker Compose file

To make it easier to manage and start multiple services, it is recommended to use Docker Compose. Create a name calledThe file, the content is as follows:

version: '3'  # Docker Compose version used
services:
  # Pulsar service definition, responsible for the main work of messaging  pulsar:
    image: apache/pulsar  # Use the official Pulsar image    environment:
      # Configure the JVM memory size      - PULSAR_MEM="-Xms2g -Xmx2g"  # Set the heap memory size of Pulsar      # Set the tenant and cluster name of Pulsar      - PULSAR_TENANT=public  # Set the default tenant      - PULSAR_CLUSTER=standalone  # Set the cluster name to "standalone"      # Configure the address of the Zookeeper service. Pulsar relies on Zookeeper to manage cluster metadata      - PULSAR_ZOOKEEPER_SERVERS=zookeeper:2181  # Specify the service address of Zookeeper      # Configure Pulsar's client, HTTP, and management service ports      - PULSAR_BROKER_SERVICE_PORT=6650  # Pulsar's client service port      - PULSAR_HTTP_SERVICE_PORT=8080  # Pulsar's HTTP interface service port (for Web UI)      - PULSAR_MANAGEMENT_SERVICE_PORT=8090  # Pulsar's management interface service port    ports:
      # Bind the local port to the container port      - "6650:6650"  # Map Pulsar client connection port      - "8080:8080"  # Map Web UI ports (used to access Pulsar Dashboard)      - "8090:8090"  # Map management interface ports    depends_on:
      # Pulsar relies on Zookeeper and Bookkeeper services to start      - zookeeper
      - bookkeeper
    command: &gt;
      bin/pulsar standalone  # Start Pulsar's standalone mode (suitable for standalone deployment)    networks:
      - pulsar-net  # Use a network called pulsar-net
  # Zookeeper service definition, coordination service for Pulsar cluster  zookeeper:
    image: wurstmeister/zookeeper  # Use the Zookeeper image provided by Wurstmeister    environment:
      - ZOOKEEPER_CLIENT_PORT=2181  # Zookeeper's client connection port    ports:
      - "2181:2181"  # Map Zookeeper client connection ports    networks:
      - pulsar-net  # Also use the pulsar-net network to enable communication between services
  # Bookkeeper service definition, Pulsar's persistent storage  bookkeeper:
    image: apache/pulsar  # Use the official Pulsar image because Bookkeeper is part of Pulsar    environment:
      - PULSAR_MEM="-Xms2g -Xmx2g"  # Set the JVM memory size of Bookkeeper      - PULSAR_ZOOKEEPER_SERVERS=zookeeper:2181  # Configure Zookeeper address (Bookkeeper also requires Zookeeper to coordinate)      - PULSAR_METRICS_PROVIDER=  # Configure performance metric providers    depends_on:
      # Bookkeeper relies on Zookeeper service startup      - zookeeper
    networks:
      - pulsar-net  # Also use pulsar-net network
# Define the networknetworks:
  pulsar-net:
    driver: bridge  # Use bridged network drivers to ensure containers can communicate with each other

thisThe file defines three services:

Pulsar: As a Pulsar broker, provide messaging services.
Zookeeper: Used to coordinate the metadata and state of the Pulsar cluster.
Bookkeeper: Used to persist messages.

Comment explanation:

version: '3':
- Specifies the version of Docker Compose. This is the standard format version of the Compose file, which is usually compatible with the version of the Docker engine.
services::
- One or more services are defined, each of which is a Docker container. In this configuration, we define three main services:pulsar、zookeeperandbookkeeper。
pulsarServe:
- image: apache/pulsar: Use the official Pulsar image.
- environment:: Set environment variables to configure the Pulsar container. Including memory size, cluster name, Zookeeper address, port, etc.
- ports:: Map Pulsar's container port to the host's port.6650For client connection,8080For use in the Web UI,8090Used to manage services.
- depends_on:: Define Pulsar service dependencyzookeeperandbookkeeperServices start, ensuring that these services are running before Pulsar starts.
- command:: Specify the command when starting the container, use it herebin/pulsar standaloneTo start Pulsar's stand-alone mode.
zookeeperServe:
- image: wurstmeister/zookeeper: usewurstmeister/zookeeperMirror, this image provides a Zookeeper container instance.
- ports:: Turn the Zookeeper port2181Exposed to the host, Pulsar communicates with Zookeeper through this port.
bookkeeperServe:
- image: apache/pulsar: Bookkeeper shares the same image with Pulsar because it is part of Pulsar.
- depends_on:: Bookkeeper relies on the Zookeeper service, so it must be started after Zookeeper is started.
networks::
- Define a name calledpulsar-netto ensure that all services can communicate with each other under the same network. This network usesbridgeDriver, it is Docker's default network driver.

Runtime steps:

usedocker-compose up -dStart the service.
usedocker-compose psCheck the container status.
Access using a browserhttp://localhost:8080, enter Pulsar's Web UI.

4. Start the Pulsar cluster

existRun the following command in the directory where the file is located to start the Pulsar cluster:

docker-compose up -d

This command pulls the image and starts the Pulsar, Zookeeper, and Bookkeeper services.

5. Verify that the Pulsar cluster is running normally

You can check the running status of the Docker container by:

docker-compose ps

If the status of all containers is "Up", it means that the service has been successfully started.

6. Access the Pulsar Web UI (optional)

Pulsar provides a web UI interface that can be used to manage and view cluster status. You can accesshttp://localhost:8080Come check out Pulsar's web console.

7. Test using the Pulsar command line client

You can use the command line tool provided by Pulsar to test whether the cluster is working properly. First enter the Pulsar container:

docker exec -it <pulsar_container_id> bash

Then, you can usepulsar-clientCommands to send and receive messages. For example, send a simple message:

bin/pulsar-client produce my-topic --messages "Hello Pulsar"

Receive message:

bin/pulsar-client consume my-topic -n 1

8. Configure multi-node Pulsar cluster

For production environments, you may need to configure a multi-node Pulsar cluster. In this case, you need:

Deploy multiple instances of Pulsar, Zookeeper, and Bookkeeper.
Configure Pulsar's cluster and partition settings to ensure high availability and scalability.

For multi-node cluster configurations, you need toConfigure different nodes in the , and set the correct environment variables. Container instances can also be deployed on multiple hosts.

9. Configure Pulsar client connection

In the client application, you need to use Pulsar's client to connect to your Pulsar cluster. Here is a C# client code example using the Apache Pulsar C# client library:

using ;

class Program
{
    static async Task Main(string[] args)
    {
        var client = await (new ClientConfiguration
        {
            ServiceUrl = "pulsar://localhost:6650"
        });

        var producer = await ()
            .Topic("my-topic")
            .CreateAsync();

        await (Encoding.("Hello Pulsar"));

        var consumer = await ()
            .Topic("my-topic")
            .SubscriptionName("my-subscription")
            .SubscribeAsync();

        var message = await ();
        (Encoding.());

        await ();
    }
}

10. Clean and Stop Pulsar Cluster

If you need to stop and clean up the Pulsar cluster, you can run:

docker-compose down

Summarize

Deploying a Pulsar cluster with Docker is ideal for development and testing environments. Through the above steps, you can quickly build a single or multi-node Pulsar cluster and use the Web UI, command line client, or programming interface for messaging.

This is the article about installing and configuring Apache Pulsar implementation in Docker. For more related content on installing and configuring Apache Pulsar, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!