Setting Up PostgreSQL For pgStreaming
pgStreaming makes use of the PostgreSQL Logical Decoding support, the under-pinnings of Logical Replication in PostgreSQL.
To use pgStreaming with PostgreSQL we will need to:
-
Enable logical decoding in
postgresql.conf
- Install the Logical Decoding Output plugins needed
-
Create a user with the
replication
role
Enabling Logical Decoding
# postgresql.conf
# required for logical replication & decoding
wal_level = logical
# our application will be receiving a replication stream, so will use up a WAL sender server side
max_wal_senders = 10
# each consumer will need a replication slot, this will track the consumers position
max_replication_slots = 10
Installing Logical Decoding Output Plugins
Logical Decoding Output plugins process the logically decoded changesets within PostgreSQL and output data in the required binary format for the consumer to read. Currently pgStreaming supports two Logical Decoding Output plugins:
wal2json
and
postgres-decoderbufs
.
wal2json
The
wal2json
output plugin will output JSON formatted changesets. This is currently the best supported output plugin by pgStreaming.
Source: (https://github.com/eulerto/wal2json)[https://github.com/eulerto/wal2json]
This plugin is available as a package from the PGDG repos.
Creating A Replication User
A pgStreaming application will connect as if it is a replication client, rather than as a normal user. As such you will need a dedicated user for pgStreaming which has the
replication
role. As an aside on security, a pgStreaming application is capable of seeing every data change that happens on the PostgreSQL server for a specific database.