Kafka connect Debezium Postgres Cloud SQL

Question

I am trying to connect to Cloud SQL postgres using Debezium Postgres connector. And I am getting the following error in logs.

connect_1    | org.apache.kafka.connect.errors.ConnectException: org.postgresql.util.PSQLException: FATAL: must be superuser or replication role to start walsender
connect_1    |  at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:127)

It occurred to me that I need to add REPLICATION to my user role. However, CloudSQL docs say the following.

When you create a new Cloud SQL for PostgreSQL instance, the default postgres user is already created for you, though you must set its password. The postgres user is part of the cloudsqlsuperuser role, and has the following attributes (privileges): CREATEROLE, CREATEDB, and LOGIN. It does not have the SUPERUSER or REPLICATION attributes.

Now because of this, I can't ALTER ROLE to add REPLICATION

ps: the cloud SQL instance is configured for high availability.

Google Cloud SQL is a managed service. You do not get those rights. If your application requires them, then you will need to spin up your own PostgreSQL on Compute Engine. Make sure that you really do need those roles. 99% of the time you do not. — John Hanley
– John Hanley, Commented Oct 20, 2019 at 0:13

Andrei Tigau · Accepted Answer · 2019-10-23 08:59:22Z

4

Debezium requires the Postgres wal2json extension, which is not currently supported by Cloud SQL.There is an ongoing feature request for this which you can check. You will need to setup Postgresql on GCE and setup Debezium there.

If this helps, do accept the answer

edited Oct 23, 2019 at 8:59

Andrei Tigau

2,0781 gold badge8 silver badges17 bronze badges

answered Oct 20, 2019 at 12:31

Jayadeep Jayaraman

2,8453 gold badges17 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Amit Basuri Over a year ago

thanks for the reply. I see the following Note from docs As of Debezium 0.10, the connector supports PostgreSQL 10+ logical replication streaming using pgoutput. This means that a logical decoding output plug-in is no longer necessary and changes can be emitted directly from the replication stream by the connector.

Jayadeep Jayaraman Over a year ago

That is correct and there is an internal feature request to make changes to allow non super users to enable replication. Once the feature is implemented you will be able to enable replication. Hope that helps.

Matt R. Wilson Over a year ago

Cloud SQL, finally, added logical replication Aug 2021. cloud.google.com/sql/docs/postgres/release-notes#August_30_2021

Collectives™ on Stack Overflow

Kafka connect Debezium Postgres Cloud SQL

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related