1,859 questions
0
votes
0
answers
76
views
How to get a deletion trigger to fire on a logical replica when a record is deleted on the source db
Due to performance concerns by the system devs and internal politics we cannot add triggers to the prod tables to capture deletions (long story, we need to to implement pseudo-CDC to a third party ...
- 3
0
votes
0
answers
46
views
Is Q Replication what's called Fast Queue in database async replication
In async replication with PostgreSQL, for example, source db will just transmit chunks of their WAL directly via network to target db, plain and simple. But I saw something like Q Replication by IBM ...
- 79
1
vote
0
answers
97
views
Perform continous data replication on Firebird database with read-only access
My task is to perform continous replication on a Firebird 3.0.5 database. I only have read-only access. The data will go to the Google Cloud Platform, either Cloud SQL or BigQuery.
Read-only means I ...
- 2,038
0
votes
2
answers
77
views
When does restart_lsn move forward in logical replication?
My understanding is that PostgreSQL retains WAL from restart_lsn because the logical decoder might need data before confirmed_flush_lsn to produce output for transactions that have not yet been ...
0
votes
0
answers
29
views
How do I replicate Oracle to PostgreSQL in real time with schema evolution support?
I want to replicate Oracle to PostgreSQL in real time with schema evolution support.
Is there is any open source tool available? I am not using Oracle Goldengate. It is too heavy and time consuming to ...
- 21
0
votes
0
answers
60
views
Distributor name as database name in SQL Server database
I was starting to configure replication on SQL Server database.
I created a distributor with the following command:
exec sp_adddistributiondb @database = N'prod_database',
@...
0
votes
0
answers
78
views
Last Synch Date of Replicated Tables on SQL Server
This question is about SQL Server replication.
I am running SQL Server 2019 v15.0.4420.2 and I get a large number of replicated tables from outside the organisation.
What I am looking for is a SQL ...
- 1,617
0
votes
0
answers
48
views
Is Lost Update without certification failure expected in Galera Cluster?
I'm using a multi-node Galera Cluster and encountered the following two scenarios:
Case 1 — Certification Conflict (Deadlock) occurs:
Node B and Node C both start transactions.
Both try to update the ...
1
vote
0
answers
36
views
replicating database in PostgreSQL
I want to replicate multiple databases to 2 different OS Centos9(master) OS Ubuntu(slave) but after the command
sudo -u postgres \
pg_basebackup -h primary-ip-addr \
-p 5432 \
...
- 11
0
votes
1
answer
112
views
PostgreSQL Standby slowing down Primary
Description
The following table lists 2 DB servers using PostgreSQL streaming replication for synchronization:
Server
Role
Method
Mode
db01
Primary
db02
Standby
Streaming replication
async
db02 is ...
- 29
0
votes
0
answers
51
views
PostgreSQL SSL Replication: Why is WAL Sender Encrypted but Not WAL Receiver?
I am setting up SSL encryption for PostgreSQL replication and have the following configuration:
PostgreSQL Version: 14.3 OpenSSL Version: 3.1.0
Configuration Details:
I have enabled encrypted ...
- 41
0
votes
1
answer
68
views
How to parse logical replication commands
Code
do $$BEGIN
perform pg_create_logical_replication_slot('test', 'pgoutput', false);
END$$ ;
create publication Jalgi_pub for all tables;
select * from pg_logical_slot_peek_binary_changes('test', ...
- 28.2k
1
vote
2
answers
149
views
Steps to stop replication on Postgres in primary
In Postgres, I want to stop the replication process on the primary. Secondary database is no more working and primary wal folder is growing in size. Can someone help me with what are the settings I ...
- 2,601
0
votes
0
answers
61
views
Scraping/chaining InfluxDB 1.8 data to InfluxDB 2.7
Is there a way to mirror or scrape data from an InfluxDB 1.8 database to a v 2.7.x database?
If I were given two v. 2 databases I think the case would be covered by the (edge) data replication. But I ...
- 333
0
votes
1
answer
101
views
Commit delay for MariaDB Galera cluster and MaxScale
I'm working in a clustered MariaDB database with Galera and MaxScale in r/w routing.
My question is if the commit command successful return implies a completed replication or not.
This is the scenario:...
- 9,605
0
votes
1
answer
112
views
Create named instance in SQL Server Express
I'd like to pull some data into SQL Server Express, from a high-current electrical test machine running SQL Server.
The first problem I'm experiencing is that I can't get replication setup, because ...
- 2,706
0
votes
0
answers
199
views
PostgreSQL v13 Streaming Replication Issue
Using PostgreSQL v13 streaming replication setup between a primary, and secondary (Hot Standby) server. After maintenance reboots of each server were complete, the secondary (a read-replica) is stuck ...
2
votes
0
answers
96
views
Suppress DBCC messages for sp_addarticle
Anyone know how to turn off this unwanted output? I have several stored procedures that are used to tear down and rebuild replication for hundreds of databases.
These stored procedures output lots of ...
- 45
0
votes
0
answers
34
views
Multiplexing read-only/read-write datasource
I've implemented CustomRoutingDataSource. It should multiplex between two DataSources depending on whether current transation is ReadOnly or not.
public class CustomRoutingDataSource extends ...
- 5,326
0
votes
1
answer
113
views
SymmetricDS How do I transform a MySQL Longtext column to a PostgreSQL Text column?
I am working on a replication between a old MySQL 8 database and a new PostgreSQL database.
This is a two way replication but I am having issues with syncing a LongText column in MySQL to a Text ...
- 1
0
votes
2
answers
54
views
SQL Server read only failover database CLR enabled
We have enabled CLR on the primary database with a read only synchronized readonly failover database and it works perfectly using the following instruction.
IF EXISTS (SELECT * FROM sys....
- 1,043
1
vote
1
answer
122
views
walsender process and connections via bouncer in Postgres is not using TLS Encryption
I have been testing TLS encryption for my Postgres database. All the external client connections are using SSL for encryption. I tried to query the pg_stat_ssl view, which will list the information ...
- 41
0
votes
1
answer
84
views
Postgresql Replication
I have a table in production whose structure I want to modify. I can ALTER the table to add column but the column gets added at the end, may be the order may not matter. But at least visually I see it ...
- 1,051
0
votes
0
answers
147
views
Logical replication not working and no errors in the logs
I have setup a logical replication in PostgreSQL and have made the changes in postgres.conf where i have set wal_level = 'logical'
The create subscription part works well but the create subscriber ...
- 3,140
0
votes
0
answers
67
views
Postgres data sync between different servers
I have 2 postgres clusters (A and B) on different servers. Whenever data is written to the table on server A, I want to copy the data to the table on server B.
But the data retention in the table on ...
- 13
0
votes
1
answer
889
views
Postgres logical replication stuck at LSN value. I want to advance past it with pg_replication_origin_advance
Postgres 13 on AWS RDS
Replication slot is growing in size
CPU and I/O are low on both master and slave
The subscriber seems to be stuck at 33533/7D2841D8 even though the received LSN keeps advancing. ...
- 6,469
0
votes
0
answers
362
views
Timeline corruption PostgreSQL
I have this error, and I don't know how to fix it correctly. I saw people doing pg_basebackup everytime it happens. There is any better way?
2024-08-27 12:27:41.521 CEST [1] LOG: starting PostgreSQL ...
- 135
0
votes
1
answer
1k
views
Postgres Hot standby stuck starting waiting for contrrecord despite wal file being present
We are experimenting with patroni Multi Datacenter HA with postgres16 using replication slots. At high level for building the remote cluster from scratch , we are by running base backup, while active ...
- 413
0
votes
1
answer
198
views
How to create notifications for Azure Data Sync errors or different way to schedule replication of Azure SQL data to on prem SQL Server?
We have a database in Azure SQL Server that we want to replicate continually to an On-Prem 2019 SQL Server. We are using Azure Data Sync but it fails every once in a while and we can't find out how to ...
0
votes
0
answers
102
views
Merkle Tree vs Replication Log / Event Log
I understand that Merkle Trees can be used to replicate data and guarantee
integrity of the data in byzantine environment (where you don't trust all the parties) and that systems like cansandra use it ...
- 41
0
votes
1
answer
570
views
Is Amazon Aurora RDS replication synchronous or asynchronous
We are using Amazon Aurora RDS with Postgres. We have Multi AZ deployment, with two instances, 1 writer and 1 reader replica. Now the question is, if the synchronization is synchronous or asynchronous ...
- 14.4k
0
votes
0
answers
167
views
DMS Source Data and Latency spike nightly
I'm using a SQL Server on-prem source for a DMS to replicate to a SQL Server RDS. Every night, at about midnight, I'm seeing a spike of ~1.6M CDC incoming changes on my DMS task, but nothing is making ...
0
votes
2
answers
519
views
How to acquire exclusive lock in replicated mysql and clustered servers?
I have this scenario:
My php web application is deployed in a clustered environment, it uses a replicated mariadb database with galera.
It has to acquire an exclusive lock on database row resource and ...
- 9,605
1
vote
1
answer
570
views
MySQL master-slave replication with SSL: got error certificate verfication failed
Last_IO_Error: Error connecting to source 'repl@mysql-master:3306'. This was attempt 1/86400, with a delay of 60 seconds between attempts. Message: SSL connection error: error:0A000086:SSL routines::...
- 41
0
votes
0
answers
3k
views
pg_basebackup error: connection to server, no pg_hba.conf entry for replication connection
There are similar posts and answers to my issue, however they could not help me.
I am using PostgreSQL v16.3 and the latest version of Docker.
I am creating a very basic PostgreSQL cluster using a ...
- 98
0
votes
1
answer
696
views
DMS replication task fails due to a corrupt binlog event
I have a mariadb RDS database deployed as a multi-AZ in my AWS account, engine version 10.6.16. This DB serves as the main backend database serving ~100 connections per second. I also have a DMS ...
- 41
0
votes
0
answers
2k
views
replication slot's wal_status reserved vs extended
When queried using
SELECT * FROM pg_replication_slots ;
There's one more column as wal_status which has can be mainly either reserved vs extended.
what is the main difference?
will one has higher ...
- 229
1
vote
1
answer
111
views
How can I sync replicas with each other?
I have two replicas for my main cassandra server. Both the replicas are not in sync with the main server. This happened because the replica servers were running into errors cause by commit logs for 3 ...
- 11
0
votes
1
answer
41
views
Postgres subscriber node took some time to reflect publisher config change which leads to postgres restart issue
I am aware of the postgres config caveats mentioned here: The easiest way to ensure this does not become a problem is to have these parameters set on the standbys to values equal to or greater than on ...
- 1,676
1
vote
1
answer
193
views
MongoReplicaSet - emulate replication lag
I'm using mongodb-memory-server for the e2e-testing on the project and want to identify potential problems with replication lag. Is there any way we can emulate a large replication delay?
Maybe there ...
1
vote
1
answer
1k
views
What happens with the old primary after a "pg_ctl promote" on a replica?
I have a PostgreSQL 11 streaming (ie. wal_type = replica) cascading replication scheme set up as follows:
db1 -> db2 -> db3
What happens to db1 and db3 if I execute a pg_ctl promote on db2?
...
- 2,511
0
votes
1
answer
195
views
does pg_restore + wal_type=replica create a consistent replica?
I have a PostgreSQL server that has a pg_dump scheduled every day at 3 AM (low traffic time window) for backup purposes. These dump files are gzipped and stored at a convenient location. This server ...
- 2,511
0
votes
1
answer
1k
views
AWS DMS Task Migration Mapping Rules - Add-Column using metadata
I need to create a new column for each record that DMS processes, right now I have the following mapping rule:
{
"rule-type": "transformation",
"rule-id": &...
- 3,626
1
vote
0
answers
263
views
Changing binlog_format in MariaDB slave
I have mariadb 10.3 master and slave running in AWS EC2, whose binlog format is MIXED for both. I want to migrate this db to Aurora MySQL using DMS and understand DMS needs the format to be in ROW.
I ...
- 73
1
vote
0
answers
72
views
How can I minimize data loss & data transfer when I failback with the requirement that I promote the old primary back to its primary status?
The definition of failback according to IBM docs is "the process of returning production to its original location after a disaster or a scheduled maintenance period".
From my reading of ...
- 672
-1
votes
2
answers
332
views
Postgresql Replication Slot grows in GB suddenly
We are using Qlik Replicate for replicating data from RDS Postgresql to Synapse.
There is logical replication slot involved in this process, so here is the case when we start the replication, the ...
0
votes
0
answers
64
views
Is there a specified order for data replication among different partitions in Cosmos DB?
I have 1 write region (e.g., US), and 2 read regions (e.g., US and EU). I have a collection, and I want to write into it, using different partition keys. I wonder, is there a specified order for data ...
- 3,757
0
votes
0
answers
263
views
What does the value 30 indicate in the type column in the MSrepl_commands table?
Result Set of MSrepl_commands
I want to infer that whether its a insert/update/delete command . We need to check at day end the amount of data replicated per article by the transactional replication. ...
0
votes
0
answers
205
views
How to Achieve Master-Master Replication in MongoDB for Two Databases in Different Locations?
I'm working on a project where I need to deploy two MongoDB databases in different geographical locations. I want both databases to work independently, allowing reads and writes on both sides. However,...
- 1
0
votes
2
answers
549
views
Real time replication from on premises Oracle to AWS
Is there any service offering from AWS, that does real time replication of data from Oracle which is on premises to Oracle instance running on EC2 in AWS ? But the catch is, the table structures are ...
- 409