How to implement change streaming with cassandra?

Question

I'm running multiple python microservices that should use Cassandra as their database. In the spirit of microservices, each one "owns" its data, but is free to emit the data to anyone willing to listen. All of the microservices are communicating with each-other via kafka.

For example, I could have a microservice called auth that own private data about users. When a user updates his username or email, the change needs to happen in the auth-owned database. But, there are other microservices (e.g. scheduler) who can have a copy of all usernames.

The question is - what is the best way for auth service to notify scheduler service about data changes. In Mongo, I would use Change Streams.

Alex Ott · Accepted Answer · 2020-04-23 13:05:43Z

2

Cassandra doesn't have ready-to-use functionality like "Change Streams" in Mongo. Cassandra has Change Data Capture (CDC), but you need to write a code to consume the changes, you'll get the same changes on all nodes that hold the replicas, you don't get notifications about data expired via TTL, etc.

I personally would implement all communication via Kafka, and used something like DataStax's Kafka Connector for submitting data to different databases/tables (it's not open source, but it's free for usage with both DSE & Cassandra).

answered Apr 23, 2020 at 13:05

Alex Ott

88.1k10 gold badges110 silver badges157 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to implement change streaming with cassandra?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related