648 questions
0
votes
1
answer
85
views
Where should roles and missions/ACLs be managed when building an authorization server?
I’m trying to build my own Spring Authorization Server (for learning purposes and possibly to use across multiple projects in the future).
I’m already familiar with Spring Security and resource server ...
- 55
0
votes
0
answers
80
views
Write-through cache with priority using redis
My service is writing to clickhouse. I want to implement a cache which will help me with batching data for CH + it will store the data when CH is unavailable so i dont lose the data. I chose redis ...
- 628
0
votes
0
answers
28
views
NuSMV - Issue with counter-based signal activation in Model
I have a project for the Embedded System Design course where I need to model a fault-tolerant sensor reading mechanism. I have reduce the system to only the important parts. The system relies on error ...
0
votes
1
answer
78
views
How do I limit the count of rows in a result set in Postgres without unnecessary locking?
This question is inspired by a 'general admission' variant of the common 'event ticketing' System Design interview question. Critically in this version, the user does not select a seat - they only say ...
- 33
0
votes
1
answer
60
views
Should I Use Separate Authentication Flows for Users and Stores in My Backend
I have two entities: User and Store.
Users log in with email/password.
Stores access a POS app using a 6-digit token.
Should I:
Create separate authentication flows (Users: email/password, Stores: ...
1
vote
1
answer
56
views
Handling Multi-Table Updates in Kafka
We have an application that serves as a configuration repository, storing data in a relational database. Whenever a user changes a configuration item, it is persisted to the database and our goal is ...
- 28.6k
0
votes
0
answers
45
views
Load balancer setup in real time application
best to avoid single point of failure in system design in case of API gateway as from controller.
In real time companies how, they are handling SPOF with API gateways.
In below image API gate way down ...
2
votes
1
answer
103
views
I don't know how to get generic objects back from sql using boost c++
Some code:
selectOperation.cpp
#include "SelectOperation.h"
SelectOperation::SelectOperation(const std::string& tableName)
{
sql_statement = "SELECT * FROM " + tableName;
...
- 29
0
votes
2
answers
282
views
Discord Bot and a website communication, what should be the system design
I'm building a website that lets users add new items in their profiles. Users can also do the same using Discord bot's commands. In both cases the Discord bot sends a message New item added by @user ...
- 5,239
2
votes
1
answer
102
views
How should file storage access be organized in relation to the API gateway?
I'm building a system that consists of multiple microservices, databases, a message broker, and a file storage system that primarily stores photos. In my setup, I'm using MinIO as my file storage ...
- 317
0
votes
1
answer
87
views
Is using a local database as the Single Source of Truth (SSOT) ideal for real-time collaboration and offline sync in mobile apps? [closed]
Suppose I am designing a mobile application that requires both real-time collaboration and offline support. Instead of using a hybrid approach (directly interacting with the server when online and ...
- 9,783
1
vote
2
answers
46
views
Compliance Central Microservice
I have multiple microservices in my solution, let's call them salary processing, cards, kyc docs, and I need to enforce compliance so that if a kyc doc has expired, I block the card, and if salary ...
- 71
-1
votes
1
answer
41
views
Suggestions for overlapping functionality between Python Endpoints [closed]
Currently I'm working on FastAPI endpoints in Python which basically handle two types of entities:
/run/entityA
/run/entityB
Now based on the type of entity, some of the arguments change but a lot of ...
- 37
1
vote
0
answers
46
views
What is the expected behavior for stopping an inactive systemd service that is configured to auto-start?
I have a service file foo.service that is configured to auto-start after a target init_done.target.
[Unit]
After=init_done.target
...
[Install]
WantedBy=multi-user.arget
If I run systemctl stop foo....
- 1,035
0
votes
1
answer
63
views
API gateway role in micro services security
I'm new to microservices and system design, and I'm trying to understand the role of the API Gateway in authentication and authorization.
I've come across two different approaches, each with its pros ...
- 506
0
votes
0
answers
110
views
Understanding efficient use of Point in Time (PIT) with search_after in Elasticsearch
I am currently trying to solve a problem statement. We have an index where I want to fetch all documents at once and apply an update on few fields. This will require pagination across documents during ...
- 1,565
0
votes
1
answer
218
views
Creating a large Zip from files present in Blob storage in Dotnet core
We are trying to create a zip of all the files related to a particular environment (which would be 100s of 1000s in number with average size of 100KB). After zipping the size of zip file would be ...
0
votes
1
answer
135
views
How to implement event-based real-time status updates
I have recently seen a system design for bidding service and one of its use cases was real-time delivery of bid updates for a specific item (or a set of items) to the client.
The design was quite ...
- 3,574
1
vote
1
answer
94
views
Cassandra/Scylla Counters; What's the most common approach to tracking the count for emoji reactions?
I am trying to design a posts table, where a user can react to the post with any emoji. I need to keep a count of each unique emoji that is used.
If I use a separate Counter table, for each post I ...
- 43
0
votes
0
answers
66
views
How to Represent Role-Specific Relationships in an ER Diagram?
I’m new to drawing ER diagrams.
In my case, the Staff entity has an attribute called role with values like Head of Department, Dean, and Admin.
The Department entity should only have a relationship ...
0
votes
2
answers
103
views
How the 2-way interaction between Redis and Rate-limiter will take place in this design
Given the diagram from Alexu Book of designing Rate-limiter.
I am wondering how the 2 way communication between Redis and rate limiter will take place.
The 2 way communication has been shown using ...
1
vote
1
answer
311
views
Fargate task-level SNS topic subscriptions
In current state, I have a AWS Batch, which is configured to run ML training jobs. At the end of its flow, model artifacts are stored in an S3 bucket. Additionally, I have an AWS Application Load ...
- 1,238
0
votes
2
answers
129
views
How to handle multiple connections to microservice behind LoadBalancer
If there is a restful microservice A which has 50 instances of the same services, and if each instance can handle a million concurrent connections, total 50 instances can handle around 50 million ...
- 2,293
1
vote
1
answer
229
views
Redis backed rate limiter -- Inconsistent?
I was following this blog on implementing Rate Limiter using Redis.
Link to the blog
Here they have used MULTI to pack all the atomic commands. This ensures that we're not concurrently writing wrongly ...
- 142
0
votes
1
answer
43
views
Optimizing retry intervals for fetching transaction status from a payment gateway
I have a system where transactions are initially marked as "Pending" in a relational database, if a payment transaction is not immediately successful by payment gateway, I need to fetch the ...
0
votes
1
answer
57
views
Rate Limit per user for sending notification like 1 notification in X seconds
I have a use case where there is a stream of messages in Queue and I want to send this message to the user using a consumer job with these limitations
There a limit like I can send only 1 ...
1
vote
1
answer
695
views
Is my understanding of a Distributed Lock correct?
I'm having some trouble understanding the need of a distributed lock. I did think of an example where it may be required but I'm not completely sure. I would appreciate some comments if I'm thinking ...
0
votes
1
answer
95
views
Can GRPC and GraphQL endpoint be there in single service
I have a service written in Kotlin which has GRPC API's written in it. I want to know can there be graphQL endpoints in the same service or a new service should be created for it. Will it be possible ...
- 11
-2
votes
1
answer
46
views
System Design-Decorator design pattern typescript
Requirement
create different type of decorator and used it with user class so we can add specific details of different type of user based on decorator and in my code I have two decorator one is ...
- 99
0
votes
1
answer
401
views
Cannot build with style dictionary getting "TransformStream is not defined" error
i'm trying to use figma tokens and therefore to transform them to scss files with style-dictionary package.
however, when i'm trying to build using the cli command style-dictinary build this is the ...
- 305
2
votes
0
answers
129
views
Concurrency issues when locking using Concurrent Dictionary
I am writing code for a wallet system. At some point we receive a webhook from our payment provider notifying us of a successful deposit. The event is saved & stored and queued as a background job ...
- 61
0
votes
1
answer
190
views
How to deal with temporal business rules in DDD+Event Sourcing?
Say we have an OTP entity that represents a one-time password that expires after e.g. 15 minutes.
Our system is event-sourced and adheres to DDD principles; for example, we have the following event:
...
- 13.6k
1
vote
1
answer
675
views
How to Build a Microservices Architecture with Centralized Authentication and Secret Management from Scratch like Google?
I am currently working on a project that involves setting up a microservices architecture with centralized authentication, authorization, and secret management.
I want to implement a centralized ...
0
votes
0
answers
62
views
Designing a multiple async/await system that is error resilient and retry capable within a multi iteration loop
I am designing a system in nodejs where I am handling 1000s of rows of user data. Each row of data will have a multiple async/await call chain to a third party endpoint, where response at each step is ...
- 13
0
votes
1
answer
68
views
How to create rich models without depending on infrastructure? (Database or external services)
I'm grappling with a challenge in domain-driven design (DDD) regarding achieving a Rich Model.
In complex systems, it often feels nearly impossible to execute most business rules without access to ...
- 133
0
votes
0
answers
187
views
Android System Design - difficulty creating High Level Design
I would need some guidance from your side.
I'm preparing for a System Design Interview but need help creating an HLD. I'm coming from the perspective of doing an Architectural Design Graph. There, you ...
0
votes
1
answer
61
views
Concurrent Locks issue
I'm new to Database and Microservices. I'm trying to understand an edge case which is very unlikely to happen but can happen. Assume there in 1 object in inventory and two users trying to process at ...
- 21
1
vote
1
answer
375
views
Using Kafka consumer in rest api CRUD Repository
We have rest-api based java service (spring REST application in Tomcat server), which is basically handling the CRUD operation against the database. Now we have a requirement to consume some kafka ...
- 97
2
votes
1
answer
125
views
What happens when B-trees database have data larger than its page size?
In the book Designing Data Intensive Applications, it says:
By contrast, B-trees break the database down into fixed-size blocks or
pages, traditionally 4 KB in size (sometimes bigger), and read or ...
- 682
0
votes
1
answer
116
views
What happens if append-only database crash during append?
In the book "Designing Data Intensive Applications", it says
Concurrency and crash recovery are simpler if segment files are
append-only or immutable. For crash recovery, you don't need to ...
- 682
0
votes
1
answer
1k
views
Handle 100K request per second
How can i design a system that handle 100k req/s at peak time, each request performs an update of an object within 20kb size?
I used nodejs as backend for non blocking io, write-behind pattern with ...
- 13
0
votes
1
answer
102
views
Is it possible to have Webhook as generic component
Let us say I have a set of systems I can integrate with, but it takes long time to implement integrations with all of them. So I build a software to communicate with other apps using Webhook. Where I ...
- 21
0
votes
3
answers
277
views
How to difference a requirement/functionality from Business Logic?
I am documenting a project and I have found myself wondering if this is a requirement, a functionality or business logic, after all it is a chat system and if this is not the business logic I would ...
-1
votes
1
answer
53
views
why they use http in apis (mobile / desktop / web / ...) , isn't http just for web
Essentially, what I'm asking is, why use HTTP for anything when TCP alone could suffice? HTTP operates on top of TCP, so by sticking to TCP alone, you simplify parsing and avoid the added complexity ...
1
vote
0
answers
334
views
Using a Sharepoint list as a global variable for concurrent PowerAutomate flows leading to issues
So I'm using power automate for the first time to build out a reservation system for 4 packages available for rent. This is my first time using power automate, but I figured a reservation system is a ...
- 277
1
vote
1
answer
69
views
System design - Server sends request but goes down, does it receive response
I have the below basic design for my service.
Flow:
User sends a request to browser. The request hits the API service through the load balancer. The load balancer redirects request to available ...
- 79
2
votes
2
answers
630
views
Effectively sorting when your data is distributed across different microservices
In a microservices environment, how do we effectively sort data which is part of two different microservices.
For example, lets say we have two microservices
Users Service (Has all information ...
- 389
0
votes
1
answer
185
views
How failures and restore operations in sharding (consistent hashing)
In consistent hashing, suppose we are using username as for hashing
hashFunction(username) = nodeA
Now from what I understand, if there is any failure or a node is removed requests will be directed to ...
- 47
1
vote
1
answer
193
views
Is data in write through cache reverted in case of transaction failure?
In a write-through caching strategy, data is first written to the cache and then immediately persisted to the database.
Reference - https://codeahoy.com/2017/08/11/caching-strategies-and-how-to-choose-...
- 47
0
votes
1
answer
40
views
how to improve DB performance by removing the query on cabs or booking table for cab management system
I have a cab management system with main entities
public class Cab {
private int cabID;
private String registrationNumber;
private String model;
private int driverID;
// ...
- 347