I built a multi-tenant microservice architecture in Nest.js, multi-tenant connection are made using TypeOrm latest Datasource API. After upgrading to latest TypeORM version we encountering the MySQL "Too many connections" error.
During searching about this I found that in their latest version they added option "PoolSize" to control number of active connections. I've added that too but the issue is still there.
Ideally, TypeOrm should close the connection once the DB operation finished or use opened connection (if any) on new request, but the connection to MySQL is keeping active but in sleep state, and on a new request it create new connection. see below:
By running show processlist; cmd on MySQL
I've created the multi-tenant connection using nest.js provider for incoming request in microservice:
databaseSource is used for initial database connection to default database and then on each tenant request we create the new DB connection.
export const databaseSource = new DataSource({
type: process.env.DB_CONNECTION,
host: process.env.DB_HOST,
port: parseInt(process.env.DB_PORT, 10),
username: process.env.DB_USERNAME,
password: process.env.DB_PASSWORD,
database: process.env.DB_DATABASE,
entities: ["src/models/entities/**/*.ts"],
synchronize: true,
poolSize: 10,
});
const connectionFactory = {
provide: CONNECTION,
scope: Scope.REQUEST,
useFactory: async (request: RequestContext) => {
const tenantHost = request.data["tenantId"] || request.data;
if (tenantHost) {
const tenantConnection: DataSource = await getTenantConnection(
tenantHost
);
return tenantConnection;
}
return null;
},
inject: [REQUEST],
};
@Global()
@Module({
providers: [connectionFactory],
exports: [CONNECTION],
})
export class TenancyModule {}
export async function getTenantConnection(
tenantHost: string
): Promise<DataSource> {
const tenantId = getTenantId(tenantHost);
const connectionName = `${tenantId}`;
const DBConfig: DataSourceOptions = {
type: process.env.DB_CONNECTION,
host: process.env.DB_HOST,
port: parseInt(process.env.DB_PORT, 10),
username: process.env.DB_USERNAME,
password: process.env.DB_PASSWORD,
database: connectionName,
entities: ["src/models/entities/**/*.ts"],
synchronize: true,
poolSize: 10,
};
const dataSource = new DataSource(DBConfig);
if (!dataSource.isInitialized) {
await dataSource.initialize();
}
return dataSource;
}
Then once datasource initalized,I inject it into Service and used it to getRepository and performed DB operation.
I researched a lot about this some saying increase the MySQL "max_connections" limit, some saying passed the "connectionLimit" options in TypeOrm config (poolSize in latest version) but nothing works for me.
Am I doing anything wrong to create the tenant connection? Is there any way to closed the connection manually after DB operation?
