Defining multiple index files in compilerOptions.paths in tsconfig.base.ts in an Angular NX project

In my Angular NX project I want to separate my app domains in folders with sub-folder for each feature/util/store etc. All exposed APIs are referenced in 'public-api.ts' barrel files in each of those subfolders. In my tsconfig.base.ts file I want to configure a Reference for each domain (like '@my-app/user'). Since the value of the 'paths' config object, in compilerOptions.paths is an array, I would assume I can reference multiple files there. But in praxis only the first file will be used for import lookups. My Folder structure looks like this:

libs
  user
    data-access
      src
        public-api.ts 
    user-feature
      src
        public-api.ts 
    user-utils
      src
        public-api.ts

and my paths config in tsconfig.base.ts contains this:

{
  compilerOptions: {
    ...
    paths: {
      '@my-app/user': [
         'libs/user/data-access/src/public-api.ts',
         'libs/user/user-feature/src/public-api.ts',
         'libs/user/user-utils/src/public-api.ts'
      ]
    }
 }

if I try to import files from that domain, like here, only the files from data-access/src/public-api are found:

import { UserStore } from '@my-app/user'; // Works
import { UserFeatureComponent } from '@my-app/user'; // Error: Module '"@my-app/user"' has no exported member 'UserFeatureComponent'.

Is there a way to configure this correctly without having references for each sub-folder like '@my-app/user/data-access', '@my-app/user/user-feature', '@my-app/user/user-utils'?