Typescript configuration with src and test

I have multiple TypeScript projects where I separated the source code from the tests to their directories (src and test). In the final package I would like to include only the source code without the tests since the runtime does not need to know anything about the test files, fixtures, etc.

If I have the following settings in the tsconfig.json:

{
  "compilerOptions": {
    // compiler options
    "outDir": "dist"
  },
  "include": ["src"]
}

And these in the package.json:

{
  // usual settings
  "main": "dist/index.js",
  "files": [
    "dist/**/*.js",
    "dist/**/*.d.ts"
  ]
}

Then the final package only includes the source code in the desired directory structure, which is what I want to see, but my environment is having issues. I am using Doom Emacs and for the tests tide throws errors like this:

Error from syntax checker typescript-tide: Error processing request. No Project.
Error: No Project.
    at Object.ThrowNoProject (/Users/ikaraszi/.../node_modules/typescript/lib/tsserver.js:152133:23)

If I change the tsconfig.json settings to include the test directory, then the tide errors go away:

{
  "compilerOptions": {
    // compiler options
    "outDir": "dist",
    "rootDirs": ["src", "test"]
  }
}

But then the directory structure for the distribution changes and there will be a dist/src and dist/test that I would like to avoid since then the users of my libraries would need to use strange import statements:

import { foo } from 'library/dist/src/foo';

I would like to avoid the extra src if possible, the dist is ugly enough but that is given.

I tried with multiple settings change the include property to have src and test, but the builds ends up in the dist directory with the same nested structure:

{
  "compilerOptions": {
    // compiler options
    "outDir": "dist"
  },
  "include": ["src", "test"]
}

I also tried to play with the package.json settings without any luck. Is there anything I can do without adding an extra step to the build process to remove the unnecessary extra directories?