webpack doesn't work when migrating files from .jsx to .tsx

after setup TypeScript for a React project I encountered a problem with webpack-dev-server's hot reload while migrating .jsx files. Whenever i take an existing .jsx file and change the name ending to .tsx, webpack is missing the.jsx file for obvious reasons. It's obviously included somewhere in the webpack output, but missing on the filesystem.

so i have to re-compile the whole project again to make these changes work. Is there any way to make webpack hot reload recognize these changes?

    module: {
        rules: [
            {
                test: /\.m?js/,
                resolve: {
                    fullySpecified: false
                }
            },
            {
                test: /\.(js|ts)x?$/,
                include: [
                    CONTENT_DIR
                ],
                use: [
                    {
                        loader: 'babel-loader?cacheDirectory',
                        options: {
                            'plugins': [
                                ['babel-plugin-styled-components', {'ssr': true, 'displayName': true}],
                            ]
                        }
                    },
                ],
                exclude: /node_modules/
            },
        ],
    },
    resolve: {
        extensions: ['.js', '.jsx', '.tsx', '.ts'],
        unsafeCache: true,
    },

const ReactRefreshWebpackPlugin = require('@pmmmwh/react-refresh-webpack-plugin');

    plugins: [
        new ReactRefreshWebpackPlugin(),
    ],

Migrate .jsx files to .tsx. I expect the hot reload module to understand these changes and correct all imports to make sure its reading the output of the .tsx file instead without re-compile everything.