Routing not working as expected in React app using react-router v6

I'm using react-router v6. I was following the tutorial https://reactrouter.com/en/main/start/tutorial#the-root-route and added my routes as specified. But there are weird things happening and I can't figure out why so.

I have created the router as specified using createBrowserRouter method.

const router = createBrowserRouter([
      {
        path: '/',
        element: <RootPage />,
        errorElement: <ErrorPage />,
      },
      {
        path: 'hi',
        element: <ErrorPage />,
        // errorElement: <ErrorPage />,
      },
    ]);

In my app.js file I have this:

<RouterProvider router={router} />

But if I type in localhost:4000/hi in my browser's address bar, I get

Cannot GET /hi

but if go to this route via a Link in my RootPage component, it works absolutely fine. Btw, this link is the Link component from react-router-dom.

The code for my RootPage component is as follows:

import { Link } from 'react-router-dom';

const RootPage = () => {

  return (
    <div>
      Home Page
      <Link to="hi">Go to hi route</Link>
    </div>
  );
};

export default RootPage;

Ideally even if I directly type in my url i.e http://localhost:4000/hi, I should be able to see my component get rendered instead of Cannot GET /hi. Any help would be appreciated.

Also I have configured my webpack manually. The start script to run my project is as follows :

"webpack serve --config webpack/webpack.config.js --env env=dev"

the contents of my webpack.config.js file are:

const { merge } = require('webpack-merge');
const commonConfig = require('./webpack.common.js');

module.exports = envVars => {
  const { env } = envVars; //check the script commands in package.json for env variable
  const envConfig = require(`./webpack.${env}.js`);
  const config = merge(commonConfig, envConfig);
  return config;
};

my prod config for webpack are:

const webpack = require('webpack');
const BundleAnalyzerPlugin =
  require('webpack-bundle-analyzer').BundleAnalyzerPlugin;

module.exports = {
  mode: 'production',
  devtool: 'source-map',
  plugins: [
    new webpack.DefinePlugin({
      'process.env.name': JSON.stringify('DAT UI prod'),
    }),
    new BundleAnalyzerPlugin(),
  ],
};

For dev:

const webpack = require('webpack');
const ReactRefreshWebpackPlugin = require('@pmmmwh/react-refresh-webpack-plugin');

module.exports = {
  mode: 'development',
  devtool: 'cheap-module-source-map',
  devServer: {
    port: 4000,
    hot: true, //enable webpack hot module replacement
    open: true,
  },
  plugins: [
    new ReactRefreshWebpackPlugin(),
    new webpack.DefinePlugin({
      'process.env.name': JSON.stringify('DAT UI dev'), //defining a env var 'name' having value 'DAT UI dev'
    }),
  ],
};

Common webpack config:

const path = require('path');
const HtmlWebpackPlugin = require('html-webpack-plugin');
// const CopyPlugin = require('copy-webpack-plugin');

module.exports = {
  entry: path.resolve(__dirname, '..', './src/index.tsx'), // entry point for our app
  resolve: {
    extensions: ['.tsx', '.ts', '.js'], // allows us to leave off extensions while importing
  },
  module: {
    rules: [
      {
        test: /\.(ts|js)x?$/,
        exclude: /node_modules/,
        use: [
          {
            loader: 'babel-loader', // use babel-loader for files with ts,tsx,js,jsx excluding node_modules
          },
        ],
      },
      {
        test: /\.css$/,
        use: ['style-loader', 'css-loader'], //The order is important here as we want css-loader to run first
      },
      {
        test: /\.(?:ico|gif|png|jpg|jpeg)$/i,
        type: 'asset/resource', //use this module to resolve these above mentioned files
      },
      {
        test: /\.(woff(2)?|eot|ttf|otf|svg|)$/,
        type: 'asset/inline',
      },
    ],
  },
  output: {
    path: path.resolve(__dirname, '..', './build'),
    filename: 'main.[contenthash].js', // instructing webpack that bundled code be placed in main.js inside build folder
  },
  plugins: [
    new HtmlWebpackPlugin({
      template: path.resolve(__dirname, '..', './src/index.html'),
      /* 
      inserts bundle.js inside of index.html, we don't manually need to specify script tag in index.html
      also we might define the output filename as bundle.[contentHash].js i.e a dynamic file name, which changes when our code changes,
      we did so for cache busting i.e prevent browser from caching our code file and not updating when our site updates
      so this plugin will help insert our js file automatically in the index.html for us
       */
    }),
    // new CopyPlugin({
    //   patterns: [{ from: 'source', to: 'dest ', noErrorOnMissing: false }],
    // }),
  ],
  stats: 'errors-only',
};