React application production build static folder path changes

I am using react-app-rewired to build my react application with the following in package.json.

"scripts": {
    "format": "prettier --write",
    "start": "PORT=3001 react-app-rewired start",
    "build": "react-app-rewired build",
    "test": "react-app-rewired test"
  },

But when it builds, it gives me the following stylesheet injected in the index.html as the following which could be the default.

<link href="/static/css/main.dc83752a.css" rel="stylesheet">

I want this to be changed as the following.

<link href="static/css/main.dc83752a.css" rel="stylesheet">

I tried with the config-overrides.js in the root folder as well. But it didn't work. Can anyone help me to sort the issue ? My config-overrides.js as the following.

const ExtractTextPlugin = require("extract-text-webpack-plugin");


// later after upgrade create-react-app to support
// webpack4 it can be removed becuase it now supports
// async css styles through mini-css-extract-plugin
// that future version of create-react-app also supports
// https://github.com/facebook/create-react-app/blob/next/packages/react-scripts/config/webpack.config.prod.js#L423

module.exports = function override(config, env) {

  config.plugins[4] = new ExtractTextPlugin({
    filename: 'static/css/[name].[contenthash:8].css',
    allChunks: true,
  });

  return config;
}