Issue with using toLocaleString in React input with decimals

I have a react input box that I want to format so that it will show a number displayed as a currency. It is currently showing commas but when I try and add a . (decimal) it doesn't work. It should only show up to 2 decimal places.

With the current setup I can enter 2000000 and it will get formatted as 2,000,000 which is correct. The issue is if I want to also add cents I am unable to do so. So to do 2000.50 is impossible. This should be 2,000.50

There is a codesandbox here for reference. It may be clearer if you try and enter an amount with decimals there.

Can anyone help here?

My code looks like this:

function Input() {
  const [value, setValue] = React.useState("");

  const formatValue = value => {
    return setValue(value.replace(/,/g, ""));
  };

  return (
    <input
      type="text"
      value={Number(value).toLocaleString()}
      onChange={e => formatValue(e.target.value)}
    />
  );
}