React is just a library, and JSX is a syntax extension to JavaScript. If you setup a boiler plate project using CRA or even without, you can see, react just renders your components into a <div id="root"></div> which exists in an HTML page probably located in the public folder.
ReactDOM.render(
element,
document.getElementById('root')
);
Whether that HTML page is HTML4 or HTML5 is controlled by that HTML page you create or if its CRA, you can modify it to be HTML4.
What happens to the JSX you write in react is:
consider this React element
const element = (
<h1 className="greeting">
Hello, world!
</h1>
)
This is equivalent to writing
const element = React.createElement(
'h1',
{className: 'greeting'},
'Hello, world!'
);
which outputs something like this, which is actually what a react element looks like. This is rendered into the HTML page.
// Note: this structure is simplified
const element = {
type: 'h1',
props: {
className: 'greeting',
children: 'Hello, world!'
}
};