What you expect would not be valid JSON.
Quotes are used to delimit strings in a JSON text.
With your expected result a JSON parser would see "{" and think that was the whole string and then a would be an error.
The escape sequence \" is how you say "This is a quote that is a part of a string" instead of "This is a quote that ends the string".
The output is fine. There is nothing wrong.
That said, nesting JSON is generally a bad idea. It is more complicated to parse and harder to read.
In general you should be creating the complete data structure and then stringifying that.
const a = {};
const str = {
'a': 'test'
};
a.str = str;
const json = JSON.stringify(a, null, 2);
console.log(`result: ${json}`);