I think I figure out the solution myself, I want to post the solution for windows local machine here; thanks for @Wayne 's suggestion that "It's just that making it effectively work can be super tricky depending on your system".
I open the powershell of windows, type the following command
[System.IO.File]::WriteAllBytes("$env:TEMP\ctrl-d.txt", @(4))
Then I open the file using command:(Open a folder,type the following command in the address field)
%TEMP%\ctrl-d.txt
then ctrl-A + ctrl-C to copy the character to clipboard in windows system
paste that character into the interactive mode prompt
I am able to get back to ipdb normal mode instead of interactive mode.
You can see the result in the picture:
By the way, I really think they should give an easy way to quit the mode, it cost me several days to figure this out...
