When trying to refactor function names, variable names, or class names within all files using F2 (as shown at https://code.visualstudio.com/docs/editor/refactoring#_rename-symbol), I only seem to have success when those other files are open in the editor. If they're closed, VS Code won't rename those instances. Am I missing something?
Before rename with files closed:

After rename with files closed:

Before rename with files open:

After rename with files open:

I tried looking into settings for refactor or replace, but didn't find anything that wasn't enabled that obviously should be. If I successfully rename something when a file is open, undo it, then close the file and re-attempt the refactor, it fails again.
EDIT: I submitted this as a bug to the vscode team on github: https://github.com/microsoft/vscode/issues/146120