Regex quantity specifiers + and * are greedy, you can add a ? onto the end of them (+? and *?) to turn them into their non greedy forms.
Greedy means that the operator will try consume everything it can before checking the next token.
so for the case of
\S+\/?
The S will try consume everything it can before the / is checked, and as the / is optional nothing needs to be done for it.
Once we make it into its non greedy form
\S+?\/?
The S will consume as little as possible before trying the / meaning that the / gets 'first dibs' on any tokens, and once it fails to grab them these tokens will be tried against the \S+?
I found success using the following:
regex = re.compile(r'^(\[)(\S+?)(\/?)(\][ST]$)')
For more information you can see the python re docs search for greedy.
As a side note if you pass the re.VERBOSE flag into re.compile then it will ignore whitespace within your string meaning you can structure it as
regex = re.compile(r'^ (\[) (\S+?) (\/?) (\][ST]$) ', re.VERBOSE)
which I found quite helpful when learning regex.
Also you have the start of string token outside a group '^ ([' but the end of string token within a group '(][ST]$)', this shouldn't make a difference except to readability.