You may split the string with a space, turn all the items not starting with & to upper and just keep the rest as is, and then join back into a string:
string mystring = "Hello World. & my name is < bob >. Thank You.";
string result = string.Join(" ", mystring.Split(' ').Select(m => m.StartsWith("&") ? m : m.ToUpper()));

Another approach is to use a regex to match &, 1+ word chars and then a ;, and match and capture other 1+ word char chunks and only turn to upper case the contents in Group 1:
var result = System.Text.RegularExpressions.Regex.Replace(mystring,
@"&\w+;|(\w+)", m =>
m.Groups[1].Success ? m.Groups[1].Value.ToUpper() :
m.Value
);