A date has no time zone
I would like to set the timezone without modifying the date.
A date-only value such as November 15 of 2020 has no notion of time zone. You have no need to worry about time zones.
Avoid legacy date-time classes
You are using terrible date-time classes that were supplanted years ago by the java.time classes defined in JSR 310. Never use SimpleDateFormat, Date, or Calendar.

LocalDate
In Java, a date-only value without time-of-day and without time-of-day is represented by the LocalDate class.
String input = "11/15/2020" ;
DateTimeFormatter f = DateTimeFormatter.ofPattern( "MM/dd/uuuu" ) ;
LocalDate ld = LocalDate.parse( input , f ) ;
See this code run live at IdeOne.com.
ld.toString: 2020-11-15
Moment
Perhaps you do want a moment rather than just a date. I assume that would mean you want the first moment of the day.
Do not assume the day starts at 00:00:00. Anomalies such as Daylight Saving Time (DST) means the day may start at a time such as 01:00:00. Let java.time determine what a particular day in a particular time zone begins.
Specify a proper time zone name in the format of Continent/Region, such as America/Montreal, Africa/Casablanca, or Pacific/Auckland. Never use the 2-4 letter abbreviation such as EST, EDT, CST, or IST as they are not true time zones, not standardized, and not even unique(!). For example, CST could be China Standard Time just as easily as Central Standard Time in the Americas.
ZoneId z = ZoneId.of( "America/Montreal" ) ;
ZonedDateTime zdt = ld.atStartOfDay( z ) ;
zdt.toString(): 2020-11-15T00:00-05:00[America/Montreal]
Generally best to store and exchange moments in UTC. To adjust from our specified time zone to UTC, extract a Instant from the ZonedDateTime. An Instant is always in UTC by definition.
Instant instant = zdt.toInstant() ;
instant.toString(): 2020-11-15T05:00:00Z
Generate text in standard ISO 8601 format. The Z on the end means UTC, and is pronounced “Zulu”.
instant.toString(): 2020-11-15T05:00:00Z
Convert
If you must interoperate with old code not yet updated to java.time, you can convert back-and-forth. Look to new to…/from… methods added to the old classes.
The legacy class Date, though misnamed, is equivalent to Instant, both representing a moment in UTC. Instant uses a finer resolution of nanoseconds rather than milliseconds.
Date d = java.util.Date.from( instant ) ;
…and…
Instant instant = d.toInstant() ;
Be aware that the Date::toString method tells a lie, applying the current default time zone. While well-intentioned, in practice this is quite confusing. One of many reasons to avoid this class.
Dateclass. It plays such tricks on you. TheTimeZoneclass has design flaws too, andSimpleDateFormatis notoriously horrible. Do yourself a great favour, don’t use those classes. Use java.time, the modern Java date and time API. For a date without time zone (so it stays the same date in all time zones) you need theLocalDateclass.