google auth2 with multi-user failed in refreshing a token, 401 Unauthorized

I've application connected to google calendar with multiple keys or users in accessing the oauth2 token. All users can register(using Servlet) and send the calendar event in around one hour after registered. After 1 hour, only one(the same person) of them can get the refresh/update token once it expired and can send calendar event. The rest users can't get the refresh token or can't send calendar event out; error log shows as the following:

    com.google.api.client.auth.oauth2.TokenResponseException: 401 Unauthorized
POST https://oauth2.googleapis.com/token
        at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:103)
        at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:308)
        at com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeTokenRequest.execute(GoogleAuthorizationCodeTokenRequest.java:166)
        at com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeTokenRequest.execute(GoogleAuthorizationCodeTokenRequest.java:72)
        at com.google.api.client.extensions.servlet.auth.oauth2.jakarta.AbstractAuthorizationCodeCallbackServlet.doGet(AbstractAuthorizationCodeCallbackServlet.java:122)...

My codes to validate & refresh token are:

String keys = googleCal.getCredential();//get credential from datastore based on a user key
 InputStream stream = new ByteArrayInputStream(keys.getBytes(Charset.forName("UTF-8")));
 GoogleClientSecrets clientSecrets = GoogleClientSecrets.load(JSON_FACTORY, new InputStreamReader(stream));
 DataStoreFactory dataStore = new JPADataStoreFactory(repository);
 NetHttpTransport HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
 GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow.Builder(
  HTTP_TRANSPORT, JSON_FACTORY, clientSecrets, SCOPES)
  .setDataStoreFactory(dataStore) 
  .setApprovalPrompt("force")
  .setAccessType("offline")
  .build();
                

Note: users are less than 10 and my app was not sent to verify yet.

Any suggestion would be appreciated.