There is a little complex workaround.
You can direct connect to Azure AD with an access token:
Connect-AzureAD
[-AzureEnvironmentName <EnvironmentName>]
[-TenantId <String>]
-AadAccessToken <String>
[-MsAccessToken <String>]
-AccountId <String>
[-LogLevel <LogLevel>]
[-LogFilePath <String>]
[-InformationAction <ActionPreference>]
[-InformationVariable <String>]
[-WhatIf]
[-Confirm]
[<CommonParameters>]
And you can get an access token with refresh token without a prompt.
To simply get a refresh token, a easy way is to use Fiddler. Open Fiddler, and run Connect-AzureAD. you will be able to find the refresh token:

Then you can get a new access token and use it to connect to AAD as following:
# The refresh token
$refresh_token="AQABAAAAAACQN9QBRU3jT6bcBQLZNUj7NLUSh_LtiE0dRWb-Vqb9RjUoNjK67G0DlSF65M_w6o1fAvQ******16Z4J0X-MEZSAA"
# Tenant id and account id
$tenant_id = "hanxia.onmicrosoft.com"
$account = "[email protected]"
# 1b730954-1685-4b74-9bfd-dac224a7b894 is a public client from Microsoft
$clientId = "1b730954-1685-4b74-9bfd-dac224a7b894"
$uri = "https://login.microsoftonline.com/${tenant_id}/oauth2/token"
$body = @{grant_type='refresh_token';resource='https://graph.windows.net';client_id=$clientId;refresh_token=$refresh_token}
$result = Invoke-RestMethod -Method Post -Uri $uri -Body $body
$accessToken = $result.access_token
# Connect to AAD
Connect-AzureAD -TenantId $tenant_id -AadAccessToken $accessToken -AccountId $account
Result

Note
The refresh contains privacy information. You need to keep it safe.