Gcloud App Engine "500 Internal Server Error" When Using Domain Mappings API to Map Domains and Auto Provision SSL Certs

I'm facing an issue where I'm unable to create domain mappings for my App Engine Project via the API. Whenever I attempt to map a custom domain and automatically provision an SSL certificate, I encounter a 500 error with the message "Internal error encountered."

I'm using Python Flask Application and the googleapiclient library to interact with the App Engine API. The rest of my application is still working fine.

Here's a snippet of my code:

def create_domain_mapping(domain_name):
    try:
        credentials = get_credentials()
        if not credentials:
            logging.error("Failed to retrieve credentials.")
            return None

        app_id = os.getenv('app_id')
        if not app_id:
            logging.error("App ID is not set.")
            return None

        service = build('appengine', 'v1', credentials=credentials)
        app_name = f"apps/{app_id}"
        domain_body = {
            "name": "apps/{app_id}/domainMapping/{domain_name}",
            "id": domain_name
            # "sslSettings": {
            #     "sslManagementType": "AUTOMATIC"
            #}
        }
   
        response = service.apps().domainMappings().create(appsId=app_name, body=domain_body).execute()
        logging.info(f"SUCCESS - Domain mapping response : {response}")
        return response
    
    except Exception as e:
        logging.error(f"Failed to provision SSL : {e}")
        return None

Error Logs Image

For this issue, I've already taken the following steps to troubleshoot:

• Verified service account permissions: Ensured the service account has the necessary App Engine Admin and Project Viewer roles.

• Checked App Engine logs: There are no other related error messages.

• Tested network connectivity: Confirmed network connectivity to Google Cloud services.

• Checked quotas: Verified that I haven't exceeded any App Engine quotas.

• Rewritten API call: Tried different variations of the API call to create domain mappings. In the code below, I even commented out the SSL provisioning just to test the domain mapping.

• DNS Settings: I've made sure that the domain I'm trying to map also has the right A/CNAME records in DNS so it points to my application

• Manual Mapping in GCloud: I've added the domain manually to see if there was an issue and it worked successfully.

• Other error: When the SSL data is included in the API call, I also get the error: "INFO:Compute Engine Metadata server call to universe/universe_domain returned 404, reason: 404 page not found". I don't have this path explicitly called anywhere in my application.

Updated Snippet (still not working):

def create_domain_mapping(domain_name):
    try:
        credentials = get_credentials()
        if not credentials:
            logging.error("Failed to retrieve credentials")
            return None

        app_id = os.getenv('app_id')
        if not app_id:
            logging.error("App ID is not set")
            return None
        
        client = appengine_admin.AppEngineAdminClient(credentials=credentials)
        domain_mapping = {
            "domainName": domain_name,
            "sslSettings": {
                "sslManagementType": "AUTOMATIC"
            }
        }

        parent = f"projects/{app_id}/apps/{app_id}"
        response = client.create_domain_mapping(
            parent=parent,
            domain_mapping=domain_mapping
        )
  
        print(f"Domain mapping created: {response.name}")
        return response
    
    except Exception as e:
        logging.error(f"Failed to create domain mapping: {e}")
        return None