Managing Kubernetes cluster from GCP with python api

Question

I would like to be able to access and manage a GKE (kubernetes) cluster from a Google Cloud function written in python. I managed to access and retrieve data from the created cluster (endpoint, username, and password at least), however I dont know how to use them with the kubernetes package api.

Here are my imports :

import google.cloud.container_v1 as container
from google.auth import compute_engine
from google.cloud.container_v1 import ClusterManagerClient
from kubernetes import client, config

Here is the code for cluster data :

project_id = 'my-gcp-project'
zone = 'my-zone'
cluster_id = 'my-existing-cluster'

credentials = compute_engine.Credentials()

gclient: ClusterManagerClient = container.ClusterManagerClient(credentials=credentials)

cluster = gclient.get_cluster(project_id,zone,cluster_id)
cluster_endpoint = cluster.endpoint
print("*** CLUSTER ENDPOINT ***")
print(cluster_endpoint)

cluster_master_auth = cluster.master_auth
print("*** CLUSTER MASTER USERNAME PWD ***")
cluster_username = cluster_master_auth.username
cluster_password = cluster_master_auth.password
print("USERNAME : %s - PASSWORD : %s" % (cluster_username, cluster_password))

I would like to do something like this after that :

config.load_kube_config()
v1 = client.CoreV1Api()
print("Listing pods with their IPs:")
ret = v1.list_pod_for_all_namespaces(watch=False)
for i in ret.items:
    print("%s\t%s\t%s" % (i.status.pod_ip, i.metadata.namespace, i.metadata.name))

However, I can't figure out how to set my endpoint and authentification informations. Can anyone help me please ?

Mariusz · Accepted Answer · 2021-01-19 08:02:53Z

9

You can use a bearer token rather than using basic authentication:

from google.auth import compute_engine
from google.cloud.container_v1 import ClusterManagerClient
from kubernetes import client


def test_gke(request):
    project_id = "my-gcp-project"
    zone = "my-zone"
    cluster_id = "my-existing-cluster"

    credentials = compute_engine.Credentials()

    cluster_manager_client = ClusterManagerClient(credentials=credentials)
    cluster = cluster_manager_client.get_cluster(name=f'projects/{project_id}/locations/{zone}/clusters/{cluster_id}')

    configuration = client.Configuration()
    configuration.host = f"https://{cluster.endpoint}:443"
    configuration.verify_ssl = False
    configuration.api_key = {"authorization": "Bearer " + credentials.token}
    client.Configuration.set_default(configuration)

    v1 = client.CoreV1Api()
    print("Listing pods with their IPs:")
    pods = v1.list_pod_for_all_namespaces(watch=False)
    for i in pods.items:
        print("%s\t%s\t%s" % (i.status.pod_ip, i.metadata.namespace, i.metadata.name))

edited Jan 19, 2021 at 8:02

Mariusz

14k3 gold badges66 silver badges66 bronze badges

answered Aug 29, 2018 at 16:58

Dustin Ingram

21.8k7 gold badges69 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ab. C. Over a year ago

Thank you :) I was completly stuck on this ! When I first printed credentials.token, I found an empty variable. It seems that getting a cluster fills it !

Lucas Over a year ago

Obtaining a cluster didn't obtain a token for me. To populate token, I had to credentials.refresh(google.auth.transport.requests.Request()).

mugdha-adhav · Accepted Answer · 2019-04-16 22:22:18Z

You can use google.oauth2 package for authentication using GCP Service Account.

from google.oauth2 import service_account
from google.cloud.container_v1 import ClusterManagerClient
from kubernetes import client, config
import os

def test_gke(project_id, zone, cluster_id):
    SCOPES = ['https://www.googleapis.com/auth/cloud-platform']
    credentials = service_account.Credentials.from_service_account_file(os.getenv('GOOGLE_APPLICATION_CREDENTIALS'), scopes=SCOPES)
    cluster_manager_client = ClusterManagerClient(credentials=credentials)
    cluster = cluster_manager_client.get_cluster(project_id, zone, cluster_id)
    configuration = client.Configuration()
    configuration.host = "https://"+cluster.endpoint+":443"
    configuration.verify_ssl = False
    configuration.api_key = {"authorization": "Bearer " + credentials.token}
    client.Configuration.set_default(configuration)

    v1 = client.CoreV1Api()
    print("Listing pods with their IPs:")
    pods = v1.list_pod_for_all_namespaces(watch=False)
    for i in pods.items:
        print("%s\t%s\t%s" % (i.status.pod_ip, i.metadata.namespace, i.metadata.name))

Refer the link below to know more about GCP Authorized API calls https://developers.google.com/identity/protocols/OAuth2ServiceAccount

Rob Audenaerde · Accepted Answer · 2021-03-31 10:39:02Z

Here's an example using a GCP service account to generate a bearer token.

Note that you should make sure you enable SSL verification when you connect to your cluster, otherwise you're vulnerable to man in the middle attacks. GKE does this based on its own certificate for the cluster that you need to configure manually.

import base64
import google.auth.transport.requests
from google.oauth2 import service_account
from google.cloud.container_v1 import ClusterManagerClient
from kubernetes import client
from python_hosts.hosts import Hosts, HostsEntry


def test_gke(request):
    project_id = "my-gcp-project"
    zone = "my-zone"
    cluster_id = "my-existing-cluster"

    # Use a service account configured in GCP console,
    # authenticating with a JSON key
    credentials = service_account.Credentials \
        .from_service_account_file('gcloud_key.json')

    # Get cluster details
    cluster_manager_client = ClusterManagerClient(credentials=credentials)
    cluster = cluster_manager_client.get_cluster(
            project_id=project_id, zone=zone,
            cluster_id=cluster_id)

    # Save cluster certificate for SSL verification
    cert = base64.b64decode(cluster.master_auth.cluster_ca_certificate)
    cert_filename = 'cluster_ca_cert'
    cert_file = open(cert_filename, 'wb')
    cert_file.write(cert)
    cert_file.close()

    # Configure hostname for SSL verification
    hosts = Hosts()
    hosts.add([HostsEntry(
            entry_type='ipv4',
            address=cluster.endpoint, names=['kubernetes'])])
    hosts.write()

    # Get a token with the scopes required by GKE
    kubeconfig_creds = credentials.with_scopes(
            ['https://www.googleapis.com/auth/cloud-platform',
             'https://www.googleapis.com/auth/userinfo.email'])
    auth_req = google.auth.transport.requests.Request()
    kubeconfig_creds.refresh(auth_req)

    configuration = client.Configuration()
    configuration.host = "https://kubernetes"
    configuration.ssl_ca_cert = cert_filename
    kubeconfig_creds.apply(configuration.api_key)
    client.Configuration.set_default(configuration)

    v1 = client.CoreV1Api()
    print("Listing pods with their IPs:")
    pods = v1.list_pod_for_all_namespaces(watch=False)
    for i in pods.items:
        print("%s\t%s\t%s" % (i.status.pod_ip, i.metadata.namespace, i.metadata.name))

Here's a list of the Python libraries for this as well (their pip project names):

kubernetes
google-api-python-client
google-cloud-container
python-hosts

Collectives™ on Stack Overflow

Managing Kubernetes cluster from GCP with python api

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related