how to call python function by getting mongob collection values

Question

how to create document and collection in mongodb to make python code configuration. Get attribute name, datatype, function to be called from mongodb ?

mongodb collection sample example

db.attributes.insertMany([
   { attributes_names: "email", attributes_datype: "string", attributes_isNull="false", attributes_std_function = "email_valid" }
   { attributes_names: "address", attributes_datype: "string", attributes_isNull="false", attributes_std_function = "address_valid" }

]);

Python script and function

def email_valid(df):

    df1 = df.withColumn(df.columns[0], regexp_replace(lower(df.columns[0]), "^a-zA-Z0-9@\._\-| ", ""))
    extract_expr = expr(
        "regexp_extract_all(emails, '(\\\w+([\\\.-]?\\\w+)*@\\[A-Za-z\-\.]+([\\\.-]?\\\w+)*(\\\.\\\w{2,3})+)', 0)")
    df2 = df1.withColumn(df.columns[0], extract_expr) \
        .select(df.columns[0])

    return df2

How to get all the mongodb values in python script and call the function according to attribues.

Lakshika Parihar · Accepted Answer · 2021-08-16 06:53:36Z

1

To create MongoDB collection from a python script :

import pymongo
# connect to your mongodb client
client = pymongo.MongoClient(connection_url)

# connect to the database
db = client[database_name]

# get the collection
mycol = db[collection_name]

from bson import ObjectId
from random_object_id import generate

# create a sample dictionary for the collection data
mydict = { "_id": ObjectId(generate()),
           "attributes_names": "email", 
           "attributes_datype": "string", 
           "attributes_isNull":"false", 
           "attributes_std_function" : "email_valid" }

# insert the dictionary into the collection
mycol.insert_one(mydict)

To insert multiple values in the MongoDB, use insert_many() instead of insert_one() and pass the list of dictionary to it. So your list of dictionary will look like this

mydict = [{ "_id": ObjectId(generate()),
           "attributes_names": "email", 
           "attributes_datype": "string", 
           "attributes_isNull":"false", 
           "attributes_std_function" : "email_valid" },
           { "_id": ObjectId(generate()),
           "attributes_names": "email", 
           "attributes_datype": "string", 
           "attributes_isNull":"false", 
           "attributes_std_function" : "email_valid" }]

To get all the data from MongoDB collection into python script :

data = list()
for x in mycol.find():
  data.append(x)

import pandas as pd
data = pd.json_normalize(data)

And then access the data as you access an element of a list of dictionaries:

value = data[0]["attributes_names"]

edited Aug 16, 2021 at 6:53

answered Aug 16, 2021 at 6:35

Lakshika Parihar

1,18311 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Naveen Over a year ago

how to do for multiple attributes.

Lakshika Parihar Over a year ago

create a list of all your data and Use insert_many() instead of insert_one(), I have updated the answer as well

Naveen Over a year ago

I am not able to write id part in direct mongodb.

Lakshika Parihar Over a year ago

Yup, I think MongoDB generates its own id when you are working directly on MongoDB without python. And if it doesn't generate that then maybe you can remove it from python code as well

Naveen Over a year ago

Yes you are right. I have removed it and working fine.

Collectives™ on Stack Overflow

how to call python function by getting mongob collection values

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related