1

I was using until now pymongo[srv] to connect my AWS lambda to my MongoDB cluster with:

mongodb+srv://username:[email protected]/database

Now I am trying to setup the IAM role connection and in this repo there is a connection string such as :

"mongodb://<access_key>:<secret_key>@mongodb.example.com/?authMechanism=MONGODB-AWS&authMechanismProperties=AWS_SESSION_TOKEN:<security_token>"

I have tried this one, but I got the following error in my Cloudwatch logs:

[ERROR] ValueError: Port must be an integer between 0 and 65535: 'Fizjqbxairn6K19Fsalbucyz'
Traceback (most recent call last):
  File "/var/lang/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/var/task/lambda_function.py", line 13, in <module>
    client = MongoClient(
  File "/var/task/pymongo/mongo_client.py", line 736, in __init__
    res = uri_parser.parse_uri(
  File "/var/task/pymongo/uri_parser.py", line 568, in parse_uri
    nodes = split_hosts(hosts, default_port=default_port)
  File "/var/task/pymongo/uri_parser.py", line 376, in split_hosts
    nodes.append(parse_host(entity, port))
  File "/var/task/pymongo/uri_parser.py", line 137, in parse_host
    raise ValueError("Port must be an integer between 0 and 65535: %r" % (port,))

So I am guessing that the String syntax is not correct.

Please tell me what is the correct connection string to be used in using the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env variables:

client = MongoClient(conn_string)

Edit

My Python string is:

client = MongoClient('mongodb://' + os.environ.get("AWS_ACCESS_KEY_ID") + ':' + os.environ.get("AWS_SECRET_ACCESS_KEY") + @clustername/databasename?retryWrites=true&authMechanism=MONGODB-AWS)

And I am installing pymongo[aws] as dependency instead of pymongo[srv].

Edit 2

I confirm that the AWS_SECRET_ACCESS_KEY contains / as characters.

Edit 3

The above error is not relevant any more, pymongo accepts the encoded connection string but this is not the good way to go with the AWS Lambda according to this documentation.

I am trying to connect my AWS Lambda to my Atlas cluster database using pymongo[aws] but it times out all the time.

The URI should look like this according to the above documentation : uri = "mongodb://example.com/?authMechanism=MONGODB-AWS"

I have tried it of course, and I have no more errors or anything, the lambda simply times out.

I have double checked the ARN role for my lambda and the one setup in MongoDB when i have created my database user.

I have also granted the dbAdmin role to be sure it's not a permission issue but still times out.

The MongoDB community support also tries to help on this case.

Any ideas where it can come from?

Improve this question
6
  • mongodb://<access_key>:<secret_key>, In place of <secret_key> it is expecting port number Commented Feb 28, 2023 at 13:12
  • have you tried \ (back slash) before : after AWS_SESSION_TOKEN. Like: AWS_SESSION_TOKEN\:<security_token> Commented Feb 28, 2023 at 13:53
  • Same error by escaping the : unfortunately, i have updated my question to add my actual string. Commented Feb 28, 2023 at 14:27
  • I don't really understand where the old question and where it's a new one. I would recommend create a new question.. Commented Apr 26, 2023 at 21:35
  • however, overall there is no difference if you provide aws creds via connection string arguments or via env variables (with omitting them in connection string), the underlying logic will be the same. So make sure: 1. You still can use the creds you set to env variable in connectionString. 2. If so, I'm not 100% confident, but it might be that you need to apply escaping to the env variable values in the same way as in connection string, or if you do it already, then try not to do. If you need authentication from EC2 or EKS ,configuration, then workflow will be different Commented Apr 26, 2023 at 21:42

2 Answers 2

Reset to default
2
+50

There are 3 moving parts that should be configured to work together:

  1. python code/libraries
  2. AWS Lambda role and VPC
  3. Atlas Network and database access

python

starts from pip install "pymongo[aws]"

at the time of answering it pulled following dependencies:

boto3==1.26.121
botocore==1.29.121
dnspython==2.3.0
jmespath==1.0.1
pymongo==4.3.3
pymongo-auth-aws==1.1.0
python-dateutil==2.8.2
s3transfer==0.6.0
six==1.16.0
urllib3==1.26.15

The minimal lambda code to test:

from pymongo.mongo_client import MongoClient
from pymongo.server_api import ServerApi


def lambda_handler(event, context):

    uri = "mongodb+srv://<cluster_name>.<subdomain>.mongodb.net/?authSource=%24external&authMechanism=MONGODB-AWS&retryWrites=true&w=majority"
    # Create a new client and connect to the server
    client = MongoClient(uri, server_api=ServerApi('1'))
    # Send a ping to confirm a successful connection
    try:
        client.admin.command('ping')
        print("Pinged your deployment. You successfully connected to MongoDB!")
    except Exception as e:
        print(e)

The cluster name and domain name in the uri can be found on Atlas connection settings. On the screenshot it's the python driver, but the domain is the same regardless of client:

enter image description here

Note: pymongo-auth-aws plays well with AWS environment variables, so no need to set them explicitly in the uri

AWS Lambda and Mongodb Atlas

Network access

The function should be deployed within VPC to let VPC peering with Atlas, or at least have a fixed egres IP to allow network connections to Atlas from this IP (circled blue on the image below). For the sake of experiment - temporarily allow access to Atlas from everywhere 0.0.0.0/0 to debug authentication in isolation. Once you get it connected on app layer, return back to the network configuration. Don't leave it open even on dev environment

Data base access:

The authentication is based on the role name, so you need to copy exact ARN of the lambda execution role, create a user on Atlas side, paste the role name, and assign mongodb level permissions to the user (red path on the image below).

enter image description here

Testing:

Once it's configured, you can trigger the function manually (payload doesn't matter, it's not being used), and observe Pinged your deployment. You successfully connected to MongoDB! in the log:

enter image description here

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thank you very much @Alex for the detailed steps, it will probably help much more people. In my case, i just forgot the +srv part of the uri that's why i could not connect.
this still times out when using collection.delete_one and only with this method, do you know why ? update_** / find / insert_** are working fine
It really requires debugging. Permissions, profiling, cold/warm run, timings, number of open connections etc. If you post the code I am happy to reproduce. There is nothing special about aws auth in this context. The session authentication is not aware of CRUD commands. The issue must be somewhere else.
You are definitely right and that's why i have asked another question : stackoverflow.com/questions/76131920/… it's directly linked to VPC and internet access but i don't know how / why
1

I think you should escape values in userName/password/sessionToken (AWS_*), search for aws here. In other words, all values you provide in connection string and that may contain special characters like : or / should be escaped. See this SO question about how it can be done in python

Improve this answer

2 Comments

Can you update your answer with an example string please ? Do you mean : mongodb://\os.environ.get("AWS_ACCESS_KEY_ID")\:\os.environ.get("AWS_SECRET_ACCESS_KEY")@cluster...
no, I meant replacing special characters like ':' , / and so on that may be in AWS_* values on escaped one. I'm not a python developer, but this looks like what you need stackoverflow.com/questions/1695183/…

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.