1

My aim is to launch an instance such that a start-up script is triggered on boot-up to download some configuration files stored in AWS S3. Therefore, in the start-up script, I am setting the S3 bucket details and then, triggering a config.sh where aws s3 sync does the actual download. However, the aws command does not work - it is not found for execution.

User data

I have the following user data when launching an EC2 instance:

#!/bin/bash
# Set command from https://stackoverflow.com/a/34206311/919480
set -e -x
export S3_PREFIX='bucket-name/folder-name'
/home/ubuntu/app/sh/config.sh

The AWS CLI was installed with pip as described in the documentation.

Observation

I think, the user data script is run with root user ID. That is why, in the user data I have /home/ubuntu/ because $HOME did not resolve into /home/ubuntu/. In fact, the first command in config.sh is mkdir /home/ubuntu/csv which creates a directory with owner as root!

So, would it be right to conclude that, the user data runs under root user ID?

Resolution

Should I use REST API to download?

Improve this question
2
  • What do you mean by "The AWS CLI was installed with pip"? It is not in your User Data script. Was it installed, then you created an AMI, then you used that AMI with this new instance? Or is it inside the config.sh file? You can always try sudo su - to become root and then paste your User Data to see if it runs. Commented Jun 1, 2018 at 4:39
  • @JohnRotenstein AWS CLI was installed on the AMI. Commented Jun 1, 2018 at 4:42

2 Answers 2

Reset to default
4

Scripts entered as user data are executed as the root user, so do not use the sudo command in the script.

See: Running Commands on Your Linux Instance at Launch

One solution is to set the PATH env variable to include AWS CLI (and add any other required path) before executing AWS CLI.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

sudo is not used any where. Will try PATH option.
Or, just refer to aws via the full path, eg /usr/local/bin/aws s3 cp...
@JohnRotenstein for me which aws shows /home/ubuntu/.local/bin/aws - probably because I didn't do a sudo pip install. Therefore, I tried with su ubuntu - right at the start of the script and now I see this, __init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: 'b'su ubuntu -'...'
Also, providing the full path /home/ubuntu/.local/bin/aws does not help because import of AWS CLI driver fails for root. Hence, the choice of using su ubuntu -.
1

Solution

Given that, AWS CLI was installed without a sudo pip, the CLI is not available for root. Therefore, to run with ubuntu user, I used the following user data script:

#!/bin/bash
su ubuntu -c '$HOME/app/sh/config.sh default`

In config.sh, the argument default is used to build the full S3 URI before invoking the CLI. However, the invocation was successful only with the full path $HOME/.local/bin/aws despite the fact that aws can be accessed with normal login.

Improve this answer

1 Comment

Having the same problem -- aws s3 not found in the user data environment. But I've come across this passage: " If you use an AWS API, including the AWS CLI, in a user data script, you must use an instance profile when launching the instance. An instance profile provides the appropriate AWS credentials required by the user data script to issue the API call." Could that be related?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.