1

I am working on a python project that involves securely retrieving and storing information in internet-accessible locations such as databases and storage buckets. Depending on various factors, I may run code either on a local machine or on hundreds of virtual machines at Amazon AWS or Google Compute Engine.

I use a Github repo to maintain version control and to facilitate access to the code by these virtual machines. I would prefer not to store credentials such as passwords in a flat file in the repository even though the repository is private. However, I don't know how else to automatically provide the necessary credentials to all of the various machines that need them. Storing the credentials in the virtual machine images is suboptimal because I do not want to create a new image whenever a credential changes.

Is there a lightweight solution for distributing credentials of this type to disparate computing environments?

Improve this question
3
  • Recommendation requests are off topic Commented Jan 31, 2015 at 0:28
  • couldnt you just add the certificates to the .ssh folder on the host? or encrypt your credentials and distribute your password directly to the people that you are distributing the package to .... you could even write aa python script to decrypt the file and put it somewhere accessible to the application Commented Jan 31, 2015 at 0:42
  • I don't see why this question is off topic. I am happy to implement the solution myself if nothing is available, but I'm not sure what the best solution is. As I understand it, key management is something of an open problem in cloud computing. Commented Feb 1, 2015 at 2:06

2 Answers 2

Reset to default
1

Hmm - I'm new, but is this request really off topic? It seems ok to me...

If you use Python and GIT, then we coded up a solution for storing passwords (and other config info) as regular variables, using RC4. It made it easy for us to maintain passwords in plaintext on dev machines, have the passwords encrypted automatically, and an encrypted version distributed automatically with every git push. You need to manually install a private key on each production machine, but thereafter git pulls on the production machine will pull in an updated encrypted file and automatically decrypt it at runtime for use within your code.

One nice feature of the code is that a decrypted (plaintext) copy of your passwords never, ever hits the hard drive on a production machine. And yet you can directly reference your passwords within your python code, and most intellisense systems (such as pyCharm's) can view the password variables during coding on the production machine. It is also very easy to change passwords - just update the plaintext file and git push.

I haven't contributed anything to SO yet, so I might as well start now. I put the code, along with a complete ReadMe, on a GitHub repo:

https://github.com/KenYounge/security

After grabbing the security.py file, implementation is as simple as:

import security
for line in security.secure(): exec line in globals()

There is a complete README.md file and helloworld.py example in the github repo. Here is the implementation code in case someone wants to comment:

"""Secure python variables: encrypt, decrypt, import into global namespace."""
__module__    = 'security.py'
__author__    = "Kenneth A Younge"
__copyright__ = "Copyright (c) 2014, Kenneth A. Younge"
__license__   = "GNU General Public License"
__email__     = "[email protected]"

import os

def crypt(data, key):
    x = 0
    box = range(256)
    for i in range(256):
        x = (x + box[i] + ord(key[i % len(key)])) % 256
        box[i], box[x] = box[x], box[i]
    x = 0
    y = 0
    out = []
    for char in data:
        x = (x + 1) % 256
        y = (y + box[x]) % 256
        box[x], box[y] = box[y], box[x]
        out.append(chr(ord(char) ^ box[(box[x] + box[y]) % 256]))
    return ''.join(out)

def secure(file_names=('passwords.py',),
           key_name='security.key',
           key_path='~/',
           pvt_path='_private/',
           verbose=False):
    """Transform files (encrypt and/or decrypt _private files).

    Keyword arguments:
        filenames  --  sequence of file names to encrypt/decrypt
        key_name   --  file name of your personal rc4 encryption key
        key_path   --  location of encryption key on production machines
        pvt_path   --  location of private files and encryption key during dev
        verbose    --   print info

    Defaults:
        filenames  --  passwords.py    currently a tuple with one file
        key_name   --  security.key     
        key_path   --  ~/               
        pvt_path   --  _private/
        verbose    --  False
    """

    # Load key (try production location first)
    if os.path.exists(os.path.join(key_path, key_name)):
        key = open(os.path.join(key_path, key_name), 'r').read()
    elif os.path.exists(os.path.join(
            os.path.dirname(__file__), pvt_path + key_name)):
        key = open(os.path.join(
            os.path.dirname(__file__), pvt_path + key_name), 'r').read()
    else:
        key = open(os.path.join(
            os.path.dirname(__file__), key_name), 'r').read()

    # secure each file
    code_lines = []
    for filename in file_names:

        filename_raw = os.path.join(
            os.path.dirname(__file__), pvt_path + filename)
        filename_rc4 = os.path.join(
            os.path.dirname(__file__),
            os.path.basename(filename).replace('.py', '.rc4'))

        # Encrypt
        try:
            if os.path.exists(filename_raw):
                with open(filename_raw, 'r') as f:
                    text = f.read()
                with open(filename_rc4, 'w') as f:
                    f.write(crypt(str(text).strip(), key).encode('hex'))
                if verbose:
                    print 'Encrypted ' + filename_raw
            else:
                if verbose:
                    print('File (' + filename_raw + ') not found')
        except Exception as e:
            print(str(e))

        # Decrypt
        try:
            if os.path.exists(filename_rc4):
                with open(filename_rc4, 'r') as f:
                    text = crypt(str(f.read()).strip().decode('hex'), key)
                    lines = [str(line).strip() for line in text.splitlines()]
                if lines: code_lines.extend(lines)
                if verbose:
                    print 'Encrypted ' + filename_rc4
            else:
                print('File ' + filename_rc4 + ' not found')
        except Exception as e:
            print(str(e))
    return code_lines
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

This approach seems perfect. It looks lightweight but effective, since I do pull the code for each VM off of Git during startup. Also, I can change the keys without manual reconfiguration or moving files around, which is another big concern.
This sounds interesting.
1

I've used symmetric ciphers with long passwords for this sort of thing when a human accessing any of the collection of machines needs access to facilities on S3. Fully automated, I doubt any of this is very safe. It is more for human intervention.

There is a plaintext file .s3cfg containing Amazon S3 credentials needed by the tool s3cmd. We want to back it up to ciphertext file .gpgs3 and have it available in various places, along with some scripts to make it easier for non-gpg users to convert it to the needed .s3cfg file if they know the password.

I put the gpg commands for the encryption in some short scripts.

Encryption script s3lock.sh

 #!/bin/bash
 umask 077
 gpg --symmetric <~/.s3cfg >~/.gpgs3

Decryption script s3on.sh

 #!/bin/bash
 umask 077
 gpg --decrypt ~/.gpgs3 >~/.s3cfg 2>/dev/null

Tidy-up script s3off.sh

 #!/bin/bash
 # remove the plain text file
 rm -f ~/.s3cfg

This methodology hasn't been subjected to any peer review, but I've yet to have a problem with it.

Some things that could happen with this setup:

  • you forget to run s3off.sh when you are done, leaving the plaintext file available and later when someone cracks your box they get the S3 credentials. They run up a huge bill, Amazon locks down your stuff, and the collections department calls daily.
  • you put s3off.sh in a script, but an impatient person interrupts the script, and later when someone cracks your box they get the S3 credentials... and delete all those backups that are in S3 for "safekeeping".
  • a disgruntled coworker notices how this works and takes the plaintext S3 credentials during an otherwise legitimate access...
Improve this answer

4 Comments

i dont see why he couldnt just gpg his whole credential file and put it in the repo ... and distribute a decryption key directly to the team
That is indeed the basic idea. I did this because in a rush I didn't care to be looking at a gpg cheat sheet. Just ask me the password, please...
This seems like a reasonable approach, but it isn't as automated as I'd like. I'd also prefer to not have the plaintext file available if I forget to deal with it.
@RobertNeely You could write a cron job that deletes the plain text file in case you forget.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.