6

When I pass the options in the program (a computational biology experiment) I usually pass them through a .py file.
So I have this .py file that reads like:

starting_length=9
starting_cell_size=1000
LengthofExperiments=5000000

Then I execute the file and get the data. Since the program is all on my machine and no one else has access to it, it is secure in a trivial way.
I can also write a similar file very easily:

def writeoptions(directory):
    options=""
    options+="starting_length=%s%s"%(starting_length,os.linesep)
    options+="starting_cell_size=%s%s"%(starting_cell_size,os.linesep)
    options+="LengthofExperiments=%s%s"%(LengthofExperiments,os.linesep)
...
    open("%s%soptions.py"%(directory,os.sep),'w').write(options)

I want to pass a function as one of the parameters:

starting_length=9
starting_cell_size=1000
LengthofExperiments=5000000

def pippo(a,b):
    return a+b
functionoperator=pippo

And of course in the real experiment the function pippo will be much more complex. And different from experiment to experiment.

But what I am unable to do is to write the function automatically. In short I don't know how to generalise the writeoptions function to keep on writing the options, if one of the options is a function. I could of course copy the original file, but this is inelegant, inefficient (because it contains a lot of extra options that are not being used), and generally does not solve the question.

How do you get python to write down the code of a function, as it writes down the value of a variable?

Improve this question
3
  • If the options are ALREADY in Python, why write them to another file? If the options are not in Python, then I'm not clear on what you can't do. It would help if you could include pseudo-code for the thing you can't do. Commented Dec 30, 2008 at 11:22
  • See stackoverflow.com/questions/48211/… Commented Dec 30, 2008 at 16:14
  • See stackoverflow.com/questions/334851/… Commented Dec 30, 2008 at 16:16

5 Answers 5

Reset to default
15 
vinko@mithril$ more a.py

def foo(a):
  print a

vinko@mithril$ more b.py

import a
import inspect

a.foo(89)
print inspect.getsource(a.foo)

vinko@mithril$ python b.py
89
def foo(a):
  print a
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks, but this solution would require that I have an extra file for each function that I pass in the parameter. Plus this has essentially the problems I explained about simply copying the original file. Let's see if someone can find a different solution, thanks though. Pietro
Huh? Why would it require an extra file per each function?
Note that this (usually) doesn't work for retrieving source code if you deleted or edited the source file. inspect.getsource has to read the source file - Python functions don't save their own source code in memory. There's some caching involved, but unless the file contents happen to be cached in linecache, this will only work if the source file still exists.
2

You might also consider some other means of data persistence. In my own (astronomy) research, I've been experimenting with two different means of storing scripts for reproducibility. The first is to have them exclusively inside a subversion repository, and then have the job submission script automatically commit them. For instance, if you just wanted to do this in bash:

alias run_py='svn ci -m "Commit before running"; python2.5 $*'

and inside the script, have the output prefixed by the current subversion revision number for that file, you'd have a record of each script that was run and what the input was. You could pull this back out of subversion as need be.

Another, substantially less full-featured, means of tracking the input to a function could be via something like LodgeIt, a pastebin that accepts XML-RPC input and comes with Python bindings. (It can be installed locally, and has support for replying to and updating existing pastes.)

But, if you are looking for a relatively small amount of code to be included, Vinko's solution using inspect should work quite well. Doug Hellman covered the inspect module in his Python Module of the Week series. You could create a decorator that examines each option and argument and then prints it out as appropriate (I'll use inspect.getargspec to get the names of the arguments.)

import inspect
from functools import wraps

def option_printer(func):
    @wraps(func)
    def run_func(*args, **kwargs):
        for name, arg in zip(inspect.getargspec(func)[0], args) \
                       + sorted(kwargs.items()):
            if isinstance(arg, types.FunctionType): 
                print "Function argument '%s' named '%s':\n" % (name, func.func_name)
                print inspect.getsource(func)
            else:
                print "%s: %s" % (name, arg)
        return func(*args, **kwargs)
    return run_func

This could probably be made a bit more elegant, but in my tests it works for simple sets of arguments and variables. Additionally, it might have some trouble with lambdas.

Improve this answer

Comments

0

Are you asking about this?

def writeoptions(directory):
    options=""
    options+="starting_length=%s%s"%(starting_length,os.linesep)
    options+="starting_cell_size=%s%s"%(starting_cell_size,os.linesep)
    options+="LengthofExperiments=%s%s"%(LengthofExperiments,os.linesep)
    options+="def pippo(a,b):%s" % ( os.linesep, )
    options+="    '''Some version of pippo'''%s" % ( os.linesep, )
    options+="    return 2*a+b%s" % ( os.linesep, )
    open("%s%soptions.py"%(directory,os.sep),'w').write(options)

Or something else?

Improve this answer

8 Comments

essentially something else. Your code would not take the function pippo as a parameter from which to write the new options file. Essentially I could as well have the function Pippo in the code itself, since I would have to change the function writeoptions in each case. Is it more clear now?
Not clear at all. Do you have the source for pippo? If so, then you don't need to write it to a file -- you already have the source.
Every experiment is placed in a different directory, and I copy there all and only the information I need. A copy of the program, of the data, and of the options(!). But the options file does not get copied but produced to avoid unnecessary parts. Thus the question. Beside the theorethical interest.
"copy of the program, of the data, and of the options(!). But the options file does not get copied" Okay. Whatever you say. Options both copied and not copied.
LOL; let me try again. The word 'copy' has two separate meaning. As a verb it means that we make a take one object and produce another like in "cp a.by b.py". But as an adverb (THIS is a copy of THAT) can both means that it was produced through a copy or that it is identical. So the confusion.
|
0

While it is possible to do what you ask (as Vinko has shown), I'd say it is cleaner to share code. Put pippo and his buddies in a submodule that both programs can access.

Improve this answer

4 Comments

Thanks, but as I explained I am not very satisfied with Vinko solution, for reasons I have explained above. Said that, the whole structure of the code is to have a main program, a module file for each type of system, and an option file for each option. (continues)
I am not really looking foreward to have an extra file for each module, for each functions I want to pass in the options.
In my programs I tend to save the code that is acting every time, for reasons too long to explain here. So they don't share modules around the computer, and I can pass an experiemnt to a collegue easily.
But on the side of all this I find the challenge to be able to save a function, as you save a parameter, to be interesting in its own right.
0

Instead of diving into the subject of disassemblers and bytecodes (e.g inspect), why don't you just save the generated Python source in a module (file.py), and later, import it?

I would suggest looking into a more standard way of handling what you call options. For example, you can use the JSON module and save or restore your data. Or look into the marshal and pickle modules.

Improve this answer

2 Comments

Thanks. While it is possible that my self-funded way to store the options is sub-optimal, the main reason why I want to learn to write the source code of a function is for its own sake. As a general investigation in the concept of programs that write their own code.
Thanks for the links, though. I have read the inspect one, and found part of the solution I was looking for. As such I am on my way. :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.