How to use metaprogramming with function args?

Question

its my second Day learning and experiment with Julia. Although I read the Documantation concerning Metaprogramming carefully (but maybe not carefully enough) and several simular threads. I still can't figure out how I can use it inside a function. I tryed to make following function for simulation of some data more flexible:

using Distributions
function gendata(N,NLATENT,NITEMS)
  latent = repeat(rand(Normal(6,2),N,NLATENT), inner=(1,NITEMS))
  errors = rand(Normal(0,1),N,NLATENT*NITEMS)
  x = latent+errors
end

By doing this:

using Distributions
function gendata(N,NLATENT,NITEMS,LATENT_DIST="Normal(0,1)",ERRORS_DIST="Normal(0,1)")
  to_eval_latent = parse("latent = repeat(rand($LATENT_DIST,N,NLATENT), inner=(1,NITEMS))")
  eval(to_eval_latent)
  to_eval_errors = parse("error = rand($ERRORS_DIST,N,NLATENT*NITEMS)")
  eval(to_eval_errors)
  x = latent+errors
end

But since eval don't work on the local scope it dont work. What can I do to work arround this?

Also the originally function, don't seem to be that fast, did I make any major mistakes concerning perfomance?

I really appriciate any recommandation. Thanks in advance.

What is wrong with just passing the distributions as arguments? Using eval seems overly complicated. — phipsgabler
– phipsgabler, Commented Mar 9, 2017 at 10:40
@phg It does not seem overly complicated, it is. The code initially came from an R Script, where this is more or less necessary. Do you have any Idea why repeat is so slow? Its a BLAS so it should be fast as lightning, it the slowest part of the function. — AaronP
– AaronP, Commented Mar 9, 2017 at 13:10

Isaiah Norton · Accepted Answer · 2017-03-09 14:39:43Z

5

There is no need to use eval there, you can retain the same flexibility by passing the distribution types as keyword args (or named args with default values). Parsing and eval'ing "stringly-typed" arguments will often defeat optimizations and should be avoided.

function gendata(N,NLATENT,NITEMS;  LATENT_DIST=Normal(0,1),ERRORS_DIST=Normal(0,1))
         latent = repeat(rand(LATENT_DIST,N,NLATENT), inner=(1,NITEMS))
         errors = rand(ERRORS_DIST,N,NLATENT*NITEMS)
         x = latent+errors
end


julia> gendata(10,2,3, LATENT_DIST=Pareto(.3))
...


julia> gendata(10,2,3, ERRORS_DIST=Gamma(.6))
...

etc.

answered Mar 9, 2017 at 14:39

Isaiah Norton

4,4061 gold badge29 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

phicr · Accepted Answer · 2017-03-10 04:32:03Z

0

You're not really supposed to use eval here (slower, won't produce type information, will interfere with compilation, etc) but in case you're trying to understand what went wrong, here's how you would do it:

Either separate it from the rest of the code:

function gendata(N,NLATENT,NITEMS,LDIST_EX="Normal(0,1)",EDIST_EX="Normal(0,1)")

  # Eval your expressions separately

  LATENT_DIST = eval(parse(LDIST_EX))
  ERRORS_DIST = eval(parse(EDIST_EX))

  # Do your thing

  latent = repeat(rand(LATENT_DIST,N,NLATENT), inner=(1,NITEMS))
  errors = rand(ERROR_DIST,N,NLATENT*NITEMS)
  x = latent+errors      
end

Or use interpolation with quoted expressions:

function gendata(N,NLATENT,NITEMS,LDIST_EX="Normal(0,1)",EDIST_EX="Normal(0,1)")

  # Obtain expression objects

  LATENT_DIST = parse(LDIST_EX)
  ERRORS_DIST = parse(EDIST_EX)

  # Eval but interpolate in everything that's local to the function
  # And you can't introduce local variables with eval so keep them
  # out of it.

  latent = eval( :(repeat(rand($LATENT_DIST,$N,$NLATENT), inner=(1,$NITEMS))) )
  errors = eval( :(rand($ERRORS_DIST, $N, $NLATENT*$NITEMS)) )
  x = latent+errors
end

You can also use a single eval with a let block to introduce a self-contained scope:

function gendata(N,NLATENT,NITEMS,LDIST_EX="Normal(0,1)",EDIST_EX="Normal(0,1)")

  LATENT_DIST = parse(LDIST_EX)
  ERRORS_DIST = parse(EDIST_EX)
  x = 
  @eval let
    latent = repeat(rand($LATENT_DIST,$N,$NLATENT), inner=(1,$NITEMS))
    errors = (rand($ERRORS_DIST, $N, $NLATENT*$NITEMS))
    latent+errors
  end
end

((@eval x) == eval(:(x)))

Well, hope you understand the eval thing a little better. Day two I mean, you should be experimenting ;)

edited Mar 10, 2017 at 4:32

answered Mar 10, 2017 at 4:26

phicr

1,41210 silver badges17 bronze badges

5 Comments

AaronP Over a year ago

Thank you very much. Very helpful to understand "eval" better. But I go with the solutions from the previus answer. Here is my solution with a bit speed optimization: using Distributions function gendata(N,NLATENT,NITEMS;LATENT_DIST=Normal(0,1),ERRORS_DIST=Normal(0,1)) latent = repmat(rand(LATENT_DIST,N,NLATENT), 1,NITEMS) for i in eachindex(latent) latent[i] = latent[i] + rand(ERRORS_DIST) end latent end

AaronP Over a year ago

But there is something I really don't get, while trying to optimize the performance it turns out the function is ten times faster with repmat instead of repeat, how comes that? Both are BLAS and come from the Linear Algebra Section from the Docs.

phicr Over a year ago

Thank you for accepting, but if the other solution is better you should accept that instead, this was more meant to be complementary. (while I am the one addressing the question directly, I think presenting an alternative is better in this case)

phicr Over a year ago

I think there was a talk a while ago about repeat being slow, and there is a Issue, and a recent PR addressing it (22 days ago). Try again on latest master and see if it's any different, otherwise you should probably ask another question about it.

phicr Over a year ago

See issue #20495 and the links referenced in it.

Collectives™ on Stack Overflow

How to use metaprogramming with function args?

2 Answers 2

Comments

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related