Julia create array of functions with matching arguments to execute in a loop

Question

It's easy to create an array of functions and execute them in a loop.
It's easy to provide arguments in either a corresponding array of the same length or the array could be of tuples (fn, arg).

For 2, the loop is just

for fn_ar in arr  # arr is [(myfunc, [1,2,3]), (func2, [10,11,12]), ...]
    fn_ar[1](fn_ar[2])
end

Here is the problem: the arguments I am using are arrays of very large arrays. In #2, the argument that will be called with the function will be the current value of the array when the arg entry of the tuple is initially created. What I need is to provide the array names as the argument and defer evaluation of the arguments until the corresponding function is run in the loop body.

I could provide the arrays used as input as an expression and eval the expression in the loop to supply the needed arguments. But, eval can't eval in local scope.

What I did that worked (sort of) was to create a closure for each function that captured the arrays (which are really just a reference to storage). This works because the only argument to each function that varies in the loop body turns out to be the loop counter. The functions in question update the arrays in place. The array argument is really just a reference to the storage location, so each function executed in the loop body sees the latest values of the arrays. It worked. It wasn't hard to do. It is very, very slow. This is a known challenge in Julia.

I tried the recommended hints in the performance section of the manual. Make sure the captured variables are typed before they are captured so the JIT knows what they are. No effect on perf. The other hint is to put the definition of the curried function with the data for the closure in let block. Tried this. No effect on perf. It's possible I implemented the hints incorrectly--I can provide a code fragment if it helps.

But, I'd rather just ask the question about what I am trying to do and not muddy the waters with my past effort, which might not be going down the right path.

Here is a small fragment that is more realistic than the above:

Just a couple of functions and arguments:

(affine!, "(dat.z[hl], dat.a[hl-1], nnw.theta[hl], nnw.bias[hl])")
(relu!, "(dat.a[hl], dat.z[hl])")

Of course, the arguments could be wrapped as an expression with Meta.parse. dat.z and dat.a are matrices used in machine learning. hl indexes the layer of the model for the linear result and non-linear activation.

A simplified version of the loop where I want to run through the stack of functions across the model layers:

function feedfwd!(dat::Union{Batch_view,Model_data}, nnw, hp, ff_execstack)  

    for lr in 1:hp.n_layers
        for f in ff_execstack[lr]
            f(lr)
        end
    end

end

So, closures of the arrays is too slow. Eval I can't get to work.

Any suggestions...?

Thanks, Lewis

why do you need an array of funcitons? why don't you just call the functions directly? — David Sainez
– David Sainez, Commented Jan 23, 2020 at 3:12
Making the arguments a closure sounds reasonable. Why is it slow? — Michael K. Borregaard
– Michael K. Borregaard, Commented Jan 23, 2020 at 8:45
How about passing the array structure explicitly, like f(dat, lr) then f can modify dat inplace which should resolve the efficiency issue. — Carl Morris
– Carl Morris, Commented Jan 23, 2020 at 9:06
I would try to rewrite the question to make it clearer. 1) Why do you need arrays of functions? 2) It is very unclear why you absolutely must have closures. Could you give simplified code example of solution with and without closures and explain why you perceive as the problem? Closures are harder to grasp from a performance perspective in Julia. 3) Don't get why you need eval for arrays. Are these very large arrays you temporary create and destroy in memory? You don't want to create them all at the same time? — Erik Engheim
– Erik Engheim, Commented Jan 23, 2020 at 9:43
@Engheim: I don't need closures. I thought I made it clear that was just a reasonable attempt to do this. Arrays of functions simplify running the code: instead of some if/else and switches in the hot loop, I "decide" what is needed in advance (for a given model it will not change), and run the functions. I am not clear on eval of the arguments that reference the arrays--hence the question. Since that cannot work, I need an alternative. The reason for the eval was to attempt to defer evaluation of the argument tuple. If I build it in advance, ... contd. next comment — Lewis Levin
– Lewis Levin, Commented Jan 23, 2020 at 16:17

Lewis Levin · Accepted Answer · 2020-01-23 23:58:02Z

I solved this with the beauty of function composition.

Here is the loop that runs through the feed forward functions for all layers:

for lr in 1:hp.n_layers
    for f in ff_execstack[lr]
        f(argfilt(dat, nnw, hp, bn, lr, f)...)
    end
end

The inner function parameter to f called argfilt filters down from a generic list of all the inputs to return a tuple of arguments needed for the specific function. This also takes advantage of the beauty of method dispatch. Note that the function, f, is an input to argfilt. The types of functions are singletons: each function has a unique type as in typeof(relu!), for example. So, without any crazy if branching, method dispatch enables argfilt to return just the arguments needed. The performance cost compared to passing the arguments directly to a function is about 1.2 ns. This happens in a very hot loop that typically runs 24,000 times so that is 29 microseconds for the entire training pass.

The other great thing is that this runs in less than 1/10 of the time of the version using closures. I am getting slightly better performance than my original version that used some function variables and a bunch of if statements in the hot loop for feedfwd.

Here is what a couple of the methods for argfilt look like:

function argfilt(dat::Union{Model_data, Batch_view}, nnw::Wgts, hp::Hyper_parameters, 
    bn::Batch_norm_params, hl::Int, fn::typeof(affine!))
    (dat.z[hl], dat.a[hl-1], nnw.theta[hl], nnw.bias[hl])
end

function argfilt(dat::Union{Model_data, Batch_view}, nnw::Wgts, hp::Hyper_parameters, 
    bn::Batch_norm_params, hl::Int, fn::typeof(relu!))
    (dat.a[hl], dat.z[hl])
end

Background: I got here by reasoning that I could pass the same list of arguments to all of the functions: the union of all possible arguments--not that bad as there are only 9 args. Ignored arguments waste some space on the stack but it's teeny because for structs and arrays an argument is a pointer reference, not all of the data. The downside is that every one of these functions (around 20 or so) all need to have big argument lists. OK, but goofy: it doesn't make much sense when you look at the code of any of the functions. But, if I could filter down the arguments just to those needed, the function signatures don't need to change.

It's sort of a cool pattern. No introspection or eval needed; just functions.

Collectives™ on Stack Overflow

Julia create array of functions with matching arguments to execute in a loop

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related