How to determine the execution time of a Lua script in Redis?

Question

I have a small Lua script to run in Redis, and I'm interested in getting the execution time.

Due to the nature of Redis and it's Lua implementation, I cannot use the TIME function at the start/return points of the script, and include this information in the return for processing (See http://redis.io/commands/eval - Scripts as pure functions). This results in an error: (error) ERR Error running script (call to f_a49ed2fea72f1f529843d6024d1515e76e69bcbd): Write commands not allowed after non deterministic commands

I have searched around for a function/call I could make which will return the execution time of the last run script, but have not found anything yet.

I am using PHP and the Predis Library. While I can check the execution time from PHP side, I wish to remove the transmission overhead and find out how long the Lua script will block access to the database. I have successfully had times returned if I do not need to alter any of the data stored in Redis, and these have been about 1/10th the time PHP reports.

How can I determine the execution time of the Lua script in Redis, and not via PHP?

Most implementations have os.clock() which returns the amount of time since the start of execution. — Darkwater
– Darkwater, Commented Nov 5, 2012 at 12:34
I've just tried calling os.clock() in a script, and I'm getting an error back saying Script attempted to access unexisting global variable 'os' . Looks like this is unavailable. Thanks anyway! — Pete C
– Pete C, Commented Nov 5, 2012 at 13:51
@PeteC OS library in not present in LUA for redis, please refer stackoverflow.com/a/31433599/1288120 — Bharat Pahalwani
– Bharat Pahalwani, Commented Apr 19, 2017 at 9:58

Didier Spezia · Accepted Answer · 2012-11-05 16:15:56Z

5

You can activate Redis slow log feature, by changing the slowlog-log-slower-than parameter to 0. It will record execution time of ALL commands (including Lua scripts, and whatever the execution time).

The slow log is kept in an in-memory queue you have to dump on a regular basis to collect data. Depending on the volume of traffic, you may have to increase slowlog-max-len to be sure to catch the execution times you are interested in.

You can use the slowlog get command to dump the slow log. Up to you to filter out the results you don't need. AFAIK, there is no possibility to filter at data collection time (to keep only Lua statistics).

answered Nov 5, 2012 at 16:15

Didier Spezia

73.6k12 gold badges194 silver badges157 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Alex · Accepted Answer · 2014-03-18 12:04:13Z

You can use os.clock() inside script if you enable os module when building Redis.

Change #if 0 to #if 1 on line 484 in https://github.com/antirez/redis/blob/unstable/src/scripting.c:

void luaLoadLibraries(lua_State *lua) {
    luaLoadLib(lua, "", luaopen_base);
    luaLoadLib(lua, LUA_TABLIBNAME, luaopen_table);
    luaLoadLib(lua, LUA_STRLIBNAME, luaopen_string);
    luaLoadLib(lua, LUA_MATHLIBNAME, luaopen_math);
    luaLoadLib(lua, LUA_DBLIBNAME, luaopen_debug); 
    luaLoadLib(lua, "cjson", luaopen_cjson);
    luaLoadLib(lua, "struct", luaopen_struct);
    luaLoadLib(lua, "cmsgpack", luaopen_cmsgpack);

#if 0 /* Stuff that we don't load currently, for sandboxing concerns. */
    luaLoadLib(lua, LUA_LOADLIBNAME, luaopen_package);
    luaLoadLib(lua, LUA_OSLIBNAME, luaopen_os);
#endif
}

Itamar Haber · Accepted Answer · 2016-02-05 13:26:17Z

0

EDIT: rld is obsolete.

You can use rld (https://github.com/RedisLabs/redis-lua-debugger) and have it print to the log - the log has timing information (although rld will cause your script to run slower, naturally).

edited Feb 5, 2016 at 13:26

answered Apr 7, 2015 at 14:43

Itamar Haber

50.5k7 gold badges108 silver badges129 bronze badges

Comments

Itamar Haber · Accepted Answer · 2016-02-05 13:29:41Z

0

As of Redis v3.2, you can choose to replicate commands instead of scripts with a call to the new redis.replicate_commands() API. This will allow you, among other things, to call TIME in scripts that perform write operations without triggering the non-determinism error.

answered Feb 5, 2016 at 13:29

Itamar Haber

50.5k7 gold badges108 silver badges129 bronze badges

Collectives™ on Stack Overflow

How to determine the execution time of a Lua script in Redis?

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related