Can I profile Lua scripts running in Redis?

Question

I have a cluster app that uses a distributed Redis back-end, with dynamically generated Lua scripts dispatched to the redis instances. The Lua component scripts can get fairly complex and have a significant runtime, and I'd like to be able to profile them to find the hot spots.

SLOWLOG is useful for telling me that my scripts are slow, and exactly how slow they are, but that's not my problem. I know how slow they are, I'd like to figure out which parts of them are slow.

The redis EVAL docs are clear that redis does not export any timekeeping functions to lua, which makes it seem like this might be a lost cause.

So, short a custom fork of Redis, is there any way to tell which parts of my Lua script are slower than others?

EDIT I took Doug's suggestion and used debug.sethook - here's the hook routine I inserted at the top of my script:

redis.call('del', 'line_sample_count') 
local function profile() 
  local line = debug.getinfo(2)['currentline'] 
  redis.call('zincrby', 'line_sample_count', 1, line) 
end 
debug.sethook(profile, '', 100)

Then, to see the hottest 10 lines of my script:

ZREVRANGE line_sample_count 0 9 WITHSCORES

Look at this Redis-wrecker that I made: github.com/RedisLabs/redis-lua-debugger — Itamar Haber
– Itamar Haber, Commented Apr 7, 2015 at 14:42

Doug Currie · Accepted Answer · 2013-05-04 04:54:25Z

2

If your scripts are processing bound (not I/O bound), then you may be able to use the debug.sethook function with a count hook:

The count hook: is called after the interpreter executes every count instructions. (This event only happens while Lua is executing a Lua function.)

You'll have to build a profiler based on the counts you receive in your callback.

The PepperfishProfiler would be a good place to start. It uses os.clock which you don't have, but you could just use hook counts for a very crude approximation.

This is also covered in PiL 23.3 – Profiles

edited May 4, 2013 at 4:54

answered May 4, 2013 at 4:48

Doug Currie

41.4k1 gold badge103 silver badges121 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Drew Shafer Over a year ago

This sent me down the right path - at least as close as I think I'm going to be able to get with the restrictions Redis requires. I'll edit my question with the profiling hook I came up with. Thanks!

user1131435 · Accepted Answer · 2013-05-04 03:42:47Z

0

In standard Lua C, you can't. It's not a built-in function - it only returns seconds. So, there are two options available: You either write your own Lua extension DLL to return the time in msec, or:

You can do a basic benchmark using a millisecond-resolution time. You can access the current millisecond time with LuaSocket. Though this adds a dependency to your project, it's an effective way to do trivial benchmarking.

require "socket"
t = socket.gettime();

answered May 4, 2013 at 3:42

user1131435

6 Comments

Drew Shafer Over a year ago

The Lua script is running inside a Redis server. The only libraries I can use are : base, table, string, math, debug, cjson, and cmsgpack.

user1131435 Over a year ago

Ah. The only way I can think of is to manually add libraries. It's open source. It would be a pain, though. I'm surprised you can't add your own libraries.

Drew Shafer Over a year ago

The limitation on timekeeping functions is purposeful - you can't even call the builtin time function, because that would break the guarantee of deterministic execution required for the redis-cluster implementation.

user1131435 Over a year ago

Can you attach your own libraries as Lua addons in DLL form?

Drew Shafer Over a year ago

I'd have to recompile Redis and link them in statically - not out of the question, but I'd like to see if there's an easy button first...

|

Collectives™ on Stack Overflow

Can I profile Lua scripts running in Redis?

2 Answers 2

1 Comment

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related