0

In Redis we have strings that represent input values. We (would like to) have a Lua script that is dynamically generated (after being defined by user using a GUI) that calculates a result string based on the input string. Each set of input values is independent of each other. So this should be trivially parallelisable, however, EVAL seems to block until completion.

Is there a way in Redis to run a single Lua script across a bunch of values without having to rewrite the script itself to do it?

Improve this question

1 Answer 1

Reset to default
2

Since Redis is implemented as a single-threaded server, it would not be possible to run multiple commands from the same client in parallel. You should be however, be able to run multiple commands, (script commands included), on multiple clients, and Redis will interleave them in its IO loop.

Having said that, Redis is not only super-fast, it is also flexible; Please consider one of the following options:

  1. Write a Redis module yourself, add a command, or add a new command to your likings.
  2. Consider a multi-sharded environment. This will enable each redis-server (or shard) to run independently, while you can direct the call to the correct shard. This is somewhat labor-intensive, if you consider this solution, please let me know I can further instruct you on the matter.

All the best!

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

Hmm, that sounds like I would have to artificially shard my rows. Something like calculate the sha1 of each row, then modulus n, where n is my number of shards and then at least I get some load sharing that way...
Exactly - to be even more precise - some hashing function on the key of the entry (Don't consider the value of the entry as input for the hash), otherwise, the (key, value) pair might need to migrate when you will update the value.
Moreover, there are some open source proxies that will do this for you out of the box, as well as a commercial one.
@Elior Malul Can multiple clients run parallel script in redis ? For example 100 clients run 100 scripts concurrent.
@MuhammadFaizanFareed The open source redis is single threaded (disclaimer - been 5-6 years since I left), but it was true for the time - so no 'real' concurrency there.Oran Agra is the open source maintainer, he will know for sure. If you are ar Redis Labs customer, than yes, for sure.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.