Can I build a WebWorker that executes arbitrary Javascript code?

Question

I'd like to build a layer of abstraction over the WebWorker API that would allow (1) executing an arbitrary function over a webworker, and (2) wrapping the interaction in a Promise. At a high level, this would look something like this:

function bake() {
  ... // expensive calculation
  return 'mmmm, pizza'
}

async function handlePizzaButtonClick() {
  const pizza = await workIt(bake)
  eat(pizza)
}

(Obviously, methods with arguments could be added without much difficulty.)

My first cut at workIt looks like this:

async function workIt<T>(f: () => T): Promise<T> {
  const worker: Worker = new Worker('./unicorn.js') // no such worker, yet
  worker.postMessage(f)
  return new Promise<T>((resolve, reject) => {
    worker.onmessage = ({data}: MessageEvent) => resolve(data)
    worker.onerror   = ({error}: ErrorEvent)  => reject(error)
  })
}

This fails because functions are not structured-cloneable and thus can't be passed in worker messages. (The Promise wrapper part works fine.)

There are various options for serializing Javascript functions, some scarier than others. But before I go that route, am I missing something here? Is there another way to leverage a WebWorker (or anything that executes in a separate thread) to run arbitrary Javascript?

I had a similar struggle a couple of days ago, and thought that it would be easy to wrap worker calls through a Promise. Turned out that it wasn't. Luckily I found this project that does the job (at least for me) npmjs.com/package/promise-worker — antoniom
– antoniom, Commented Jul 22, 2022 at 22:10
1. It sounds like you understand it. 2. Yes, you can de-/serialize functions, but if they're not pure, then you must consider variable scope issues at the sites in your code. 3. All the usual dangers/caveats of injection/eval apply here, etc. — jsejcksn
– jsejcksn, Commented Jul 22, 2022 at 22:10
@antoniom Could you say what pitfalls you ran into with wrapping in a Promise? The simple code above seems to work fine for me but I haven't tested much. — Sasgorilla
– Sasgorilla, Commented Jul 25, 2022 at 14:21
@Sasgorilla not sure if SO comments are the correct place to explain but imagine that I have a single worker object which is instantiated once (not in every function call like the wotkIt function in your case). Then a React component was responsible to use that worker object via postMessage based on user events. But each message in worker is async, so I was not able to distinguish which worker reply was corresponding to which request. I had to create an identifier mechanism so that each request had its own requestId, and then wrap that requestId on each reply — antoniom
– antoniom, Commented Jul 27, 2022 at 14:37

jsejcksn · Accepted Answer · 2022-07-22 23:17:48Z

2

I thought an example would be useful in addition to my comment, so here's a basic (no error handling, etc.), self-contained example which loads the worker from an object URL:

Meta: I'm not posting it in a runnable code snippet view because the rendered iframe runs at a different origin (https://stacksnippets.net at the time I write this answer — see snippet output), which prevents success: in Chrome, I receive the error message Refused to cross-origin redirects of the top-level worker script..

Anyway, you can just copy the text contents, paste it into your dev tools JS console right on this page, and execute it to see that it works. And, of course, it will work in a normal module in a same-origin context.

console.log(new URL(window.location.href).origin);

// Example candidate function:
// - pure
// - uses only syntax which is legal in worker module scope
async function get100LesserRandoms () {
  // If `getRandomAsync` were defined outside the function,
  // then this function would no longer be pure (it would be a closure)
  // and `getRandomAsync` would need to be a function accessible from
  // the scope of the `message` event handler within the worker
  // else a `ReferenceError` would be thrown upon invocation
  const getRandomAsync = () => Promise.resolve(Math.random());

  const result = [];

  while (result.length < 100) {
    const n = await getRandomAsync();
    if (n < 0.5) result.push(n);
  }

  return result;
}

const workerModuleText =
  `self.addEventListener('message', async ({data: {id, fn}}) => self.postMessage({id, value: await eval(\`(\${fn})\`)()}));`;

const workerModuleSpecifier = URL.createObjectURL(
  new Blob([workerModuleText], {type: 'text/javascript'}),
);

const worker = new Worker(workerModuleSpecifier, {type: 'module'});

worker.addEventListener('message', ({data: {id, value}}) => {
  worker.dispatchEvent(new CustomEvent(id, {detail: value}));
});

function notOnMyThread (fn) {
  return new Promise(resolve => {
    const id = window.crypto.randomUUID();
    worker.addEventListener(id, ({detail}) => resolve(detail), {once: true});
    worker.postMessage({id, fn: fn.toString()});
  });
}

async function main () {
  const lesserRandoms = await notOnMyThread(get100LesserRandoms);
  console.log(lesserRandoms);
}

main();

edited Jul 22, 2022 at 23:17

answered Jul 22, 2022 at 23:09

jsejcksn

34.6k5 gold badges55 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Sasgorilla Over a year ago

Man, that is some twisty stuff! I love it. I'm trying to reproduce it with a standalone worker.js file to replace workerModuleText (simplifying for the moment by removing the eval stuff). As soon as I do that, though, the custom id event listener (({detail}) => {console.log('custom id event'); resolve(detail)}) no longer fires. (The two 'message' event handlers, one above and one inside worker.js, both work correctly.) Any idea why that would be?

jsejcksn Over a year ago

@Sasgorilla “the eval stuff” is core to the problem you asked about. It can’t work without it, so you’ll have to clarify.

Sasgorilla Over a year ago

Yes, for sure, but there are (I think?) two different things going on in your answer -- the arbitrary code eval, and the multiplexing identifier mechanism solving the issue raised by @antoniom. It turns out I quickly ran into the problem he mentioned in my own code. So for the moment I was putting aside the eval part (i.e., the original question) and just using a static/conventional worker.js file to try to get the identifier part working.

jsejcksn Over a year ago

@Sasgorilla Yes, using a message-oriented API in a task-oriented way requires a solution that involves a pattern like this. See my answer here which includes an example of an abstraction that addresses this. You can use that module code to register a worker task that includes "the eval stuff".

Collectives™ on Stack Overflow

Can I build a WebWorker that executes arbitrary Javascript code?

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related