3

I have to do several tasks in one Perl script. Better not use fork or thread.

  1. task A: make HTTP request to a server every 5 seconds. Infinity and should not be blocked. And if gets:

    1. 'TASK', task A will call a time consuming sub process, so how to avoid the assignment execution block task A's loop?
    2. 'PAUSE', pause the request
    3. 'UPDATE', tell task B to do something
    4. make POST request to server when receive data from task B or task C
  2. task B: make mysql request every 1 minute, and depend on results, will tell task A to do POST request

  3. task C: accept socket connection and tell task A or B to do something.

Three parallel infinitive loop processes, and will communication with each other. How can I do that?

5
  • 2
    What is the problem with fork or therad? They were designed for this kind of stuff. Commented Jun 4, 2013 at 5:58
  • I'd actually use threads here, as communication via queues is rather simple, and optionally non-blocking. If you'd fork, you can communicate via pipes (connected filehandles). If you put each task in a seperate script, you could use named pipes or sockets. Which is a lot more complexity. At least for task A you'd use threads to achieve nonblocking properties (e.g. five http workers which execute requests, enough workers for the time consuming sub) Commented Jun 4, 2013 at 6:32
  • Why does 1.4 need to be done by task A??? Commented Jun 4, 2013 at 6:37
  • 1
    You can also use POE for this sort of thing. Commented Jun 4, 2013 at 6:53
  • ...or Coro or AnyEvent or ... How about we keep things simple? Commented Jun 4, 2013 at 7:07

1 Answer 1

11

That design makes no sense whatsoever, and the claim that it's better not to use threads or child processes makes even less sense.

You have three sources of requests:

  • Request Source A: Make web request to a server every 5 seconds.
  • Request Source B: Make database request to a server every 60 seconds.
  • Request Source C: Accept requests from a socket.

Create a thread for each Request Source. Their job is solely to monitor each Request Source in order to ensure that the Sources are checked when they should be checked. As such, none of these threads should do any real work. If a task must be performed, they delegate the work to a worker thread. They don't POST anything. They don't write to the database.

The actual tasks (including sending POSTs and writing to the database) are performed by one or more worker threads (your choice). The worker threads receive requests from a singular Thread::Queue queue populated by the three Request Sources.


So the code would look like:

use threads;
use Thread::Queue qw( );

use constant NUM_WORKERS => 5;  # Tweak this. Can be as low as 1.

sub poll_web {
   my ($request_q) = @_;
   ... init ...
   while (1) {
      ...
      $request_q->enqueue([post => ...]);
      ...
   }
}

sub poll_db { ... }              # Just like poll_web

sub accept_connections { ... }   # Just like poll_web

sub post_handler { ... }         # Receives args passed to enqueue

{
   my $request_q = Thread::Queue->new();

   my %job_handlers = (
      post => \&post_handler,
      ...
   );

   for (1..NUM_WORKERS) {
      async {
         while (1) {
            my $job = $request_q->dequeue();
            my ($job_type, @args) = @$job;
            my $handler = $job_handlers{$job_type};
               or do { warn("Unknown job type $job_type"); next };
            $handler->(@args);
         }
      };
   }

   async { poll_web($request_q); };
   async { poll_db($request_q); };
   accept_connections($request_q);
}

If you want to use processes instead of threads, change

use threads;

to

use forks;

but keep using Thread::Queue.

Sign up to request clarification or add additional context in comments.

1 Comment

+1 for having the patience to write an answer that suits more than the single question. Thanks.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.