Adding Key/Value To Hash(Perl)

Question

Hi I'm trying to add multiple keys/values into a hash. Basically file names and their data. Each of the json file's content have hash and array referencing. The hash that contains the file names and data will be handled else where.

This is my code:

 sub getDecode {
    my $self = shift;
    my @arrUrls = ('http://domain.com/test.json', 'http://domain.com/test_two.json', 'http://domain.com/test3.json');

    my $resQueue = Thread::Queue->new();
    my $intThreads = 10;
    my @arrThreads = ();
    my %arrInfo = ();

    foreach my $strUrl (@arrUrls) {
          for (1..$intThreads) {
              push (@arrThreads, threads->create(sub {
                          while (my $resTask = $resQueue->dequeue) {
                                      my $resData = get($strUrl);
                                      my $strName = basename($strUrl, '.json');
                                      my $arrData = decode_json($resData);
                                      $arrInfo{$strName} = $arrData;
                          }
              }));
          }
   }

   $resQueue->enqueue(@arrUrls);
   $resQueue->enqueue(undef) for 1..$intThreads;
   $_->join for @arrThreads;

  return %arrInfo;       
  }

When I try data dumping on %arrInfo no output is given. Please help!

Community · Accepted Answer · 2017-05-23 12:28:31Z

1

You're multithreading and not sharing the variable. When a thread spawns, the existing variable-space is cloned - so each thread has it's own local copy of %arrInfo which is discarded when it exits.

You need to:

use threads::shared;
my %arrInfo : shared;

You're also doing something a bit odd with your thread spawn loop - you're spawning 10 threads x 3 URLs - for 30 threads, but only queuing 3 URLs to process. But then you're also not actually using $resTask at all, which doesn't make a lot of sense.

So I'm prepared to bet your code hangs at the end, because you're trying to join some threads that aren't complete.

You might find $resQueue -> end() is more suitable than queuing up undef.

Example using a shared hash:

use strict;
use warnings;
use threads;
use threads::shared;

use Data::Dumper;

my %test_hash : shared;
my %second_hash;

$test_hash{'from_parent'}       = 1;
$second_hash{'from_parent_too'} = 1;

threads->create(
    sub {
        $test_hash{'from_first'}       = 2;
        $second_hash{'from_first_too'} = 2;
    }
);
threads->create(
    sub {
        $test_hash{'from_second'}       = 3;
        $second_hash{'from_second_too'} = 3;
    }
);

foreach my $thr ( threads->list() ) { $thr->join }

print Dumper \%test_hash;
print Dumper \%second_hash;

For a 'worker threads' style approach, I'd offer: Perl daemonize with child daemons

edited May 23, 2017 at 12:28

CommunityBot

11 silver badge

answered Feb 4, 2015 at 11:59

Sobrique

53.6k8 gold badges63 silver badges107 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

user2524169 Over a year ago

There's actually more urls but I removed them. Thank you though :)

Sobrique Over a year ago

You're still spawning 10 threads per URL, and it looks like 9 of those will be redundant.

user2524169 Over a year ago

Just a question, it keeps saying "invalid value for shared scalar"

Sobrique Over a year ago

Not sure why. My test case works ok. Which line gives that error? (And are you still assigning an empty array to %arrInfo because that's redundant)

Sobrique Over a year ago

Edited answer with example

|

Collectives™ on Stack Overflow

Adding Key/Value To Hash(Perl)

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related