iterating through data array/hashes

Question

I am looking to iterate through these data structures (basically a directory structure) which I pass a given path.

The goal is to list the root/base path and then list out all the sub paths if they exist and for every sub path that exists, list the file from that sub path.

I know this probably requires looping through HoH or AoH. Can someone show me the perl techniques for doing this? Thanks.

Base level: /work/eng/feeds

 $VAR1 = {
              'recursive' => 'no',
              'version' => '0.20.202.1.1101050227',
              'time' => '2011-10-26T00:20:18+0000',
              'filter' => '.*',
              'path' => '/work/eng/feeds',
              'directory' => [
                             {
                               'owner' => 'tst_act',
                               'group' => 'eng',
                               'permission' => 'drwxrwxr-x',
                               'path' => '/work/eng/feeds',
                               'accesstime' => '1970-01-01T00:00:00+0000',
                               'modified' => '2011-08-27T03:13:53+0000'
                             },
                             {
                               'owner' => 'tst_act',
                               'group' => 'eng',
                               'permission' => 'drwxr-xr-x',
                               'path' => '/work/eng/feeds/customer_care',
                               'accesstime' => '1970-01-01T00:00:00+0000',
                               'modified' => '2011-10-25T23:54:17+0000'
                             }
                           ],
              'exclude' => ''
            };

Next level: /work/eng/feeds/customer_care

$VAR1 = {
          'recursive' => 'no',
          'version' => '0.20.202.1.1101050227',
          'time' => '2011-10-26T00:21:06+0000',
          'filter' => '.*',
          'path' => '/work/eng/feeds/customer_care',
          'directory' => [
                         {
                           'owner' => 'tst_act',
                           'group' => 'eng',
                           'permission' => 'drwxr-xr-x',
                           'path' => '/work/eng/feeds/customer_care',
                           'accesstime' => '1970-01-01T00:00:00+0000',
                           'modified' => '2011-10-25T23:54:17+0000'
                         },
                         {
                           'owner' => 'tst_act',
                           'group' => 'eng',
                           'permission' => 'drwx------',
                           'path' => '/work/eng/feeds/customer_care/abc',
                           'accesstime' => '1970-01-01T00:00:00+0000',
                           'modified' => '2011-10-25T17:12:56+0000'
                         },
                         {
                           'owner' => 'tst_act',
                           'group' => 'eng',
                           'permission' => 'drwx------',
                           'path' => '/work/eng/feeds/customer_care/def',
                           'accesstime' => '1970-01-01T00:00:00+0000',
                           'modified' => '2011-10-25T21:05:50+0000'
                         },
                         {
                           'owner' => 'tst_act',
                           'group' => 'eng',
                           'permission' => 'drwx------',
                           'path' => '/work/eng/feeds/customer_care/test',
                           'accesstime' => '1970-01-01T00:00:00+0000',
                           'modified' => '2011-10-25T21:28:14+0000'
                         }
                       ],
          'exclude' => ''
        };

One more level: /work/eng/feeds/customer_care/test (here a file exists)

$VAR1 = {
          'recursive' => 'no',
          'version' => '0.20.202.1.1101050227',
          'time' => '2011-10-26T00:30:02+0000',
          'filter' => '.*',
          'file' => {
                    'owner' => 'tst_act',
                    'replication' => '3',
                    'blocksize' => '134217728',
                    'permission' => '-rw-------',
                    'path' => '/work/eng/feeds/customer_care/test/q_data_20111023.dat',
                    'modified' => '2011-10-26T00:29:46+0000',
                    'size' => '379085',
                    'group' => 'eng',
                    'accesstime' => '2011-10-26T00:29:46+0000'
                  },
          'path' => '/work/eng/feeds/customer_care/test',
          'directory' => {
                         'owner' => 'tst_act',
                         'group' => 'eng',
                         'permission' => 'drwx------',
                         'path' => '/work/eng/feeds/customer_care/test',
                         'accesstime' => '1970-01-01T00:00:00+0000',
                         'modified' => '2011-10-26T00:29:46+0000'
                       },
          'exclude' => ''
        };

What have you tried? What specific issues are you running into? Do you just need to know the names of each, keys and values, or to read perlreftut? — derobert
– derobert, Commented Oct 26, 2011 at 1:43
If I can capture each of the keys and values, that would be a good start. — jdamae
– jdamae, Commented Oct 26, 2011 at 2:01

stevenl · Accepted Answer · 2011-10-26 02:21:04Z

Here's a starter:

sub list_path_files {
    my ($data) = @_;

    say $data->{path}; # get value from a hashref

    my @directories;
    # check whether it is a single value or an arrayref of values
    if (ref $data->{directory} eq 'ARRAY') {
        @directories = @{ $data->{directory} }; # dereference the arrayref to get an AoH
    } else {
        @directories = $data->{directory}; # just get the single value
    }

    for my $dir (@directories) {
        next if $dir->{path} eq $data->{path};
        say $dir->{path};
    }

    # I'll leave the rest for you to do
}

Update:

To iterate the hashref, you first need to dereference it, then use the each, keys or values functions:

%hash = %$VAR1; # dereference

while (my ($key, $value) = each %hash) {...}

for my $key (keys %$VAR1) {
    my $value = $VAR1->{$key};
}

for my $value (values %$VAR1) {...}

You will also need to dereference the hash value if it is a nested structure:

if (ref $val eq '') {
    # $val is just a scalar - don't need to deref
}
elsif (ref $val eq 'HASH') {
    my %hash = %$val;
}
elsif (ref $val eq 'ARRAY') {
    my @array = @$val;
}

John · Accepted Answer · 2011-10-26 15:46:00Z

0

Since there's XML-esque timestamps in the data, and the directory/file structures are either a hash or an array of hashes, I'm going to guess that you're parsing XML to build the initial hash ?

If you're using XML::Simple to read this from XML,

ForceArray => [ 'directory','file' ]

will avoid having to check ref types.

Then you can simplify to something like this :

my %fs

for my $item (values %your_giant_hash_with_fs_data) {

    $fs{$_->{path}}='d' for ( @{ $item->{directory} } );
    $fs{$_->{path}}='f' for ( @{ $item->{file} } );

}

my $search_path='/work/eng/feeds/customer_care/test';
my $search=qr/^$search_path/;

for (sort keys %fs) {

    print "$fs{$_} $_\n" if /$search/;

}

Output :

d /work/eng/feeds/customer_care/test
f /work/eng/feeds/customer_care/test/q_data_20111023.dat

or if you don't want to create a temporary structure

my $search_path='/work/eng/feeds/customer_care/test';
my $search=qr/^$search_path/;

{
    my %seen;
    for my $item (values %your_giant_hash_with_fs_data) {
        for my $type (qw/directory file/) {
            for ( @{ $item->{$type} } ) {

                if ( $_->{path}=~/$search/ and not $seen{$_->{path}} ) {

                    printf("%-10s%s\n",$type,$_->{path});
                    $seen{$_->{path}}=1;

                }

            }
        }
    }
}

This only works becaues each of your paths are absolute anyway, and absolute paths can't be duplicated

edited Oct 26, 2011 at 15:46

answered Oct 26, 2011 at 15:22

John

5732 silver badges6 bronze badges

3 Comments

jdamae Over a year ago

thanks, yes I am using a module which is using XML::Simple to get the data. I'll start testing this.

jdamae Over a year ago

I'm still trying to figure out how to loop through this. Here is how I get the hash data: my $listDetails=$cmd->ls("/work/eng/feeds/customer_care");. Does this replace my hash? from: for my $item (values %your_giant_hash_with_fs_data) to for my $item (values %listDetails)? thanks again.

John Over a year ago

Ah. I assumed that all the data was stored in a giant hash, and you didn't read individual files as they became a dependency.

Collectives™ on Stack Overflow

iterating through data array/hashes

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related