How can I run multiple tests scripts in perl giving parameters from another one?

Question

I have being taking a look at some modules of perl, and I have seen it is kind of a standard to have a lot of tests under a folder t whith the extension .t.

Those scripts can be executed directly with perl <test>.t.

I want to do a little framework to test one application. I want to have many simple tests. And then a main script which calls this tests to validate everything at once, but still being able to run the little tests individually without having to disable manually the other tests in the main script.

My question is how should I do this calling? As suggested in How do I run a Perl script from within a Perl script? this interprocess communication should be avoided and instead use modules but that would make not possible to run the tests directly.

My guess was to do something as follows. In an example where I need to give some parameters in a selenium script This is the folder structure:

├── main_script.pl
└── t
    ├── 00-Test_1.t
    ├── 01-Test_2.t
    ├── 02-Test_3.t
    └── 03-Test_4.t

And I would call them like this:

system($^X, "./t/00-Test_1.t", "-b $browser", "-s $server", "-d $debug");
my $results = capture($^X, "./t/00-Test_1.t", "-b $browser", "-s $server", "-d $debug";
$log->info("Result is $results");

And each test would have something like this:

my ($browser, $server, $debug);
GetOptions(
    'debug|d'                   => \$debug,
    'debug|d'                   => \$trace,
    'trace|t'                   => \$browser,
    'server|s=s'                => \$server,
) or die "[ERROR] Invalid options passed to $0\n";

Is this a correct approach or is there something more appropiate for doing this using modules?

You already have the prove utility to run several tests at once. If you want to control their behaviour, you might want to look at one of the existing frameworks like Test::Class or Test::Class::Moose to write your tests in. If you want to ship to CPAN that'll pull in extra dependencies, but for proprietary stuff it's great. — simbabque
– simbabque, Commented Sep 24, 2020 at 12:24
@simbabque I have been taking a look at it. But I think by using you can not get the result from the main script, for example to make a report from it. — nck
– nck, Commented Sep 24, 2020 at 13:21

Dave Sherohman · Accepted Answer · 2020-09-24 13:20:16Z

6

You already have "a main script which calls this tests to validate everything at once, but still being able to run the little tests individually" as part of the standard Perl distribution. It's called prove.

Running prove with no parameters will run all test scripts matching t/*.t (so you want the tests in a t/ subdirectory when you run prove, not in the same directory), or you can use prove t/00-Test_1.t to run a single test script.

Note that prove expects the test scripts to emit output in TAP ("Test Anything Protocol") format, which is usually accomplished by using Test::More or other Test::* modules in the test scripts.

If you follow the convention of putting your application-specific modules in a lib/ subdirectory, then running prove -l will add them to @INC to make them available when running the test scripts. i.e.,

Root directory (run `prove` from here)
├── lib
|   └── MyProject
|       └── MyModule.pm
└── t
    ├── 00-Test_1.t
    ├── 01-Test_2.t
    ├── 02-Test_3.t
    └── 03-Test_4.t

answered Sep 24, 2020 at 13:20

Dave Sherohman

46.4k14 gold badges67 silver badges104 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

nck Over a year ago

But does this allow to do post processing on the results of the test?. For example if I want to generate a personalized report in the main script given the results.

simbabque Over a year ago

@nck how do you mean? You can use various Tap::Harness:: modules that will work with prove, or make your own. For example, there's metacpan.org/pod/TAP::Harness::JUnit. You will see that prove and tests written in Perl are very widely used in a lot of open source software where the main program isn't written in Perl. Good examples are Linux core utilities written in C that all have tests in Perl.

zdim Over a year ago

@nck Can always run prove out of a script that you'd write specifically for processing its output and generating the report. (Or out of whatever other "main" program you have.) Make sure to browse the docs of related components, and of prove itself, for a myriad of options.

nck · Accepted Answer · 2020-09-28 14:47:40Z

Although the answer of @Dave Sherhman is good enough, I will post how I made it work for my project together with the suggestion of @simbabque. As it took me a while to understand how it works and maybe it helps another one save some time:

My *.t script are now as the following script. Where I specify the number of tests that are inside it, and also I can give some input parameters:

use strict;
use warnings;

use Test::More tests => 2;

use Getopt::Long qw(GetOptions);
use Drivers::Driver;
use Data::Dumper;

my ($browser, $server, $debug, $trace, $user, $password);
GetOptions(
    'debug|d'                   => \$debug,
    'trace|t'                   => \$trace,
    'user|u:s'                  => \$user,
    'password|p:s'              => \$password,
    'browser|b=s'               => \$browser,
    'server|s=s'                => \$server,
) or die "[ERROR] Invalid options passed to $0\n";

ok(do_some_test($browser) == 1);
ok(do_another_test($user, $password) == 1);

done_testing;

Then my main script calls the scripts I want as follows:

use strict;
use utf8;
use open ':std', ':encoding(UTF-8)';

use TAP::Harness;
use TAP::Formatter::HTML;

my $fmt = TAP::Formatter::HTML->new();
$fmt->output_file( 'foo.html' );

my $test_args = {
  '00_test'    => ['--server', "$server", $debug?'-d':undef, $trace?'-t':undef, '--browser', "firefox"],
  '01_test'    => ['--server', "$server", $debug?'-d':undef, $trace?'-t':undef, '--browser', "chrome"],
};
my $test_output;
my $harness = TAP::Harness->new( {
        test_args => $test_args,
        formatter => $fmt,
        merge => 1
    } );
    
my $result = $harness->runtests( 
    [ 't/00-test.t',    '00_test' ],
    [ 't/01-test.t',    '01_test' ],
);


print "Passed: ". scalar $result->passed . "\n";
print "Failed: ". scalar $result->failed . "\n";

generate_report($result);

This won't work out of the box as I just deleted some lines from my project but gives an idea on how is it able to run the other files and get the results to generate a report, while still being able to run each project individually and also being able to send common parameters to the different tests from the main script.

Collectives™ on Stack Overflow

How can I run multiple tests scripts in perl giving parameters from another one?

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related