[tap-l] Parallel testing

[Eric Wilhelm]

# from H.Merijn Brand
# on Thursday 10 April 2008 05:58:
>In Oslo Andy and I came up with a way to define how Test::Harness
> should deal with parallel tests, like this:

I think this discussion belongs on the harness list, as it has little to 
do with the tap protocol.

>conflict:
>    folder:     ../ext/DB_File/t
>    folder:     ../ext/IO_Compress_Zlib/t
>    folder:     ../lib/CPANPLUS
>    folder:     ../lib/ExtUtils/t
>#   folder:     ../lib/Pod/t
>
># first file should be run before file 2, 3, 4, ...
>depends:
>    depend:     [ foo.t bar.t ]

I can understand the desire to parallelize a crufty test suite which is 
making bad assumptions, but I'm troubled by the idea that this needs to 
be specified in a config file and I think the 'depends' section is just 
blessing bad practice.

A long time ago, my TAP::Harness::Parallel implemented a very simple 
partition() method which allowed an option to just automatically 
partition the process queue by directory.  This almost always proved to 
be sufficient for working-around tests which assumed zero concurrency.  
It did potentially sacrifice some speed if your directories were too 
few or imbalanced.  But externally configuring it to optimize speed 
while working around assumptions which cause collisions implies an 
attempt to polish something with a soft brown surface.

As far as situations where automatic directory partitions are incorrect 
(two files in a directory have a resource conflict), you haven't 
addressed that unless you add a 'file' keyword.  But then you need to 
e.g. spec a list of files to be run in sequence in one process and 
suddenly you're writing two copies of your MANIFEST.

I saw the next step up from "simple workaround" as "fix your tests", and 
this proposed config file appears to require half of that work but 
without getting you halfway to the ideal state of non-broken tests.

Further, clustered (multi-node parallelization) testing breaks dependent 
tests in new and different ways, so whatever is done to line-up the 
dependencies seems likely to get in the way of clustering.

--Eric
-- 
"Time flies like an arrow, but fruit flies like a banana."
--Groucho Marx
---------------------------------------------------
    http://scratchcomputing.com
---------------------------------------------------