[tap-l] multi-file TAP

[Fergal Daly]

On 07/04/07, Michael Peters <mpeters at plusthree.com> wrote:
> Bringing this over from Perl-QA:
>
> Fergal Daly wrote:
> > On 07/04/07, Andy Armstrong <andy at hexten.net> wrote:
> > To address the whatever, BEGIN/END draws you into the XMLish problems
> > of waiting for a complete document before acting on the contents and
> > even if you avoid this it still makes it hard to interleave streams.
> >
> > I'm starting to think that every line should be prefixed by a test
> > number, including diags (diags are not currently associated with a
> > particular test but could be made so) so something like
> >
> > 1.2.3 not ok blah
> > 1.2.3 # got: 4
> > 1.2.3 # not: 3
> > 1.2.3 end
>
> I think this has problems for identifying which test file it came from. For one,
> there is no mention of which file is being processed.

That's because I've omitted most of the output. Each block has a name
(just like each test has a name) embed the script name in there or for
threads/forks give them a sensible name related to what the fork is
doing.

> But the biggest problem is
> that in order for a file (or stream) to know which number it is, the harness
> needs to let it know (passing some environment var to Test::Builder?). This
> means the harness tells the test file which prefix to use just so that the
> harness can then group them properly. Seems backwards. The TAP producers
> shouldn't care about the bookkeeping that the TAP harness needs.

If you have multiple scripts being fired up by a harness, the harness
can blockifies the output of each one. The script doesn't need to know
anything about the block inside which it runs. Just like the harness
is what adds the BEGIN/END lines and indentation in the other
proposal.

F

> Now I'm not saying that nested groups wouldn't be nice, but used within a single
> file/stream, not mixing between them.
>
> Here are my simple thoughts on the matter (and you can substitute "file" for
> "stream"):
>
> + A single test file can be run and output valid TAP and be considered a full
> test run.
> + Multiple test files can be run in whatever sequence the harness wants and
> there should be a way for the harness to join those results together into a
> single valid TAP file so that the entire test run can be captured.
> + Joining the results of multiple files is the job of the harness, not the
> individual test files, so the behavior of the individual files should not be
> altered whether they are being run by themselves, or with others.
>
> In my opinion it would be nice if TAP::Harness would act as both a consumer and
> producer of TAP with the option of joining the individual TAP streams into a
> single file that can then be consumed by something else downstream. If things
> get really complicated with multiple streams running at the same time, then it's
> up to the harness to do the coordinating. I agree with Andy Armstrong in the
> other part of this thread where he said to just give each stream it's own
> TAP::Parser and then coordinate the results when they're done.
>
> And with regard to BEGIN/END blocks, I don't think we need an END, just a BEGIN
> since we can treat it like Perl's "package" and just assume one file until we
> see another.
>
> --
> Michael Peters
> Developer
> Plus Three, LP
>
>