Test Attributes

From Test Anything Protocol

Jump to: navigation, search

Contents

[edit] Status

This proposal is largely supplanted by the TAP diagnostic syntax and the TAP meta information. It is considered to be rejected.

[edit] For

[edit] Against

[edit] Proposal

I propose using of subset of the Config::Std syntax to allow tests to declare attributes that are captured in Test::TAP::Model as a hash. That would allow the test to declare configuration information, such as its version id or its dependencies, so that information would be available to downstream processors of the test results.

I don't think the [Sections] declarations from Config::Std can be justified, but just about everything else seems appropriate. (Actually, if Config::Std was updated to allow parsing of a string instead of a file, then we could use it instead of creating a new parser.)

I assume everything should appear to be a comment to old Test::Harness parsers, so a leading "#" is required. If that assumption could be dropped it would be nice, but only because the declarations would look cleaner. That is especially true for nested comments.

I also assume there will be a unique string marker to indicate that TAP attributes are about to be declared. I use "TAP ATTR" for my examples below, but that could easily be changed.

I'll leave the parsing details, the treatment of white space for example, to Damian's description in the Config::Std module. The examples below should be sufficient for discussion.

The "TAP ATTR" marker can be used on every line where a configuration variable is being declared: 

       # TAP ATTR key1 = value1
       # TAP ATTR test version = $Revision: 1.3 $
       # TAP ATTR depends on = blib/Test/Koala/Tea.pm
       # TAP ATTR depends on = blib/Test/Koala/Eucalyptus/Tea.pm
       # TAP ATTR depends on = blib/Test/Koala/Tree.pm

Or using a << block can be used for a less cluttered view: 

       # TAP ATTR << Kangaroo_on_a_pogo_stick
       #
       #       key1 = value1
       #
       #       test version = $Revision: 1.3 $
       #
       #       depends on = blib/Test/Koala/Tea.pm
       #       depends on = blib/Test/Koala/Eucalyptus/Tea.pm
       #       depends on = blib/Test/Koala/Tree.pm
       #
       # Kangaroo_on_a_pogo_stick

By the way, that would create in the TAP model object a hash of attributes such as: 

       {key1}          = "$Revision: 1.3 $";
       {test version}  = "$Revision: 1.3 $";
       {depends on}    = ["blib/Test/Koala/Tea.pm",
                          "blib/Test/Koala/Eucalyptus/Tea.pm",
                          "blib/Test/Koala/Tree.pm"
                        ];

Multi-line values can be declared as in Config::Std by using the key/value separator, which can be either '=' or ':', as the first character on subsequent lines: 

       # TAP ATTR << Kangaroo_on_a_pogo_stick
       #
       # multi-line    = First line.
       #               = Second line.
       #               = And so on.
       #
       # Kangaroo_on_a_pogo_stick

Lists can also be constructed by reusing the same key. 

       # TAP ATTR << Kangaroo_on_a_pogo_stick
       #
       # key2 = First value.
       # key2 = Second value.
       # key2 = Third value.
       #
       # Kangaroo_on_a_pogo_stick

And finally, nested comments are easy, though cluttered in appearance. 

       # TAP ATTR << Kangaroo_on_a_pogo_stick
       #
       # # Is it really true that Koala's do not directly drink water?
       # # I heard that they get all the moisture they need just from 
       # # eating Eucalyptus leaves.
       # source of water = Eucalyptus
       #
       # Kangaroo_on_a_pogo_stick

One area where this general scheme will not support my old dependency system are indirect declarations. My old system allowed both implicit and explicit dependencies. If DEPENDS_ON had nothing before it it was the test itself that was implicitly dependent. 

       # DEPENDS_ON blib/Test/Koala/Tea.pm

Explicit, and likely indirect dependencies could be conveyed with a leading file: 

       # gold_data.txt DEPENDS_ON script/generate_sample_data.pl

Even with the [Section] syntax I discarded as overkill for TAP attributes earlier, I don't see a clean way to define what is essentially a hash of hashes. Instead I expect the downstream processor would need to inject their own syntax and their own parsing: 

       # TAP ATTR depends on = gold_data.txt => script/generate_sample_data.pl


Questions:

When you say "allow tests to declare attributes" do you mean test as in individual ok/not ok tests - or test as in the output of a series of tests with a plan?

Retrieved from "http://testanything.org/wiki/index.php/Test_Attributes"
Personal tools