[tap-l] TODO Tests

[Bram]

[This idea was first submitted to p5p... See:  
http://www.nntp.perl.org/group/perl.perl5.porters/2008/05/msg136540.html  
]

While adding some todo tests (for t/op/range.t) I ran into some  
limitations (IMHO).

Assume the following TODO test:

{
   local $TODO = "test that foo() returns 1";
   my $s1 = foo();
   is($s1, "1");
}

At the moment foo() returns 3.


Time passes and code changes.

Now there are 3 options:

foo() returns 1, this will result in 'unexpected todo test passed'  
being outputted;
foo() returns 3, no special output is produced;
foo() returns 4, no special output is produced;

To me the last one seems like a limitation.
The result of foo() is now different then when the TODO test was  
written but there is nothing in the output that indicates this.

(The different output could be (or most likely is) accidental or a  
side effect of another change.)


My proposal would be to add syntax that catches this and produces an  
output like:

Test Summary Report
-------------------
op/range.t (Wstat: 0 Tests: 135 Failed: 0)
   TODO unexpected output:   122 (output 4 - expected 3)
Files=1, Tests=135,  0 wallclock secs ( 0.05 usr  0.00 sys +  0.04  
cusr  0.02 csys =  0.11 CPU)
Result: PASS


The important thing in this is that the result is PASS.
It still is a TODO test and therefore should never produce FAIL.


Regarding the syntax:


David Nikol suggested:

{
   local $TODO = "test that foo() returns 1";
   local $WAS = 3;
   my $s1 = foo();
   is($s1, "1");
}


I initially suggested another syntax but I'm now in favor of David's  
suggestion.


(Obviously this would also be applied to isn't, like, unlike, ....)


Comments? Ideas?

Would a patch that adds this behavior be considered?


Kind regards,

Bram