[tap-l] User Supplied YAML Diagnostic Keys: Descriptive Version

[Michael G Schwern]

chromatic wrote:
>>> If TAP v15 adds a
>>> new reserved key, anyone who deliberately upgrades may need to modify
>>> both the producer and consumer to deal with the collision, if that person
>>> even cares.
> 
>> I don't understand.  There can be no collision.  Official TAP keys all
>> start with a lower case letter.  User defined keys don't.  A new, official
>> key cannot collide with anything.
>>
>> Can you provide an example of your scenario?
> 
> 1) Suppose that TAP discards this silly idea of namespacing or prefixing or 
> reserving a character set subset for reserved keys
> 
> 2) Suppose that a DarkPAN TAP producer/consumer pair uses a diagnostic key not 
> reserved by TAP
> 
> 3) Suppose that the next version of TAP uses that diagnostic key to mean 
> something else
> 
> Is this a problem?  No.  As you yourself point out, producer/consumer pairs 
> have to upgrade in tandem to process diagnostic keys in a way that's 
> semantically correct.
 >
> A) If the user does not upgrade to other tools which produce or consume the 
> new version of TAP, there's no problem.
> 
> B) If the user performs the upgrade and upgrades the DarkPAN producer/consumer 
> pair, there's no problem.
> 
> C) If the user performs the upgrade and does not upgrade the producer/consumer 
> pair, the only problem is that some diagnostic information may display 
> incorrectly.  Tests do not fail.  Planes do not fall out of the sky.  You are 
> willing to live with this; you said as much in your previous message.

The stability of user-defined keys is a concern.  The incorrect/misleading 
display of diagnostic information is a problem.  We want people to use them 
without having to worry about being blown over by an official key later on. 
We also do not want to require the producer and consumer to have to upgrade in 
lock-step.

The question is how much readability and user extendability are we willing to 
sacrifice to solve it?  This is a question of magnitude and balance, not a 
binary either/or.

I am willing to sacrifice a small amount of readability and extensibility to 
solve 90% of the problem.  Thus the official vs user distinction.  Solving the 
user vs user clash sacrifices too much for too little (or no) gain.


> Now replace #1.
> 
> 1) Suppose that TAP retains the silly idea of namespacing/prefixing/character 
> set reserving.
> 
> A) If the user does not upgrade, there's no problem.
> 
> B) If the user performs the upgrade and upgrades the DarkPAN producer/consumer 
> pair, there's no problem.
> 
> C) If the user performs the upgrade but does not upgrade the producer/consumer 
> pair, the only problem is that some diagnostic information may display 
> incorrectly (that is, not at all).  Tests do not fail.  Planes do not fall 
> out of the sky.
 >
> Thus the silly idea of namespacing/prefixing/character set reserving only 
> changes the type of the trivial upgrading problem.
> 
> Why?
> 
> Because custom diagnostic keys produced by a custom producer need to be 
> consumed by a custom consumer.  There is a mutual dependency.

Ahh, there's the misunderstanding.

We recommend that the displayer just show any unrecognized key/values rather 
than totally ignore them.  So the user still sees the new information, just in 
a raw form.  Sorry if this wasn't made clear.


>>>> You're either going to wind up with a big list of prefixes and keys,
>>>> which is annoying work, or you're going to break down and match on
>>>> /-time$/ and defeat the point of prefixing.
>>> Every part of customizable diagnostics has this problem.
>> Sorry, I don't understand.  Can you provide an example?
> 
> If the default behavior of a consumer is to ignore unknown keys, then the 
> consumer needs to know which keys it knows.  You're either going to wind up 
> with a big list of keys, which is annoying work, or you're going to... well 
> okay, you're going to wind up with a big list of keys.
> 
> The silly idea of namespacing/prefixing/character set reserving has no bearing 
> on this.

Since all unknown key/values should be displayed by default, there's no need 
to specify a key in the displayer unless you're doing something special with it.

The important distinction is that in the prefixed scheme, if you have a 
behavior for a user-defined key you have to list out each prefixed-key which 
triggers the same behavior.  'Perl.Test.Stuff-time', 'Perl.Test.Things-time', 
'Perl.Test.Wibble-time', and so on... would all have to be listed to trigger 
the same custom behavior and added to each time a new producer starts using 
that key.

With the unprefixed scheme, you just have a one-to-one map of a behavior for 
each key.  Once you determine what "Time" should do you're done.


>> Separating official vs user keys solves a good chunk of the collision
>> problem,
> 
> It may solve the problem where a collision can occur between official keys and 
> a single producer/consumer pair, but that's not actually a problem, and it 
> does nothing to solve the problem of collisions between unofficial keys in 
> multiple producers, which is much likelier to happen.
> 
> Compare the number of TAP producing modules on the CPAN to the number of 
> versions of TAP.  There's an order of magnitude difference.  Now imagine that 
> half or even a quarter of them decide to use diagnostic keys.
> 
> If you really need to solve a collision problem in TAP diagnostics, solve that 
> one.  The official/non-official key collision problem is a red herring.
> 
> I don't know how to put this any more clearly, so I'm content to let this 
> thread die here and watch TAP v15 careen off into crazy town.  (Alternately, 
> I could be the one careening off into crazy town, but at the risk of making 
> an argument from authority, I *have* written three TAP producers still in use 
> today.)

We'll see what happens.  I think encouraging user key convergence is worth the 
risk.  Fortunately, if user-vs-user clashes do become the problem users can 
begin prefixing on their own.  We're not walling that possibility off.


-- 
3. Not allowed to threaten anyone with black magic.
     -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
            http://skippyslist.com/list/