Test Suite - next steps

[Hausenblas, Michael]

All,
 
After some discussion and some contemplation [1] and [2],
I'd like to propose the following resolutions w.r.t. the
RDFa Test Suite:
 
1. We avoid the term 'conformance testing' at all
   (the former CQ is now called Test Query);
 
2. The TC manifest [3] and the content of [4] are the _only_
   normative parts of the RDFa Test Suite, collectively referred
   to as the 'RDFa Test Cases' (TC);
 
3. The Test Suite document [5] serves as an informative document
   that highlights the use of the TC;
 
4. New TC can be introduced at any time, their source (mail, IRC, etc.)
   is recorded (cf. [6]);
 
5. Section 4 in the Test Suite [5] will be removed; a link to the
   TC manifest [3] along with an appropriate XSLT that provides for a human
   readable (HTML) view replaces this. Note: This enables a more dynamic
   handling of TC;
 
6. The Test Runner [7] will be completely removed from the TS
   and serves as a supplemental, only.
 
Comments welcome; hope DanC can agree ;)
 
Wing: IIRC we've already discussed the usage of an XSLT to
      produce the HTML from the manifest, haven't we?
 
Cheers,
       Michael
 
[1] http://lists.w3.org/Archives/Public/www-qa/2007Mar/0004.html
[2] http://www.w3.org/2007/03/05-rdfa-minutes#item03
[3] http://www.w3.org/2006/07/SWD/RDFa/testsuite/testcases/rdfa-test-manifest.rdf <http://www.w3.org/2006/07/SWD/RDFa/testsuite/testcases/rdfa-test-manifest.rdf>
[4] http://www.w3.org/2006/07/SWD/RDFa/testsuite/testcases/ <http://www.w3.org/2006/07/SWD/RDFa/testsuite/testcases/>
[5] http://www.w3.org/2006/07/SWD/RDFa/testsuite
[6] http://www.w3.org/2007/03/05-rdfa-minutes.html#action11
[7] http://www.w3.org/2006/07/SWD/RDFa/testsuite/run-tc.html
 
----------------------------------------------------------
 Michael Hausenblas, MSc.
 Institute of Information Systems & Information Management
 JOANNEUM RESEARCH Forschungsgesellschaft mbH
 Steyrergasse 17, A-8010 Graz, AUSTRIA
----------------------------------------------------------

[Ben Adida-2]

Hausenblas, Michael wrote:
> All,
>  
> After some discussion and some contemplation [1] and [2],
> I'd like to propose the following resolutions w.r.t. the
> RDFa Test Suite

This sounds very good to me. +1.

-Ben

[Dan Connolly]

Hausenblas, Michael wrote:
> All,
>  
> After some discussion and some contemplation [1] and [2],
> I'd like to propose the following resolutions w.r.t. the
> RDFa Test Suite:
>  
> 1. We avoid the term 'conformance testing' at all
>    (the former CQ is now called Test Query);
>  

That suits my preference, thanks.

I have to admit my case is less than compelling. The meaning of the
term seems to have drifted since I learned it, or something.
> Comments welcome; hope DanC can agree ;)
>  

Again, all of this infrastructure stuff is well and good, I guess, but
what I want most is a handful of fairly complete tests; preferably
tests based on examples that are being advertised widely, e.g.
those in the primer.

--
Dan Connolly, W3C http://www.w3.org/People/Connolly/

[Wing C Yung]

Dan Connolly wrote on 03/05/2007 08:03:12 PM:
> Again, all of this infrastructure stuff is well and good, I guess, but
> what I want most is a handful of fairly complete tests; preferably
> tests based on examples that are being advertised widely, e.g.
> those in the primer.

We are working towards assembling a full test suite. I took the tests
assembled by Elias several months ago and reduced the redudancy of the
tests. Then I added some additional tests to test parts of the syntax
document [1] that weren't covered so that nearly every section has at least
one test. I'm sure that some cases are missing, but it is a start. This is
why Michael and I would appreciate some help in looking over the tests.

I did not take tests from the Primer [2] or the examples section (section
6) of the syntax document because they tend to be longer and test multiple
aspects of the syntax document. My original aim was to have each test test
a single behavior specified by the syntax document. Is this not the correct
way to assemble the test suite?

[1] http://www.w3.org/2006/07/SWD/RDFa/syntax/
[2] http://www.w3.org/2006/07/SWD/RDFa/primer/

Wing

[Dan Connolly]

Wing C Yung wrote:

Dan Connolly wrote on 03/05/2007 08:03:12 PM:
  

Again, all of this infrastructure stuff is well and good, I guess, but
what I want most is a handful of fairly complete tests; preferably
tests based on examples that are being advertised widely, e.g.
those in the primer.
    

We are working towards assembling a full test suite. I took the tests
assembled by Elias several months ago and reduced the redudancy of the
tests. Then I added some additional tests to test parts of the syntax
document [1] that weren't covered so that nearly every section has at least
one test. I'm sure that some cases are missing, but it is a start. This is
why Michael and I would appreciate some help in looking over the tests.
  

I'm happy to take a look at a few test cases.

Starting from
  http://www.w3.org/2006/07/SWD/RDFa/testsuite/testcases/
I find a repository
http://www.w3.org/2006/07/SWD/RDFa/testsuite/

and the first input document in there is

http://www.w3.org/2006/07/SWD/RDFa/testsuite/testcases/000002.htm

It bears a DTD but it's not valid. I still don't see any point in that. I understand
some sort of module is in progress; is there an open issue on that?

At least the MIME type isn't text/plain any more. I get the impression that
some javascript/DOM things only work with application/xhtml+xml ; I think
that's worth tracking as an issue. This test case says that
at least some RDFa documents work with text/html; that seems worth
adding to the RDFa requirements document, for review by the HTML WG.

I'm still concerned that there's no follow-your-nose path from the document to
the RDFa spec. Is there an open issue on that? The current issues list seems
to be http://www.w3.org/2006/07/SWD/track/products/2 ; I don't see this
issue there. This seemed to be acknowledged as an open issue last August
 http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2006Aug/0025
Maybe it was addressed and I missed it. Even so, it seems worth adding
to the issues list post-hoc.

I did not take tests from the Primer [2] or the examples section (section
6) of the syntax document because they tend to be longer and test multiple
aspects of the syntax document. My original aim was to have each test test
a single behavior specified by the syntax document. Is this not the correct
way to assemble the test suite?
  

It's a reasonable way. It's not the only correct way.

I suggest that the examples that are being used to teach people about RDFa should get
the sort of scrutiny that comes from including them in the test suite. But as I said earlier,
there are many reasonable paths to good test coverage.

[1] http://www.w3.org/2006/07/SWD/RDFa/syntax/
[2] http://www.w3.org/2006/07/SWD/RDFa/primer/

Wing
  

p.s. please excuse any odd formatting in this message. I'm new to Thunderbird;
I'd like it to just send plain text, but I can't always get it to do that.