One of the charges of the CNXML Spec working group is to devise a way to produce the finished, consumable specification from the canonical Relax NG schema insofar as is possible, perhaps in a way analogous to the production of Java documentation from commented source-code via javadoc. Such an approach has an obvious attraction: the documentation for an element is kept in close proximity to the definition of the element in the schema, which makes it easier for maintainers to keep the spec in harmony with the language itself. Furthermore, the schema definition provides enough information to permit the description of attributes and child elements (and their number and order) to be automatically generated.

Sounds like a useful idea. But what are our options in implementing it?

Some background

(1) Relax NG permits the embedding of well-formed elements from other namespaces just about anywhere in the schema. Parsers that validate Relax NG will simply ignore any elements not defined by Relax NG (see this section of Eric van der Vlist's book on RelaxNG). Let's take as an example a simplified version of the RNG definition of CNXML's para element, with the foreign CNXML elements in green, and the RNG elements in light blue:

<element name="para">
  <cnx:para id="para-def">The <cnx:code>para</cnx:code>
  element denotes paragraphs in the structure of a CNXML
  document.</cnx:para>
  <attribute name="id"/>
  <oneOrMore>
    <text/>
    <ref name="code"/>
  </oneOrMore>
</element>

(2) For grouping related pattern definitions together, Relax NG supplies a <div> element in its own namespace (see here). For instance, we might choose to document groups of related elements like so:

<div>
<cnx:name>Links to other documents</cnx:name>
<cnx:para id="links-p1">CNXML provides two elements 
for linking to other documents and to other parts 
of the same document: the 'cnxn' tag creates links 
to Connexions modules in the same repository, and 
the 'link' tag creates links to documents that aren't 
CNXML modules.</cnx:para>

  <define name="cnxn">
    <cnx:para id="cnxn-p1">Use the 'cnxn' element to 
    make links to other Connexions modules in the same 
    repository, and to other parts of the same 
    module.</cnx:para>
    <element name="cnxn">
      ...
    </element>
  </define>

  <define name="link">
    <cnx:para id="link-p1">Use the 'link' element to 
    create links to other documents that are not 
    Connexions modules in the same 
    repository.</cnx:para>
    <element name="link">
      ...
    </element>
  </define>
</div>

(3) The litprog tools from the DocBook project can be used to apply the literate programming methodology to Relax NG schemas. According to Eric van der Vlist:

The basic idea of literate programming (or litprog) is to include a snippet of code (or a snippet of schemas in our case) within the documentation, which can be written in any XML format, including XHTML or DocBook. From this single document embedding code in documentation, a couple of XSLT transformations generate a formatted documentation and the source code.

Options

1. The schema lives (fragmented) within the specification, à la literate programming (documentation-centric).
2. All the specification (including introduction, examples, lives within the schema (schema-centric).
3. Some, but not all of the specification lives in the schema, probably with the parts not dealing with elements and element groups in a separate framework document, which is combined at doc build-time with the parts of the spec inside the schema (schema-centric).
   1. Element- and element-group-specific parts of the spec, as well as examples, live in the schema; everything else lives in the spec framework document.
   2. Element- and element-group-specific parts of the spec live in the schema; examples live in a separate file; everything else lives in the spec framework document.
   3. Element-specific parts of the spec live in the schema; examples live in a separate file; everything else (including element-group-specific parts of the spec) lives in the spec framework document.

If we select an approach that results in a cluttered-seeming schema (say, 2 or 3.b), we can generate not only the specification from the schema and framework docs, but also an uncluttered version of the schema to stow in /usr/share/xml (or wherever). So we would be generating not only the specification but also the schema used in validation from our source docs.

My Preference

My own preference is for something like 3.a. or 3.b., exercising the option to extract a lightly-documented schema from the schema cum spec. I'm somewhat inclined to want the examples in a separate file, so that one doesn't have to operate on the schema to add or modify the examples. One might object that we now have part of the spec stored apart from the canonical schema, thus obviating the chief advantage of our approach. In response I would say that, if we follow our goal of making all future versions of CNXML backwards-compatible with CNXML 0.5, then we will chiefly be adding new examples in response to changes in the schema, rather than deleting or modifying existing examples.

I prefer to avoid the literate programming model, since I think I would find it unnatural to have to maintain a fragmentary schema.

So the pipeline for producing the specification would look something like this: The elements of the specification living in the schema (and optionall the examples in the examples file) would be extracted and stuffed into the framework document as CNXML; XSLT transformations could be applied to produce XHTML and PDF output (or WAP/WAML output--I mean, who wouldn't want to read the CNXML spec on his cell phone? :-).

A lightly-annotated schema could be extracted via XSLT from the schema cum spec for deployment to the system XML directory.

Discussion?

So, use the comments feature to tell me what you think about how we should approach the integration of schema and specification.