Schema generation

From Digital Scholarship Group
Jump to navigation Jump to search

Generating Schemas (and Documentation) from ODD

The ODD itself

The source customization ODD file for the wwp-store encoding language is in our textbase Subversion repository in textbase/schema/wwp-store.odd. It should be valid against the current or the development version of tei_odds. It should almost be valid against either the current or the development version of tei_customization. (There are a few “errors”, as that schema is designed to help write the ODD, not validate it.) In all 4 of those cases it should also be valid against the associated Schematron rules.

The edition number is stored on /TEI/teiHeader/fileDesc/editionStmt/edition/@n in major.minor.fix format. Any change to the ODD that represents a deliberate change to the set of documents validated by the schema ups the minor version number. Changes to only prose or class names or such may be reflected in only the fix number. The major number is reserved for overhauls or other very significant changes. (The last one, e.g., was the change from the TEI ODD language to the TEI PureODD language — i.e., purging (most of) the RELAX NG from the ODD.) The edition number is usually also reflected on the @n of the <change> element describing the change that brings the ODD up to the specified version.

The Subversion $Id:$ is also stored as the content of that <edition. In general it should not be updated by hand, but rather by Subversion on check-in. (This requires that the svn:keywords property be set, which it is.)

Currently the <schemaSpec> element is stored in <body>; someday soon I would like to move it to <back>, reserving the body for prose discussion.

The ODD file is loosely divided into “sections” delimited by nothing other than box comments. A list of those sections follows, in the order they currently occur in the ODD file. (But since the order of the specification children of <schemaSpec> is not constrained by TEI and is insignificant to an ODD processor, it may have changed in the ODD with no effect.)

element & module inclusion, and element deletion section
<moduleRef>s with @include or @except. If we had any schemaSpec/elementRef elements they would probably be here, too.
ODD/Schematron hack section
section for top-level <constraintSpec>s. In looking at this now, it should either be re-named or divided into two sections (and renamed).
element renaming section
set of <elementSpec> elements for cases in which the primary change being made is to change the name of the element. Often other changes are made, as well, but they are usually “minor” changes. E.g., the <titlePage> element is changed to the <titleBlock> element here, but also has a controlled vocabulary for its @type established here. (Fnote: this is because you cannot have two <elementSpec> elements with the same @ident and @ns attributes in a single ODD, yet.)
element addition section
new elements we have added. Note that this should be a subsection of the EXTENSIONS section, below.
class deletion section
for deletion of entire classes (“scorched earth” approach for attribute deletion; we do not delete any model classes)
attribute deletion section
for removing attributes from elements or classes (the “selective thinning” and “repossession” approaches to attribute removal)
class addition section
for addition of attribute classes. The model classes are added in the “new class subsection” of the “EXPANSIONS” section, not sure why.
other class and macro manipulation section
changes to datatypes, macros, and classes
section for changes to content models of "normal" TEI elements
in some cases the change is only to attributes, not content model
required element section
a complex Schematron check to require certain elements that are optional in TEI
attribute constraint section
constraining some attributes further than vanilla TEI, e.g. constrained value list or required instead of optional
EXPANSIONS
Changes in this section need to be in a non-TEI namespace. Since our entire markup language is in our own non-TEI namespace, this doesn't really mean much to us. But most such changes are segregated to here in order to make changing this to a TEI-conformant customization a bit easier.
class manipulation subsection
non-conformant changes to classes
new class subsection
added model classes
macro addition section
macros we added (to make writing the ODD easier)
stuff leftover from EMPB that I still need to look at
I guess
miscellaneous section
as it says
XInclude section
the XInclude attributes defined in ODD


Generating Closed Schema

Any of the following methods should work. The only one I have thoroughly tested is the first, using the command fiumicino.bash on my system. It is my local front-end to roma, so I don’t have to type a really long command every time.

commandline roma
$ cd /path/to/textbase/schema/
$ /path/to/roma2 --patternprefix=ws_--xsl=/path/to/TEI_Stylesheets_repo --noxsd --nodtd --localsource=/path/to/local/p5.xml ./wwp-store.odd .
Note that this creates both RELAX NG XML syntax and compact syntax output, wwp-store.rng and wwp-store.rnc (unless you forget the ultimate dot, in which case they get stuck in a RomaResults/ directory).
Note that the patternprefix can be whatever you want, but it makes life easier if it matches the value of schemaSpec/@prefix.
Note that localsource is optional. I think the Stylesheets will default to using the file /path/to/TEI_Stylesheets_repo/source/p5subset.xml, but I’m not 100% on that.
Stylesheets “bin” commands
$ cd /path/to/textbase/schema/
$ /path/to/TEI_Stylesheets_repo/bin/teitornc --odd --localsource=/path/to/TEI_Stylesheets_repo/source/p5subset.xml wwp-store.odd
Note: I don’t know why you have to specify localsource, you shouldn’t have to.
If you want XML syntax instead of compact, you can use teitorelax instead of teitornc.
oXygen
Open schema in oXygen.
configure scenario
Only has to be done the first time
Select Document > Transformation > Configure Transformation Scenario(s)…
Select TEI ODD to RELAX NG
Click Save and close (or Apply associated, which does next step, too)
Select Document > Transformation > Apply Transformation Scenario(s)
Output is placed in /path/to/textbase/schema/out/wwp-store.rnc
OxGarage
Surf over to OxGarage
Click on Documents (as “Convert from ?”)
Click on ODD Document
Click on RELAX NG compact syntax
Back towards top of page, click Browse… and then find and select the wwp-store.odd file
Click Convert
Save ZIP file (goes wherever your browser puts it, probably into ~/Downloads/)
Extract the document.rnc file from the ZIP file and rename it
Roma β
Surf over to Roma beta
Should be self-explanatory
Roma
Surf over to Roma
Select Upload a customization, and click Browse… and select the wwp-store.odd file
Click the red Start button
Click the Schema tab
Select RELAX NG schema (compact syntax) from the “Which format do you prefer?” drop-down (it is typically the default)
Click the red Generate button
Save output file (goes wherever your browser puts it, probably into ~/Downloads/)


There are no changes required to the output RELAX NG except (sometimes) moving & renaming the file. However, the compact syntax output is more human-readable if you run it through my “fixer”: /home/syd/bin/fix_rnc_whitespace.perl --patternprefix="ws_" < wwp-store.rnc > FIXED.rnc && mv FIXED.rnc wwp-store.rnc.

Generating Open Schema

There are fewer choices for generating the open schema (i.e., the ISO Schematron). No matter how you initially generate the file, though, it needs to be post-processed. Currently that post-processing is done by hand. Note that oXygen has no built-in process for doing this. (At least, not yet.) Also our ODD just makes Romaβ hang. (I have not told Raff yet. I probably should, but I know he is very busy teaching a new undergraduate course this semester; I believe this is the first time in his career he has taught a course.)

commandline roma
$ cd /path/to/textbase/schema/
$ /path/to/roma2 --patternprefix=ws_--xsl=/path/to/TEI_Stylesheets_repo --noxsd --nodtd --localsource=/path/to/local/p5.xml --isoschematron ./wwp-store.odd .
Note that this creates wwp-store.isosch (unless you forget the ultimate dot, in which case I think it will get put in a RomaResults/ directory).
Note that localsource is optional. I think the Stylesheets will default to using the file /path/to/TEI_Stylesheets_repo/source/p5subset.xml, but I’m not 100% on that.
Stylesheets “bin” commands
$ cd /path/to/Stylesheets_repo/bin/
$ ./teitoschematron --odd /path/to/textbase/schema/wwp-store.odd /path/to/textbase/schema/wwp-store.isosch
OxGarage
Surf over to OxGarage
Click on Documents (as “Convert from ?”)
Click on ODD Document
Click on ISO Schematron constraints
Back towards top of page, click Browse… and then find and select the wwp-store.odd file
Click Convert
Save ZIP file (goes wherever your browser puts it, probably into ~/Downloads/)
Extract the document.rnc file from the ZIP file and rename it
Roma
Exactly the same process as for the closed schema, except select ISO Schematron from the “Which format do you prefer?” drop-down

post-processing

The generated wwp-store.isosch file is NOT usable as is. This is because the Schematron extraction routine does not realize we use an entirely different namespace, and because one of our Schematron processors chokes on multiple declarations of the same namespace. So open up the file in a text editor and perform the following steps.

  • One easy way to get both the <ns> elements and a comment describing what you did right is to copy them from the previous version of the wwp-store.isosch file. (Referred to as “copy-and-replace” below.) You would want to copy from the line immediately following the “This file generated DATETIME by 'extract-isosch.xsl'” to the “constraints” comment, and replace the equivalent portion in the new file. However, if you generate the changes this way a) you still have to perform the s/tei:/wwp:/g; change, and you should definitely test the schema, at least with oXygen if not probatron. This is because once in a while things do change, and a simple copy is insufficient.
  • If you have not used the copy-and-replace method above, look through the <ns> elements near the top of file. (There are generally 6–10 of them divided into up to two sections, “namesapces, declared” and “namespaces, implicit”.)
    • All but one of any set that have the same `@prefix` but different `@uri`s must be deleted (these are actual Schematron errors). This usually does not occur or occurs only with the one pair, the prefix "tei", which is bound to both the TEI namespace and the WWP namespace. Delete the TEI namespace one. (Keep in mind that "TEI" should remain bound to the TEI namespace.)
    • All but one of any set that have the same `@uri` but different `@prefix`s should be deleted (these are not Schematron errors, but probatron treats them as an error when it shouldn’t). This usually does not occur or occurs only with the one pair that bind the WWP namespace to both "tei" and "wwp". Delete the "tei" one.
    • All but one of any set that have the same `@prefix` and the same `@uri` should be deleted (these are not Schematron errors, but probatron treats them as an error when it shouldn’t). This usually does not occur.
  • The Schematron was extracted from both the TEI Guidelines and from the customization ODD using paths that address elements in the same WWP namespace. The former use the hard-coded prefix tei:, the latter use wwp:. But (because probatron is lame) we just deleted the declaration for the tei: prefix. So now do a global change, changing all occurences of tei: with wwp:. Be sure not to change TEI: (to either wwp: or WWP:, the latter of which is what many editors would do by default).
  • You may wish to format-and-indent or the equivalent just to make file more human-readable.
  • If you did not use the copy-and-replace method above, make sure to add a comment near top-of-file declaring the changes you made.
Retrieved from "https://dsg.northeastern.edu/wiki/index.php?title=Schema_generation&oldid=3675"