Difference between revisions of "WWO Publishing"

From Digital Scholarship Group
Jump to navigation Jump to search
(first crack at publication page)
 
Line 8: Line 8:
  
 
# <tt>$ cd /path/to/source/dir/probably/distribution/</tt>
 
# <tt>$ cd /path/to/source/dir/probably/distribution/</tt>
# <tt>$ make wf</tt> &nbsp; &nbsp; # Test well-formedness; '''all must pass''' except that in some cases duplicate IDs might be tolerable. Still, a very bad idea. Fix it.
+
# <tt>$ make wf</tt> &nbsp; &nbsp; # Test well-formedness. You don’t really have to do this, as it is performed by Process step 1, below. But finding errors earlier is always better, '''all must pass''' except that in some cases duplicate IDs might be tolerable. Still, a very bad idea. Fix it.
 
# Issue either <tt>make validate</tt> OR <tt>make both</tt> — they do the same thing, just in different order. ("validate" tests each file against RELAX NG, then tests each file against Schematron; "both" tests each file first against RELAX NG and then against Schematron.)
 
# Issue either <tt>make validate</tt> OR <tt>make both</tt> — they do the same thing, just in different order. ("validate" tests each file against RELAX NG, then tests each file against Schematron; "both" tests each file first against RELAX NG and then against Schematron.)
 +
 +
There may be lots of validity errors, of course. Figuring out which ones are important and which should be ingored is left as an exercise to the reader.
 +
 +
=== Process Data ===
 +
 +
The data we generate (e.g., in <tt>distribution/</tt>) is the canonical data we want to store. (Hence the name of the schema: “wwp-store”.) It is not quite ready to be published by XTF in that state. For example, XIncludes have to be processed and the <tt>&lt;charDecl></tt> has to be deleted (due to a Java bug, but we don’t use it in XTF, anyway). Pretty much one call to <tt>make</tt> does all this, but it can be a pain to set up the directories properly for it.
 +
 +
# <tt>$ cd /path/to/source/dir/probably/distribution/</tt> &nbsp; &nbsp; # you might already be there, of course
 
# <tt>$ make WWO</tt> &nbsp; &nbsp; # N.B. that you will likely have to specify some paths to needed directories on the commandline; issue just <tt>make</tt> for further info. The output goes to <tt>/var/lib/tomcat8/webapps/WWO/data/wwp/</tt> by default, but this is easily changed by setting XTF= on the commandline; e.g. <tt>make XTF=/var/lib/tomcat6/webapps/WWOtesting WWO</tt>. Note that if you are in a directory named <tt>distribution/</tt>, this command will also go build emerson.almanack.xml.
 
# <tt>$ make WWO</tt> &nbsp; &nbsp; # N.B. that you will likely have to specify some paths to needed directories on the commandline; issue just <tt>make</tt> for further info. The output goes to <tt>/var/lib/tomcat8/webapps/WWO/data/wwp/</tt> by default, but this is easily changed by setting XTF= on the commandline; e.g. <tt>make XTF=/var/lib/tomcat6/webapps/WWOtesting WWO</tt>. Note that if you are in a directory named <tt>distribution/</tt>, this command will also go build emerson.almanack.xml.
 +
 +
=== Generate XTF Index ===
 +
 
# Take the output of <tt>make WWO</tt> and put it in the <tt>data/wwp/</tt> directory of the XTF installation where you want to build the index. (Depending on what machine you are on and what directories you specified, above, this may already be done.) Replace old data. Note that our entire XTF directory is under Subversion control, so an easy way to do this is to check the newly created files in wherever you created them and check them out where you want to index them.
 
# Take the output of <tt>make WWO</tt> and put it in the <tt>data/wwp/</tt> directory of the XTF installation where you want to build the index. (Depending on what machine you are on and what directories you specified, above, this may already be done.) Replace old data. Note that our entire XTF directory is under Subversion control, so an easy way to do this is to check the newly created files in wherever you created them and check them out where you want to index them.
 
# <tt>$ cd /var/lib/tomcat6/webapps/WWO/</tt> &nbsp; &nbsp; &nbsp; &nbsp;# on indexing machine
 
# <tt>$ cd /var/lib/tomcat6/webapps/WWO/</tt> &nbsp; &nbsp; &nbsp; &nbsp;# on indexing machine
# <tt>$ tar xzf index index_PREV_''DATE''.tgz</tt> # create a backup, as not under Subversion control
+
# <tt>$ tar xzf index index_PREV_''DATE''.tgz</tt> &nbsp; &nbsp; &nbsp; &nbsp; #create a backup, as not under Subversion control
 
# <tt>$ cd /var/lib/tomcat6/webapps/WWO/bin/</tt> &nbsp; &nbsp;# could have been just cd bin/
 
# <tt>$ cd /var/lib/tomcat6/webapps/WWO/bin/</tt> &nbsp; &nbsp;# could have been just cd bin/
 
# <tt>$ ./textIndexer -clean -index default</tt> &nbsp; &nbsp; # I actually preface with `time`
 
# <tt>$ ./textIndexer -clean -index default</tt> &nbsp; &nbsp; # I actually preface with `time`
 
# <tt>$ sudo chgrp -R tomcat ../index/ &nbsp;&& &nbsp;sudo chmod -R g+w ../index/
 
# <tt>$ sudo chgrp -R tomcat ../index/ &nbsp;&& &nbsp;sudo chmod -R g+w ../index/

Revision as of 13:29, 5 December 2018

WWO Publishing

Testing

Before publishing it is easiest to move all files you want to publish into one directory (typically distribution/, but more honest to leave the newest ones in on_deck until you have actually published. The process is the same either way, it just means you have to perform some of the steps twice.

And remember to remove any files you don’t want published from the directories you are playing with. (Only files of the glob format “*.*.xml” will be processed.)

  1. $ cd /path/to/source/dir/probably/distribution/
  2. $ make wf     # Test well-formedness. You don’t really have to do this, as it is performed by Process step 1, below. But finding errors earlier is always better, all must pass except that in some cases duplicate IDs might be tolerable. Still, a very bad idea. Fix it.
  3. Issue either make validate OR make both — they do the same thing, just in different order. ("validate" tests each file against RELAX NG, then tests each file against Schematron; "both" tests each file first against RELAX NG and then against Schematron.)

There may be lots of validity errors, of course. Figuring out which ones are important and which should be ingored is left as an exercise to the reader.

Process Data

The data we generate (e.g., in distribution/) is the canonical data we want to store. (Hence the name of the schema: “wwp-store”.) It is not quite ready to be published by XTF in that state. For example, XIncludes have to be processed and the <charDecl> has to be deleted (due to a Java bug, but we don’t use it in XTF, anyway). Pretty much one call to make does all this, but it can be a pain to set up the directories properly for it.

  1. $ cd /path/to/source/dir/probably/distribution/     # you might already be there, of course
  2. $ make WWO     # N.B. that you will likely have to specify some paths to needed directories on the commandline; issue just make for further info. The output goes to /var/lib/tomcat8/webapps/WWO/data/wwp/ by default, but this is easily changed by setting XTF= on the commandline; e.g. make XTF=/var/lib/tomcat6/webapps/WWOtesting WWO. Note that if you are in a directory named distribution/, this command will also go build emerson.almanack.xml.

Generate XTF Index

  1. Take the output of make WWO and put it in the data/wwp/ directory of the XTF installation where you want to build the index. (Depending on what machine you are on and what directories you specified, above, this may already be done.) Replace old data. Note that our entire XTF directory is under Subversion control, so an easy way to do this is to check the newly created files in wherever you created them and check them out where you want to index them.
  2. $ cd /var/lib/tomcat6/webapps/WWO/        # on indexing machine
  3. $ tar xzf index index_PREV_DATE.tgz         #create a backup, as not under Subversion control
  4. $ cd /var/lib/tomcat6/webapps/WWO/bin/    # could have been just cd bin/
  5. $ ./textIndexer -clean -index default     # I actually preface with `time`
  6. $ sudo chgrp -R tomcat ../index/  &&  sudo chmod -R g+w ../index/