WWO Publishing

WWO Publishing

Testing

Before publishing it is easiest to move all files you want to publish into one directory (typically distribution/, but more honest to leave the newest ones in on_deck until you have actually published. The process is the same either way, it just means you have to perform some of the steps twice.

And remember to remove any files you don’t want published from the directories you are playing with. (Only files of the glob format “*.*.xml” will be processed.)

1. $ cd /path/to/source/dir/probably/distribution/
2. $ make wf     # Test well-formedness. You don’t really have to do this, as it is performed by Process step 1, below. But finding errors earlier is always better, all must pass except that in some cases duplicate IDs might be tolerable. Still, a very bad idea. Fix it.
3. Issue either make validate OR make both — they do the same thing, just in different order. ("validate" tests every file against RELAX NG, then tests every file against Schematron; "both" tests each file first against RELAX NG and then against Schematron.)

There may be lots of validity errors, of course. Figuring out which ones are important and which should be ingored is left as an exercise to the reader.

Process Data

The data we generate (e.g., in distribution/) is the canonical data we want to store. (Hence the name of the schema: “wwp-store”.) It is not quite ready to be published by XTF in that state. For example, XIncludes have to be processed and the <charDecl> has to be deleted (due to a Java bug, but we don’t use it in XTF, anyway). Pretty much one call to make does all this, but it can be a pain to set up the directories properly for it.

1. $ cd /path/to/source/dir/probably/distribution/     # you might already be there, of course
2. $ make WWO     # N.B. that you will likely have to specify some paths to needed directories on the commandline; issue just make for further info. The output goes to /var/lib/tomcat8/webapps/WWO/data/wwp/ by default, but this is easily changed by setting XTF= on the commandline; e.g. make XTF=/var/lib/tomcat6/webapps/WWOtesting WWO. Note that if you are in a directory named distribution/, this command will also go build emerson.almanack.xml.

Generate XTF Index

1. Take the output of make WWO and put it in the data/wwp/ directory of the XTF installation where you want to build the index. (Depending on what machine you are on and what directories you specified, above, this may already be done.) Replace old data. Note that our entire XTF directory is under Subversion control, so an easy way to do this is to check the newly created files in wherever you created them and check them out where you want to index them.
2. $ cd /var/lib/tomcat6/webapps/WWO/        # on indexing machine
3. $ tar xzf index index_PREV_DATE.tgz         # create a backup, as not under Subversion control
4. $ cd /var/lib/tomcat6/webapps/WWO/bin/    # could have been just cd bin/
5. $ ./textIndexer -clean -index default     # Expect this to take a long time, several days. (And you probably want to time it by prefacing with time.) It just took 30.3 hours on my machine at home, which is perhaps twice as fast as wwp-test.
6. $ sudo chgrp -R tomcat ../index/  &&  sudo chmod -R g+w ../index/

If you’re not already on the target machine where you want the index to be, transfer it. (Probably best to make a tarball of it and then transfer that single file, but any mechanism that moves it intact should work fine. I typically move it with scp, first from my desktop to paramedic:/tmp/, and then from there to wwp-test:/var/lib/tomcat6/webapps/WWO/, and then untar it there. It is fine to leave the tarball lying around as a backup (as long as wwp-test has lots of disc space available), it is fine to delete it.

Generate Text Lists

Note: This can be done on any machine on which you have a working copy of https://liblab.neu.edu/svn/DSG/wwp/website/trunk/wwo/texts, but it is somewhat easier to just do it on wwp-test.

1. Log in to wwp-test
2. $ cd /var/www/html/WWP/wwo/texts
3. $ make     # Gives you instructions
4. $ make HOST=www.wwp-test.northeastern.edu all     # Unless you want to do something other than the usual. You can expect hundreds of (inappropriate) warnings that xml-model is an invalid name for a PI target.
5. Do NOT check-in the modified lists. This is the hardest part of this process. The lists are different on wwp and wwp-test; thus we don't want to have the wwp-test ones checked-in just to avoid the problem of someone inadvertently checking them out on wwp.

When you actually want to generate these for wwp you can either

• Use the Perl command in the output of make to change what you have to be suitable for use on wwp, and check it in. (Then put it back if you still want wwp-test to work :-). OR
• Re-generate the lists for wwp, check those in. (Then re-generate for wwp-test if you still want wwp-test to work. :-)