WWP Internal Markup Documentation (new)

From Digital Scholarship Group
Jump to navigation Jump to search

Note: This page is about the new P5-based internal markup documentation that has been gradually updated from the old version (see WWPmarkupdoc (old) for details of that system and the planning process for updating it). See also the Internal Documentation Update (old) page for details about that conversion/update process, as well as information about what remains to be done in the new system.

This page is primarily intended to provide details about the components and functions of the new single-page internal documentation web app, and to provide instructions on how to modify or update those components when needed.

The page describing the old internal markup system is now a child of this page. Here's a convenient link to it: WWP Internal Markup Documentation (old)

Overview

The new (summer/fall 2012) internal documentation interface is built as a single-page web application consisting of two primary parts:

  1. client-side code: HTML, CSS, and JavaScript and served by Apache just like any other files in the WWP web hierarchy
  2. server-side code/data: XML, XQuery, and XSLT stored and served by eXist

While the server-side files are stored by, and served from, our eXist database instance, all of them also live on disk in the normal WWP web hierarchy and are under Subversion control. As a general rule, these should be considered the canonical version of the files, and any time they are modified they should be re-loaded into eXist. In the long term, we can automate this process via a cron job, but for the time being updated files will need to be added to eXist manually.

Client-Side Code

Code structure

A very simple directory structure is in place for the client-side code. Inside of the main application directory (currently ../internal/, eventually ../encoding/ or whatever else we choose) it looks like this: 

internal/
  index.html
  src/
    css/
      wwpdoc.css
    js/
      wwpdoc.js
      lib/
        backbone-0.9.1.min.js
        jquery-1.7.1.min.js
        underscore.min.js

HTML

The HTML component of the documentation interface consists of a single HTML template file, index.html. Its sole purpose is to provide a minimal HTML page structure and load the CSS/JavaScript for the page.

Following standard practice for optimizing script loading, all of the <script> tags appear at the end of the <body> element, after all other content on the page.

CSS

A single CSS file defines styles for the page: wwpdoc.css (in src/css/). This is nothing too fancy \-\- it's all pretty standard CSS stuff \-\- so there's not much more to say about it.


JavaScript

Most of the interface work is done in JavaScript, using the Backbone.js framework to provide some code structure, some data binding behaviors, and to handle URL hash changes, and the JQuery library to simplify a lot of common JS tasks. The main JS file is wwpdoc.js; the framework/library files are located in ../src/js/. Complete documentation for the Backbone framework can be found at http://backbonejs.org/ if it's needed. JQuery is documented at http://www.jquery.com. Note that Backbone requires the Underscore library be present, so make sure it is loaded first if you're doing any major reworking of the system.

wwpdoc.js

In general, if you find that you want to modify some aspect of the page display or behavior you'll want to do that only in the wwpdoc.js file. This is where all of the real interface work gets done. It's not particularly complicated, but as with most typical Backbone JS apps/pages it is divided into several MVC-like sections that bear some explaining:

  • a single Router (aka Controller)
  • several Views
  • a single Collection (consisting of individual Models)

Everything is "namespaced" (in the JavaScript sense of using a single global object) under the WWP object.

While there might be better ways to design parts of this JS, and I have no doubt done some slightly idiosyncratic things in putting it together, by and large it's a very simple/standard Backbone app that should be fairly easy to understand for anyone with minimal JS knowledge.

Collection

The application functions with a single collection, WWP.models.Entries. This is simply a collection of individual JS objects returned by a GET request sent to doc2entrylist.xquery on the server. This collection contains all of the data needed to display lists of documentation entries and to enable selection, filtering, etc. The data returned for a single entry will generally look something like this: 

{
  id: "advertisement_narrative",
  keywords: ["concept:advertisement", "concept:back matter", "concept:divisions of the text"],
  head: "Advertisements",
  abstract: "Encoding advertisements for printed works..."
}

That's the only information needed to generate the basic application components.

Sidebar

The Sidebar view (WWP.views.Sidebar) handles the display of the left-hand sidebar, which contains lists of documentation Narratives, Elements, and Keywords. When the view is rendered, it iterates through all models in the Collection and outputs the appropriate form select fields. There are a couple of methods defined for selecting an individual entry from one of the three panels (either by mouse or by keyboard), as well as a method to reset the selection and view entire collection.

List

The List view (WWP.views.List) handles the display of a list of entries based upon the user's selection. When the page first loads, this is the view that presents a listing of all documentation entries. When a particular type of entry has been selected from the sidebar (e.g. all Narratives), the showType() method is used to filter the list. Similarly, the showKeyword() method displays all entries that are associated with a given keyword. As with the Sidebar view, the List view's *render()* method does most of the work of generating and inserting the appropriate HTML for every item in the Collection.

Entry

The Entry view (WWP.views.Entry) displays the full text of a single documentation entry. It is displayed only after the user has chosen to view a particular entry (either by clicking on an internal link within another entry, or by selecting an entry from the Sidebar view). The Entry view departs a little from the typical Backbone MVC approach in that, for convenience, it requests the full text of an entry directly from eXist.

The Entry view also handles navigation between individual entries by listening for "click" events on a variety of elements that can appear in an entry: keywords, attribute names or values, element names, etc. A "click" event registered on any of these will trigger the view's select() method, which communicates with the Router (see below) to update the URL hash and load new content as needed.

The HTML that contains the full text of the entry itself is transformed via XSLT in eXist; the Entry view simply receives the generated HTML when it requests it from the server, and then injects it into view's wrapper element.

Router

The Router (WWP.control.Workspace) defines several routes that are fired when the browser's URL hash changes to match a particular pattern; these invoke different application views that respond accordingly. In this case, the application defines separate routes for the default page, an individual entry view, a list of one or more entries, and a keyword view. When any one of these routes it matched, the Router may perform some basic cleanup (e.g. hiding a view that is no longer needed, or clearing out stale information) and then hand things off to the appropriate view.

The Router also handles basic application initialization when the page first loads, setting up the Collection and all of the views. It also defines an application-wide Event Dispatcher (just a clone of the Backbone.Events object) that is used to coordinate a single event across multiple views (e.g. a "click" event in the Sidebar will trigger a corresponding action in the List view).

Templates

With the exception of the Entry view, all of the HTML content that appears on the page is generated by templates defined at the top of wwpdoc.js. These use the Underscore template method to generate snippets of HTML that can be inserted into the DOM dynamically. In general, if you want to change some small aspect of the HTML that appears on the page \-\- for instance, add a class attribute to a specific element so that it can be styled with CSS, or perhaps change the way that the entry list gets displayed \-\- this is the place to do it.

Note, however, that if you want to modify the output of a full entry you'll have to modify the XSLT in eXist.

JS Frameworks and Libraries

Backbone.js, Underscore.js, and JQuery are the three required framework/library components. All three of these live in the ../src/js/lib/ directory. They are loaded via the main *index.html* page, so if you end up needing to swap in a new version you'll want to update the appropriate <script> tag there.

Server-Side Code

Everything on the server side is stored in the eXist database. We're currently running version 1.4, which is installed in /opt/local/eXist/. Our eXist setup is proxied, via Apache Web Server, behind /exist/ (e.g. http://www.wwp.brown.edu/exist/). It _should_ be configured to start/stop automatically on both Teller and Penn; so far, it seems to be working properly. If the doc page stops working at some point, the first thing to check should be whether eXist is running on the server in question.

If you need to start/stop the eXist service manually (which you shouldn't need to do in most cases), use: 

sudo -u exist /etc/init.d/exist [start|stop]

There is a special exist user set up on both Teller and Penn, and this should always be used to run eXist. For security reasons, do not run eXist as the root user.

Structure

The basic eXist structure for the internal documentation looks like this: 

wwpdoc/
  app/
    doc2entrylist.xquery
    doc2html.xquery
    entry2html.xsl
  schema/
    [assorted schema files]
  xml/
    [the documentation XML files]

In general, if you want to make permanent changes to the generated HTML for an entry or to the kind of information about all entries that gets included in the initial JSON response (for instance, if at some point we want to include additional metadata used for filtering the results), you would modify one of the three files listed below.

All of these files are under Subversion control within the normal WWP web hierarchy, so as a general practice any changes that you make should occur and be checked in there, THEN loaded into eXist. Changes made directly in eXist will NOT be reflected in the Subversion repository.

doc2entrylist.xquery

Reads the full wwpdoc/xml/ collection and returns a JSON structure with basic information about the documentation files (see above, under "Collection" for an example of the information contained in that structure).

When the page first loads, the results of this XQuery populate the application's collection and are then used to generate the entire visible page, including the sidebar region and the full entry list.

doc2html.xquery

A very simple XQuery that takes a single request parameter (the name of a file in the xml/ collection) and then transforms it with the entry2html.xsl stylesheet. The resulting HTML is what will be displayed when a single entry is being viewed.

entry2html.xsl

The XSLT that transforms a single XML file into an HTML chunk that can be inserted into the page when requested. This is nothing special, and Syd could no doubt do a better job, but it gets the job done for the time being.

Loading/Updating documentation XML

The XML files for the documentation are under Subversion control, and live in the WWP's normal web hierarchy. Because the documentation interface pulls data from eXist, modifying these files on disk will not result in any changes to the documentation visible to the reader. In order to update that, you'll need to add or update the XML files that are stored in eXist. This will need to be done on both Teller and Penn if you want to changes to be visible to the world.

The easiest (?) way to load files into eXist is to use the Java Admin Client:

  1. Log in to an X11 session on wwp-test or wwp.
  2. Make sure eXist is running (if not, use sudo /opt/local/eXist/bin/startup.sh).
  3. Make sure you are in the /opt/local/eXist/ directory (this may not be necessary, depending on your path is set up — but that seems to vary)
  4. Issue the following command:
java -jar ./start.jar
  1. The Java Admin Client should launch in a separate window. When it does, log in as the admin user. You shouldn't need to change any of the other settings in the login dialog.
  2. Once you have logged in, go to the wwpdoc/xml/* collection. This is where the XML files are stored.
  3. To load new (or updated) XML files, select File > Store files/directories from the top menu (or use the Ctrl-S keyboard shortcut). This will launch a file chooser. Navigate to the directory on wwp-test (or wwp) where the XML files are stored, select the file(s) you want to upload, and then click Select files or directories to store.

7) That's it! Once the upload is complete, the new XML files will be used to generate the documentation visible in the documentation interface.

Retrieved from "https://dsg.northeastern.edu/wiki/index.php?title=WWP_Internal_Markup_Documentation_(new)&oldid=1326"