Recreating Playground

From Digital Scholarship Group
Jump to navigation Jump to search

Installing CentOS and WWO onto playground

This page is notes for re-creating the WWO environment from RH 6.10 (wwp.neu.edu) to CentOS 8.2 (playground). It will serve as a guide to start, and then hopefully a useful description for future similar upgrades.

CentOS

Installing CentOS was essentially trivial. We used the CentOS-8.2.2004-x86_64-boot.iso.

It appears that yum has been superceded by dnf.

Things we should or have installed

  • Emacs — installed 2020-11-30 with sudo yum install emacs-nox.x86_64
  • Emacs/nxml — was automagically installed by above (and has nice colors! :-)
  • make — installed 2020-11-30 with sudo yum install make.x86_64
  • Subversion — installed 2020-11-30 with sudo yum install subversion.x86_64
  • libxml2/xmllint — libxml2-2.9.7-7.el8.x86_64 is already installed
  • libxslt/xsltproc — installed 2020-11-30 with sudo yum install libxslt.x86_64
  • ant — intsalled 2020-11-30 with sudo yum install ant.noarch
  • screen — nope, Red Hat recommends tmux instead — installed 2020-11-30 with sudo yum install tmux.x86_64
  • Tomcat — no yum package found
  • git — installed 2020-11-30 with sudo yum install git.x86_64 (installed 46 dependencies, mostly Perl packages)
  • eXist — no yum package found, so see detailed description below
  • MariaDB (maybe? maybe not)
  • CouchDB (maybe? maybe not)
  • lessc (pkg = node-less) — no yum package found
    • Ash checked, they recommend installing via npm
    • npm — installed 2020-12-02 with sudo dnf install npm.x86_64
    • installed lessc with sudo npm install less -g
  • Perl — v 5.26.3 already installed
    • HOWEVER, does not have DBI package, so …
    • # dnf --enablerepo=PowerTools install perl-Digest-SHA1
    • # dnf install perl-Apache-DBI
    • # dnf install perl-DBD-mysql
  • PHP — installed 2020-11-30 with sudo yum install php.x86_64
  • Python — I do not know if we want version 3.6 or 3.8 (or 2.7, which I doubt); see the RHEL8 doc
  • Awstats —not found
  • locate — already installed, but need to generate and cronify udpatedb; done 2020-11-30, I think. Ran sudo /usr/bin/updatedb and now locate seems to work. Symlinked /usr/bin/updatedb to /etc/cron.daily/updatedb. Hope that works. I think we check by loooking at timestamp of /var/lib/mlocate/mlocate.db.
  • gcc — installed 2020-12-02 with sudo dnf install gcc.x86_64
  • apxs — installed 2021-02-23 with sudo dnf install httpd-devel
  • libtool — installed 2021-02-23 with sudo dnf install libtool

Apache setup, modules, etc.

We (Syd & Ash) worked on configuring the web server for awhile, but were still having problems. Ernesto (user=nestochan) gave us a hand, performed some firewall magic, and now it is working. Need to find out what he did.

  • server side includes
  • whatever we need for security and textbaseIP stuff
    • mod_jk
    • ✓ /etc/httpd/passwd/
  • redirection stuff (mostly part of configuration)
  • probably want to check several files in wwp-test:/etc/httpd/ into version control; consider:
    • conf.d/webalizer.conf
    • conf.d/limitipconn.conf
    • conf.d/reqtimeout.conf
    • conf.d/wwp-blog.conf
    • conf.d/wwp.conf-ssl.conf
    • conf.d/mod_security.conf
    • conf.d/dsc-wiki.conf
    • conf.d/dsg-wiki.conf
    • modsecurity.d

files from /etc/httd/ whose differences between wwp and wwp-test need to be investigated

  • Files wwp-test/httpd/conf/httpd.conf and wwp/httpd/conf/httpd.conf differ
  • Files wwp-test/httpd/conf.d/awstats.conf and wwp/httpd/conf.d/awstats.conf differ
  • Only in wwp-test/httpd/conf.d: dsc-wiki.conf
  • Only in wwp-test/httpd/conf.d: dsg-wiki.conf
  • Files wwp-test/httpd/conf.d/limitipconn.conf and wwp/httpd/conf.d/limitipconn.conf differ
  • Files wwp-test/httpd/conf.d/mod_security.conf and wwp/httpd/conf.d/mod_security.conf differ
  • Files wwp-test/httpd/conf.d/redirects.conf and wwp/httpd/conf.d/redirects.conf differ
  • Files wwp-test/httpd/conf.d/ssl.conf and wwp/httpd/conf.d/ssl.conf differ
  • Only in wwp-test/httpd/conf.d: useless
  • Files wwp-test/httpd/conf.d/wwp-blog.conf and wwp/httpd/conf.d/wwp-blog.conf differ
  • Files wwp-test/httpd/conf.d/wwp.conf and wwp/httpd/conf.d/wwp.conf differ
  • Files wwp-test/httpd/conf.d/wwp.conf-ssl.conf and wwp/httpd/conf.d/wwp.conf-ssl.conf differ
  • Files wwp-test/httpd/conf.d/wwprd.conf and wwp/httpd/conf.d/wwprd.conf differ
  • Files wwp-test/httpd/conf.d/wwprewrite.conf and wwp/httpd/conf.d/wwprewrite.conf differ

other things to be moved over

  • WWO
  • WWP website
    • including configuration files
    • include git repos per /var/www/html/WWP/lab/README-WWP
  • eXist applications (see WWP_eXist-DB_applications)
  • either Awstats or something better
    • remember, /usr/local/bin/rgxg is used by Awstats — installed 2021-04-07 by downloading the 0.1.2 release tarball, ./configure, make, make install.
  • WordPress
  • /usr/local/bin/wwpwwwup.bash

things to be moved only to wwp-test

  • MediaWiki
  • CouchDB and the data produced for Cultures of Reception

things we need help with

  • MediaWiki (Karl)
  • WordPress (Karl)
  • https: access part of Apache config (Ernesto)
  • documenting what was done to firewall (Ernesto)
  • mod_jk?

Version control plan

Our plan is to keep all the Apache2 configuration stuff (/etc/httpd/) under Subversion on playground, wwp-test, local machines, etc.; BUT not on production (wwp) itself. To update that machine you will have to copy files over by hand. This is in part to help reduce chance of messing things up with an svn up and in part as an acknowledgement that a) karl & Ernesto work in this dir, too, and b) some of the files actually have to be different (due to name of host being different, i.e. wwp vs wwp-test vs 129.10.107.233).

eXist-DB

wget the installation JAR from GitHub. WWP is currently using eXist v5.2.0. In v5.x, the configurations below are mostly the same, except most of the files are now stored in $EXIST_HOME/etc. 

useradd --home-dir /opt/local/eXistDB --create-home --system existdb
chmod 0775 /usr/local/eXistDB
sudo su - existdb

Add environmental variables to .bash_profile: 

export EXIST_HOME=/opt/local/eXistDB/eXist-current
export RUN_AS_USER=existdb

Then, in the terminal: 

source ~/.bash_profile
chmod a+x /tmp/exist-installer-5.2.0.jar
sudo su - existdb
java -jar /tmp/exist-installer-5.2.0.jar -console

As part of the install process, you'll need to do some configuration of the database. Use these:

  • Install path: /opt/local/eXistDB/eXist-VERSION
  • Data directory: default is okay
  • Administrator password: set it rather than leaving it blank!
  • Applications: default is okay
  • Do not create any shortcuts

Create a symlink so that eXist-current ($EXIST_HOME) points to the latest version of eXist: ln -s eXist-VERSION eXist-current.

Navigate into $EXIST_HOME.

Jetty settings

Change the eXist port number, since 8080 is already in use on the WWP servers.

  • Create a backup of $EXIST_HOME/etc/jetty/jetty-http.xml.
  • Change the HTTP port from 8080 to 8088.

Increase the maximum file size accepted for upload. (The default is 200,000.)

  • Create a backup of $EXIST_HOME/etc/jetty/webapps/exist-webapp-context.xml
  • Add a new property as a child of <Configure>:
<Set name="maxFormContentSize">400000</Set>

Webapp settings

Keep guest users from executing arbitrary XQueries. See Mathias Göbel's 2017-03-11 post on Exist-open, "get parameter _query = open proxy".

  • Create a backup of $EXIST_HOME/etc/webapp/WEB-INF/web.xml.
  • Set //init-param[param-name = ('xquery-submission', 'xupdate-submission')] to "authenticated".

Database configuration

  • Create a backup of $EXIST_HOME/etc/conf.xml
  • Set //indexer/@preserve-whitespace-mixed-content to "yes".
  • Set //serializer/@indent to "no".
  • Disable execution of arbitrary XQueries: set //builtin-modules/module[@uri eq 'http://exist-db.org/xquery/util']/parameter[@name eq 'evalDisabled']/@value to "true".
  • Add Saxon settings to //transformer[@class eq 'net.sf.saxon.TransformerFactoryImpl']:
<attribute name="http://saxon.sf.net/feature/recoveryPolicyName"
           value="recoverWithWarnings"
           type="string"/>
<attribute name="http://saxon.sf.net/feature/strip-whitespace"
           value="none"
           type="string"/>

At this point, if you can log into the eXist-DB Dashboard from the web, install the eXist-DB webapps. When the persistent scheduler has been installed, return to conf.xml and add this job to //scheduler: 

        <!-- Reschedule XQuery jobs after a restart. -->
        <job type="user" name="persistent-scheduler" 
             xquery="/db/apps/persistent-scheduler/rescheduler.xq"
             period="5000" repeat="0" />

The next time eXist starts up, the changes made in all these files will be applied.

Install eXist as a service

eXist v5.x doesn't require a wrapper at all. Use the instructions in the eXist documentation to set up a service that will run the database on reboot. Once done, you will be able to start and stop eXist using systemctl.

Progress Notes

November 2019

“playground” is the name Ashley & Syd give to the box they are using for experimentations. (Currently for web stat packages.) This page describes a procedure for installing a CentOS (i.e., RedHat clone) OS on it quickly.

  • Remember to back up iptables and restore it
  • We re-set the clock with instructions from a website we found.


2021-02-03

We have the apache2 web server up and running; loading http://129.10.107.233/WWP/ works in that data is loaded. However SSIs are not working. (Tomcat and mod_jk are not working yet either, but we did not expect them to.)

Next step: Figure out how to configure locally so that SSIs work. We tried moving in the old /etc/httpd/conf/httpd.conf file into the new environment and the entire web server failed to start up.


2021-02-08

We copied the Apache2 configuration files from the SVN trunk (read: following WWP Test configuration). Restarting the server failed due to missing modules in /etc/httpd/modules. That path is a symlink to ../../usr/lib64/httpd/modules, and the contents differ between Playground and Test.

We will need to figure out how to install some of the missing modules:

  • mod_limitipconn or its equivalent in Apache 2.4
  • mod_perl

We may need mod_fcgid (FastCGI). We don’t think we need mod_auth_mysql, mod_autoindex, or mod_wsgi (Python).


2021-02-09

We discovered that Playground had a date of 2018-07, and reset it to the correct date with sudo date --set. That also allowed us to install mod_limitipconn and mod_perl, since those failed due to a problem.

Continuing with our Apache configuration test, we found that we need an SSL certificate in order to fully test the server setup (for this wiki, for example).

Ernesto installed the SSL certificate from WWPDev, so we can continue working on Apache configuration with apachectl configtest.

We also created a symlink from /var/lib/tomcat9 (correct?) to the /usr/local/share/apache-tomcat.../webapps directory. Tomcat still needs to be set up as a service to run on server restart.


2021-02-23

We’ve installed the Apache module mod_jk from source (mod_jk connects the server to Tomcat). We found and followed instructions here, though we couldn’t start httpd in step one. The JkMount directives now seem to work with the new mod_jk.so file installed at /etc/httpd/local-modules.

We fixed the paths to the SSL certificate provided by Ernesto.

There are some warnings remaining, and expected paths and/or permissions that are missing in our setup.


2021-02-26

We continued debugging our Apache2 configuration. We discovered that our newly-compiled mod_jk.so module had to have the same SELinux policy as httpd. We used the chcon command, with the "reference" argument to copy the policy from httpd to the module.

We also created the /var/log/apache2 directory, and added READMEs in that directory and /var/log/httpd, explaining what should be stored in each.

After these changes, the Apache configuration syntax came back OK.

The httpd service still fails to start, however, and there don’t seem to be clues in the command line output of [?].

We checked the /var/log/apache2/error_log, discovered debugging and info messages about the security certificate. We updated the VirtualHost names to match playground’s, which resulted in a warning in the error log: the server name does not match the names in the security certificate. We expected this, and it really shouldn’t affect the server starting up.

Next step is to discover why Apache still won’t run.


2021-03-02

On the thought that the reason the webserver will not start might be because of the attempt to load modules multiple times, we removed all duplicate LoadModule calls. To see what it was before we removed these, check out 42719 or later, but before 42805.

The good news is we managed to remove all the duplicate module warning messages. The bad news is that (kinda as expected) it did not change the fact that the webserver will not start, nor the (kinda useless) error messages in the log file.


2021-03-09

We were able to fix the SSL server certificate warnings by changing wwp.conf-ssl.conf to use wwp-test server names instead of playground's.

Still no luck on getting Apache to run. The terminal output just says that the startup process failed, and /var/log/apache2/error_log now only has info and debug messages. We asked Ernesto for help.

2021-03-16T10/11

Trying to get PHP (in particular PHP7) to work with the Apache2 install. Follwed some instructions on this website, to wit (mostly requiring sudo):

  1. dnf install dnf-utils http://rpms.remirepo.net/enterprise/remi-release-8.rpm
  2. dnf module reset php
  3. dnf module enable php:remi-7.4
  4. dnf install php
  5. restarted apache w/o error
  6. PHP still did not work
  7. commented out the <IfModule> in php.conf
  8. ... and it did not work. We got an error message on restarting webserver: Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe. You need to recompile PHP, which leads us to believe that either we need to install prefork.c (whatever that is), or we need to recompile PHP (however one does that).


2021-03-16T14/15

Tried lotsa stuff, but eventually found that replacing what we had as /etc/httpd/conf.d/php.conf with what we has as /etc/httpd/conf.d/php.conf.rpmnew worked, in that the web server starts and loads our front page with the “Upcoming Events” looking correct, not like an excerpt of raw PHP code.


2021-03-18

Removed FastCGI Apache module loader and config files, since our PHP fix doesn't require it.

Tomcat

Looks like we never finished installing & configuring Tomcat, as it is not talking to Apache. There are instructions at both the Tomcat doc and at a lovely gist page Ash found. So far we have created a tomcat user with sudo useradd --home-dir /usr/local/share/apache-tomcat-9.0.40 --system tomcat.


2021-03-23

We finished setting up the Tomcat 9 systemd daemon, using the gist page linked above as a starting point. The working script is available in /etc/systemd/system/tomcat9.service.

To get systemd to start Tomcat, we needed to go into the Tomcat directory (/usr/local/share/apache-tomcat-9.0.40/) and give ownership of everything to the Tomcat user and group. To get Tomcat to successfully run, we needed to additionally change permissions on the directories symlinked in the Tomcat home directory.

To test that Tomcat is running, we used the command sudo systemctl status tomcat9.service, and also a wget localhost:8080 to make sure Tomcat was serving out webpages. However, Tomcat’s port is not available outside Playground; our next step is to go back to Apache2 and make sure it’s configured to allow access to Tomcat, or at least the XTF servlet.


2021-03-29

We finally found what we think was stopping Apache2 from talking to Tomcat9 — in file /usr/local/share/apache-tomcat-9.0.40/conf/server.xml we found the bit defining port 8009 as the ajp13 contact point was commented out, and uncommented it. Still doesn’t work.

Start next time by looking to see what is going on with the redirect. Is 129.10.107.233/WWO getting forwarded to right place?


2021-03-30

We figured out that we had adjusted the URLs for our two Virtual Hosts in /etc/httpd/conf.d/wwp.conf, but skipped the same definitions in the SSL configuration file. (Side note: do we still need the full plain-HTTP configuration file? Ask Ernesto.) We adjusted the values in the SSL configuration to match the regular one. This didn’t help us with being able to view Tomcat.

We weren’t sure whether we had the server names and aliases right for the Apache2 virtual hosts, especially for Tomcat (WWO). To see if we could access Tomcat from the web at all, we set the WWP virtual host to a port that doesn’t exist, leaving the WWO host as the only one configured. However, we still couldn’t access Tomcat in a browser. We would be prompted for authentication, but the connection to [playground.wwp.northeastern.edu/WWO /WWO] hung indefinitely. The Tomcat logs had not been touched; Apache did not connect to it.

we found a Perl error in /var/log/apache2/error_log: Can't locate object method "remote_ip" via package "Apache2::Connection" at /etc/httpd/Apache2/IPAccess.pm line 277.. We found that remote_ip had been removed in Apache 2.4, and the API changelog suggested we replace it with either request_rec->useragent_ip or conn_rec->client_ip. We chose the former. The new line: my $remote_addr = $r->request_rec->useragent_ip ();.


2021-04-05

After restarting Apache2, we got a new error at the same line: [perl:error] [pid 1217232:tid 139646902204160] [client 10.15.249.129:57712] Can't locate object method "request_rec" via package "Apache2::RequestRec" at /etc/httpd/Apache2/IPAccess.pm line 277.\n. Ash found an explanation of the request_rec object, which suggests that a number of different Apache2 modules contribute to request_rec. Figuring it was extraneous, Ash deleted request_rec-> in favor of $r->useragent_ip ();.

IP address checking now appears to work. There are a number of Perl warnings and one error, but these have to do with the contents of /etc/httpd/passwd/textbase_ipaddrs. IPAccess.pm seems to be able to recover from this. There is also a Perl warning from Apache2::IPAccessLog1 that an expected MySQL table is not accessible.

Still no access to WWO. These warnings aren’t likely to be causing our problems with the Apache2 virtual hosts. Probably best to move on, maybe look at the configured redirects.

2021-04-06

After bumbling around for an hour, Ash figured out we had not followed all of the instructions for setting up mod_jk. Once we finished configuring Tomcat, we got a better message in the log file. Following that, we found we had to set the secretRequired parameter on the <Connector> for AJP/1.3 in /usr/local/share/apache-tomcat-9.0.40/conf/server.xml.

2021-04-23

Ash had discovered that eXist died because the system had erased a file in /tmp/. We played with /usr/lib/tmpfiles.d/tmp.conf to try to get it to stop, but were unsuccesful in that on reboot /tmp/ did not exist.

2021-04-27

Found that `chronyd` was not running on boot (and system clock was set to 2018-06-22). Ash enabled it with systemctl enable chronyd.service, and that seems to have worked.

Then we tweaked /usr/lib/tmpfiles.d/tmp.conf some more, and we think we have the /tmp/ erasure problem licked. Will test on 2021-05-08 or 09.

2021-05-02

Installed awstats 7.8 (using awstats-7.8-1.noarch.rpm via dnf). Re-created config dirs by creating then checking out Subversion repository https://liblab.neu.edu/svn/DSG/wwp/webstats/branches/RHEL8/. Made the new /usr/local/awstats/ dir group writable by group wheel. Changed all /usr/share/awstats/ to /usr/local/awstats/ both in this new dir and in web config files. Made symlink in /etc/cron.daily/ to the run-em-all pgm. Created new /mnt/WWPplayground-Data/ and told that pgm to use it. Note that SiteDomain= is not being set correctly (but it isn’t on wwp-test, either). Current problem is that web config doesn’t seem to know about usage/stats pages.


Setting up a new RHEL 8 server

Starting on nb9565, 2021-06-10.

Step-by-step

dzdo su - root
dnf install tmux
dnf install emacs
# Syd switched to emacs here
dnf install make subversion ant git npm php gcc httpd-devel libtool
# libxml (xmllint) and libxslt (xsltproc) already installed
subscription-manager repos --enable "codeready-builder-for-rhel-8-$(arch)-rpms"
dnf install perl-Digest-SHA1
# perl-DBD-mysql already installed
wget https://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/e/epel-release-8-10.el8.noarch.rpm
rpm -Uvh epel-release*rpm
dnf install perl-Apache-DBI
npm install less -g

Now install our Apache2 configuration: 

systemctl stop httpd
cd /etc
mv httpd httpd_orig && svn checkout https://liblab.neu.edu/svn/DSG/wwp/apache2/branches/RHEL8 httpd
mkdir /var/log/apache2
# get our clever README files:
cd /var/log/apache2/
ln -s /etc/httpd/WWP_docs/var_log_apache2_INFO.txt ./README.txt
cd /etc/httpd/logs/
ln -s /etc/httpd/WWP_docs/etc_httpd_logs_INFO.txt  ./README.txt

Install missing Apache2 modules, including mod_perl, mod_limitipconn: 

# apachectl configtest
dnf install mod_perl mod_limitipconn mod_qos mod_security mod_ssl

Install mod_jk from source: 

cd /usr/local/src
wget https://mirrors.ocf.berkeley.edu/apache/tomcat/tomcat-connectors/jk/tomcat-connectors-1.2.48-src.tar.gz
tar -xvf tomcat-connectors-1.2.48-src.tar.gz
cd tomcat-connectors-1.2.48-src/native
./configure -with-apxs=/usr/bin/apxs
make
cp apache-2.0/mod_jk.so /etc/httpd/local-modules

Get locate to work properly (just duplicating what we did on playground, above): 

# time /usr/bin/updatedb
# cd /etc/cron.daily/
# ln -s /usr/bin/updatedb updatedb

Install rgxg by following cmds above, and then also sudo chmod a+w /usr/local/src/rgxg-0.1.2/libs. Install and configure Awstats: 

dnf install awstats
mv /usr/share/awstats /usr/local/
cd /usr/local/awstats
mv tools tools-orig
svn checkout https://liblab.neu.edu/svn/DSG/wwp/webstats/branches/RHEL8/ tools
chgrp -R wheel .
chmod -R g+w .
rm -R /etc/awstats/*
cd /etc/awstats
sudo ln -s /usr/local/awstats/tools/awstats.ALL.conf awstats.conf

Note that we updated the code in /usr/local/awstats/tools/generateAndRunAllConfFiles.bash (on Subversion’s RHEL8 branch) to generate the needed symlink to index.php. Then we copied over the website (from wwp-test.neu.edu): 

time tar czpf - WWP | (ssh nb9565.neu.edu "cd /var/www/html/; tar xzpf -")

Hope that works.

In fact, it worked too well, and /var threatened to crash the server. Ernesto deleted /var/www/html and mounted an NFS directory in its place — the same NFS storage used on WWP Test. This gives us the WWP website as it appears on WWP Test for free. We had to create a “wwp” group at the GID used on Test. The nb9565 SVN version (1.10) can’t make any changes to the website directory, due to an incompatible SVN repository format.

For now, we plan to make updates to that working copy on Test only, and run svn upgrade after throwing the switch.

Ernesto and Karl installed the SSL certificate that Apache requested. Apache still wouldn’t start, as it could not bind to Playground’s IP address. Updated references to the server IP and server name in Apache configs:

  • /etc/httpd/conf/httpd.conf: "Listen", "ServerName"
  • conf.d/wwp.conf: "VirtualHost", "ServerName", "ServerAlias", "ServerAdmin", "Redirect"
  • conf.d/wwp.conf-ssl.conf: "VirtualHost", "ServerName", "ServerAlias", "ServerAdmin"
  • conf.d/wwprewrite.conf: RewriteRules "^/WWO/php/wAll\.php", "^/WWP/WWO", "^texts/"

With these changes, Apache syntax is OK, and Apache starts up, but the website is inaccessible outside of nb9565. Some of this is firewall-related. There are also indications that the security certificate and the Apache config aren’t meshing well (curl complains when calling nb9565 from itself). We consulted Ernesto and Karl.

In the meantime, we started installing Tomcat from source. We downloaded a tarball for version 9.0.50 to /usr/local/src, and extracted the directory to /usr/local/share.

It turned out that the firewall problem was due to an ITS backlog. Ernesto contacted them and got them to fix it. The firewalls on the two new servers now match the settings of the older servers. With the new firewall rules in place, nb9565 now works in the browser!

eXist-DB set-up

Done after the firewall was updated. The #eXist-DB section above has more details on hows and whys.

Installation

cd /tmp
wget https://github.com/eXist-db/exist/releases/download/eXist-5.3.0/exist-installer-5.3.0.jar
# user root
dzdo su - root
mkdir /opt/local
useradd --home-dir /opt/local/eXistDB --create-home --system existdb
chmod 0775 /opt/local/eXistDB
chmod a+x /tmp/exist-installer-5.3.0.jar 
# user existdb
su - existdb
vi .bash_profile
# Add:
#   export EXIST_HOME=/opt/local/eXistDB/current
#   export RUN_AS_USER=existdb
source .bash_profile
java -jar /tmp/exist-installer-5.3.0.jar -console

Use the installation notes above. 

ln -s eXist-5.3.0 current
cd $EXIST_HOME/etc
cp jetty/jetty-http.xml jetty/jetty-http.xml.orig
vi jetty/jetty-http.xml
# Change port 8080 to 8088
cp jetty/webapps/exist-webapp-context.xml jetty/webapps/exist-webapp-context.xml.orig
vi jetty/webapps/exist-webapp-context.xml

Add property to increase max upload size 

cp webapp/WEB-INF/web.xml webapp/WEB-INF/web.xml.orig
vi webapp/WEB-INF/web.xml.orig

Make sure XQueries can only be executed by authenticated users. 

cp conf.xml conf.xml.orig
vi conf.xml

Fill in listed conf.xml settings. 

exit
# user root
vi /etc/systemd/system/exist-db.service

Used the sample service file in the eXist documentation as a starting point. 

chown existdb /etc/systemd/system/exist-db.service && chgrp existdb /etc/systemd/system/exist-db.service
systemctl start exist-db
rm /tmp/exist-installer-5.3.0.jar

Transfer data

eXist v5.2.0 (on WWP Test) is binary-compatible with v5.3.0 (on nb9565). Because of this, transferring eXist data to the new server was fairly simple:

  1. On WWP Test, assume the existdb user identity (you -> root -> existdb)
  2. Zip up the eXist-db data directory and put it somewhere other users can see it.
  3. scp the directory to nb9565, /tmp
  4. On nb9565, shut down the eXist daemon (if necessary).
  5. Assume the existdb user identity.
  6. Rename the existing data directory.
  7. unzip /tmp/DATADIR.zip -d eXist-5.3.0
  8. As root, start the eXist daemon again.

eXist’s documentation suggests that these steps alone should be sufficient, but in fact, the web Dashboard was blank, though I could log in.

This may not have done anything to help, but it didn’t hurt anything. In eXide, repair the package repository by executing this script: 

xquery version "3.1";

import module namespace repair="http://exist-db.org/xquery/repo/repair" 
at "resource:org/exist/xquery/modules/expathrepo/repair.xql";

repair:clean-all(),
repair:repair()

The release notes for eXist v5.3.0 say that, when upgrading from v5.2.0 or earlier, the eXist Dashboard must be reinstalled at a higher version. In eXide, execute this script: 

xquery version "3.1";

repo:remove("http://exist-db.org/apps/existdb-packageservice"),
repo:install-and-deploy("http://exist-db.org/apps/existdb-packageservice", 
    "http://exist-db.org/exist/apps/public-repo/find")

The eXist Dashboard is now fully functional.

The Women Writers in Review app has been upgraded to v1.8.3, which gets around a bug in xml-to-json().

Tomcat

Previously we had downloaded Tomcat and put it /usr/local/share/.

Created a tomcat user with sudo useradd --home-dir /usr/local/share/apache-tomcat-9.0.50 --system tomcat.

Installed WWO into /usr/local/share/apache-tomcate-9.0.50/webapps/ (just an svn co).

Copied the daemon file playground:/etc/systemd/system/tomcat9.service into the same directory on new system. Also changed the path in that file from “.40” to “.50”.

Also made same “.40” to “.50” change to /etc/httpd/misc/workers.properties file.

And uncommented the <Connector statement for AJP/1.3 on line ~119 of /usr/local/share/apache-tomcat-9.0.50/conf/server.xml

Note: We find that there is both an error and two warnings from the apachectl configtest command. Here is the pin to fix the warnings sometime later: 

[Thu Aug 12 10:22:30.939891 2021] [so:warn] [pid 3280953:tid 139889576147264] AH01574: module limitipconn_module is already loaded, skipping
[Thu Aug 12 10:22:30.946045 2021] [so:warn] [pid 3280953:tid 139889576147264] AH01574: module ssl_module is already loaded, skipping

We also need to deal with the error, which is line 39 of /etc/httpd/conf.d/mod_security.conf not being able to write its file out.

We made the symlinks to logs, temp, and work from the tomcat-9.0.50/ directory.

Started Tomcat. No errors! Might even be working, we can’t tell yet due to redirect problems.

NEXT: Fix redirects. Probably in /etc/httpd/conf.d/wwprewrite.conf.


2021-08-13

We tried a few things, to no avail.

  • Added entries in our own local /etc/hosts/ for nb9565 (and wwo.nb9565)
  • Tweaked wwprewrite.conf
  • Made changes to all files in /etc/httpd/conf.d/ with sudo perl -p -i -e 's,\bwwp-test?\.(neu|northeastern),nb9565.neu,g;' *

So far we are unable to figure out why website works, but redirecting to Tomcat does not seem to. (And certainly Tomcat is not serving up our stuff.)

2021-08-16

Index WWO texts in XTF.

In /etc/httpd/conf.d/wwprewrite.conf, all redirects to WWO had to be prefixed with the base server address (https://wwo.nb9565.neu.edu, which isn't actually available through DNS; modify your /etc/hosts).

Add secretRequired="false" to the AJP <Connector> in TOMCAT/conf/server.xml.

Restart Tomcat aaaaand got it.

2021-08-19/20

1. Note-to-selves (which is a bit weird, this whole page is notes-to-selves): we should go through /mnt/WWPdev-Data/backups_of_excerpts_from_RHEL6_wwp-test_system/usr_local.tgz and var_lib.tgz someday and weed out a lot of it, as much of it is not needed even as a backup.

2. Left for tomorrow: backup or selectively backup /var/lib/, /var/svn/, /var/log/; selectively backup /var/www/ — turns out there is nothing interesting in /var/www/ except html/, and that is already a shared volume, no reason to backup.

Retrieved from "https://dsg.northeastern.edu/wiki/index.php?title=Recreating_Playground&oldid=3828"