Difference between revisions of "AmiGO 2 Manual: Installation (2.4.x)"

From GO Wiki
Jump to navigation Jump to search
(No difference)

Revision as of 19:39, 20 February 2015

Preliminaries

These instructions are intended to illustrate how to install the AmiGO 2 client software on a Ubuntu 14.04 LTS. With careful reading, these instructions should be generalizable to other cases. The installation of this client software only covers AmiGO 2 and some Apache 2 configuration.

For information on the data server that AmiGO 2 uses (covering the installation of and loading using the server software: GOlr schema for Solr, OWLTools, etc.), please see the Installation section of the GOlr documentation.

Handling permissions and the like for the various servers is not explicitly covered in this document, but I'd like to make the observation that AmiGO 2 and client stuff can be handled as any user except for the necessary Apache 2 installation (when not using dev deployment directly) and the Solr/GOlr.

Packages

In general, package managers are the preferred way of getting libraries onto a system in a consistent, reversible, and secure manner. However, often you'll find yourself in a situation where you'll need to use CPAN because your preferred distribution or system does not have a robust set of perl libraries.

For users who need to use CPAN, but don't want to mess too deeply with their system (CPAN can turn into a nightmare when installing globally), we'd suggest using CPAN's local::lib and just putting that into your development and run paths: https://metacpan.org/pod/local::lib#The-bootstrapping-technique .

Ubuntu/Debian

Also, in addition to anything that you might need for BBOP JS (in the optional case that you are going to develop with it), the following packages are needed:

sudo apt-get -u install git make rhino kwalify yui-compressor naturaldocs libcgi-application-perl libcgi-application-plugin-session-perl libcgi-application-plugin-tt-perl libclone-perl libconfig-yaml-perl libdbi-perl libdbd-sqlite3-perl libdbd-mysql-perl libdata-formvalidator-perl libossp-uuid-perl libfile-type-perl libfile-slurp-perl libfreezethaw-perl libgraph-perl libgraphviz-perl libjson-xs-perl liburi-perl libwww-mechanize-perl libxml-libxml-perl libxml-xpath-perl

The package libsql-tokenizer-perl is also required (by GOOSE for query checking), but may no longer be available in repositories. If you're interested in using a locally configured GOOSE, you may need to manually install this package along the lines of:

sudo apt-get -u install dh-make-perl
cd /tmp
dh-make-perl --build --cpan SQL::Tokenizer
sudo dpkg -i libsql-tokenizer-perl_???_all.deb

Depending on what you're doing with CPAN in this case, you may also need to install a helper library first:

sudo apt-get -u install liblocal-lib-perl

CentOS/RedHat example

An example comparative table of packages installed between Ubuntu and CentOS; in this case, the perl packages were manually installed rather than using CPAN (see CPAN note above).

Ubuntu CentOS install type
git git yum install
kwalify kwalify gem install
libcgi-application-perl CGI-Application-4.50 CGI-Application-Standard-Config-1.01 CGI-Application-Plugin-ErrorPage-1.21 CGI-Application-Plugin-Redirect-1.00 rpm -i
libcgi-application-plugin-session-perl CGI-Application-Plugin-Session-1.03 rpm -i
libcgi-application-plugin-tt-perl CGI-Application-Plugin-TT-1.05 rpm -i
libclone-perl Clone-0.34 rpm -i
libconfig-yaml-perl Config-YAML-1.42 rpm -i
libdata-formvalidator-perl Data-FormValidator-4.81 rpm -i
libdbd-mysql-perl DBD-mysql-4.023 rpm -i
libdbd-sqlite3-perl DBD-SQLite-1.39 rpm -i
libdbi-perl DBI-1.628 rpm -i
libfile-slurp-perl File-Slurp-9999.19 rpm -i
libfile-type-perl File-Type-0.22 rpm -i
libfreezethaw-perl FreezeThaw-0.5001 rpm -i
libgraph-perl Graph-0.96 rpm -i
libgraphviz-perl GraphViz-2.14 rpm -i
libjson-xs-perl JSON-XS-2.34 rpm -i
libossp-uuid-perl uuid-1.6.2.tar.gz rpm -i
liburi-perl URI-1.60 rpm -i
libwww-mechanize-perl WWW-Mechanize-1.72 rpm -i
libxml-xpath-perl XML-XPath-1.13 rpm -i
make make-3.81-20.el6.x86_64 yum install
naturaldocs NaturalDocs-1.52.zip See Note [1]
rhino rhino-1.7-0.7.r2.2.el6.noarch yum install
rsync rsync-3.0.6-9.el6.x86_64 yum install
yui-compressor yuicompressor-master See Note [2]
[not defined] jline yum install
[not defined] jpackage-utils yum install

Note [1]: NaturalDocs is a set of perl modules that is not distributed or installed in the standard fashion. It is available from http://www.naturaldocs.org/ and is distributed as a .zip file. The zip file does not create a subdirectory, so it simply extracts the files into the current directory. It's probably best to create a directory first (in our case, we usr /usr/local/naturaldocs), copy the zip file into the directory, then unzip the zip file there.

Note [2]: yui-compressor is an Ubuntu-specific shell script front-end to the Java based yuicompressor tool. There is no equivalent script in the RHEL/CentOS distributions, but the AmiGO 2 tools require its use. We have written a home-grown script for RHEL/CentOS to substitute for the Ubuntu version. This script is installed in /usr/bin/yui-compressor. The following is the script contents:

#!/bin/sh
# Quick and dirty front-end script to the yui-compressor that emulates
# /usr/bin/yuicompressor script from Ubuntu release
#
JAVA_HOME=/usr/java/default
YUI_JAR=/usr/local/yuicompressor/build/yuicompressor-2.4.7.jar
# Run YUI Compressor
/usr/bin/java -jar $YUI_JAR "$@"

AmiGO 2: Client software installation

The AmiGO 2 client software can be used completely separately from the server software as long as a Solr server with a compliant schema is available somewhere.

AmiGO 2 no longer depends on an additional download of BBOP JS, it now will use the bundled copy by default. Most users can skip the next paragraph.

For widget developers, since most of the JavaScript in AmiGO 2 relies on this library, the first step in installing AmiGO 2 is to get BBOP JS somehow available. While AmiGO 2 repository (below) comes with a packaged version of bbop.js in the _data/ directory (which is also the default when installing AmiGO 2), if you're interested in hacking around BBOP JS in conjunction with AmiGO 2, you'll have to build it yourself and change the BBOP_JS variable in the Makefile to the appropriate location so you can use the "make refresh" command. Since we don't want to repeat ourselves here, full downloading and building instructions can be found on its homepage at https://github.com/kltm/bbop-js . Scroll down a bit to "Local (building)" to get to the pertinent information.

AmiGO 2 code

This software contains the GO-specific things that BBOP JS does not support--the client (web server) code for AmiGO 2. For the sake of convenience (and using a similar directory layout to the developers), we'll get it like:

mkdir -p ~/local/src/git
cd ~/local/src/git
git clone git://github.com/kltm/amigo.git
cd amigo

Configuration

Next, we have to configure the AmiGO 2 installation given what we have with our BBOP JS and Apache 2 installations. The file to edit is:

conf/amigo.yaml

If you do not have a conf/amigo.yaml file, you can start from the default template with:

cp conf/.initial_values.yaml conf/amigo.yaml

The most important (and fiddly) part of installing AmiGO 2 is this configuration file. For our case, in a local testing environment, the resulting amigo.yaml might look like the following. Please note that:

  • variables names have occasionally been changed, so this might not be exactly like your output
  • we are using a random local IP address for testing--you will have to change that
  • we are using the username foo
  • for the time being we're using a remote GOlr server--again you'll want to setup your own and change this

The configuration file for our test setup looks like:

---
AMIGO_1X_PUBLIC_CGI_BASE_URL:
  comment: The public URL of an available AmiGO 1.x instance for link references.
  type: url
  value: http://amigo1.geneontology.org/cgi-bin/amigo
AMIGO_BALANCER:
  comment: 'Mark this AmiGO 2 installation as behind a load balanacer or similar system. The practical effect of this flag is that if AmiGO detects *.error files (usually created by the global-message.pl script), it will return a 503 page signalling any upstream system that the instance is not currently ready.'
  type: boolean
  value: 0
AMIGO_BETA:
  comment: Mark the AmiGO site as BETA (triggers some changes in variables and installed features).
  type: boolean
  value: 1
AMIGO_DYNAMIC_PATH:
  comment: The full path to the location of the perl binaries.
  type: directory
  value: /home/foo/local/src/git/amigo/perl/bin
AMIGO_DYNAMIC_URL:
  comment: The URL that AMIGO_DYNAMIC_PATH maps to.
  type: url
  value: http://localhost:9999
AMIGO_GOOGLE_ANALYTICS_ID:
  comment: Google analytics ID. Optional.
  type: string
  value: 
AMIGO_LAYOUT_SEARCH:
  comment: A whitespace separated list of class IDs to use for live search tabs on the search page.
  type: string
  value: annotation ontology bioentity
AMIGO_PUBLIC_GOLR_URL:
  comment: The public URL of the GOlr server (including slash).
  type: url
  value: http://golr.berkeleybop.org/
AMIGO_ROOT:
  comment: 'Unless you are doing something strange, the parent directory of this file.'
  type: directory
  value: /home/foo/local/src/git/amigo
AMIGO_SERVER_NAME:
  comment: 'The name by which your AmiGO instance wishes to known--think load balancer or cache situations. If you would rather this information not be reported, leave blank.'
  type: string
  value: localhost
AMIGO_STATIC_PATH:
  comment: The full path to a usable static document directory where static and staging are located.
  type: directory
  value: /home/foo/local/src/git/amigo/static
AMIGO_STATIC_URL:
  comment: The URL for the static content. Can be local or remote.
  type: url
  value: http://localhost:9999/static
AMIGO_TERM_REGEXP:
  comment: The regular expressions to identify internal ontology ids.
  type: string
  value: 'all|GO:[0-9]{7}'
AMIGO_VERBOSE:
  comment: Log verbose AmiGO debugging messages.
  type: boolean
  value: 1
AMIGO_WORKING_PATH:
  comment: Please enter the full path to readable/writable directory that will be used for things like temporary files and logs (if enabled).
  type: directory
  value: /tmp
GOLR_TIMESTAMP_LOCATION:
  comment: 'This is the full path to the timestamp file that is optionally produced by OWLTools during GOlr loading. It contains information about download attempt times for the different loaded files. If not found or unusable, AmiGO 2 will gracefully ignore it.'
  type: file
  value: /home/foo/local/src/git/amigo/golr_timestamp.log
GO_BBOPJS_LOCATION:
  comment: 'If this is not redefined to a new location (e.g. the BBOP-JS staging/ directory, it will use the bbop.js bundle packaged with AmiGO 2 (the one used in the unit tests) in the external/ directory.'
  type: file
  value: /home/foo/local/src/git/amigo/external/bbop.js
GO_YAML_LOCATION:
  comment: 'The full path to the YAML config files directory. In that directory, all files like *config.yaml will be scanned for use.'
  type: directory
  value: /home/foo/local/src/git/amigo/metadata

If everything is correct, installing a dev instance AmiGO 2 should be as simple as:

make install

This will create configuration and amigo2 JavasScript packages, as well as run unit tests and create API docs.

Further more, there will be variations for whether we are developing on the embedded server (all perl) or onto an Apache 2 CGI setup. Both of these are given different subsections.

Development server (embedded perl)

To run the local embedded development server, you will have to add the following package:

sudo apt-get -u install libcgi-application-server-perl

Running should now be as simple as a:

make run

The dev server should now be running on port :9999 (given the amigo.yaml above).

Production server (Apache 2)

While we are not officially supporting any particular web server or configuration, we make available here some notes about setting up a server on a new installation.

AmiGO 2 was developed with Apache 2 in mind, but I'd suspect that any CGI-capable web server would be sufficient. Configuring and running a web server is outside the scope of this document.

First, we need to install the Apache 2 web server:

sudo apt-get -u install apache2

If you are going to use the suggested ModMacro, you'll also need:

sudo apt-get -u install libapache2-mod-macro
sudo a2enmod macro

Great, now we need to get Apache setup to aim at our source/deployment directory. We have a template to work with using conf/amigo2-apache.conf.

sudo cp conf/amigo2-apache.conf /etc/apache2/sites-available/

This is easily changed by changing sjcarbon to foo. Now start the server.

sudo a2ensite amigo2-apache
sudo a2enmod rewrite
sudo a2enmod cgi
sudo service apache2 restart

You may be able to check the server now at: http://localhost:9999

You may run into permission issues for newer versions of Apache 2 like

Forbidden

You don't have permission to access /amigo on this server.

In this case, please see these documents:

Afterwards

System scripts

You may want to check out various AmiGO 2 system scripts referred to in the message at the end of the installation process: global-message.pl, clean-filesystem.pl, and blank-kvetch.pl. More information can be found in AmiGO_2_Manual:_Administration.

All Glory to the Makefile

For some time now, all day-to-day development, handling, distribution, and loading of AmiGO_2 and GOlr have been handled with the Makefiles in amigo/Makefile and amigo/golr/Makefile respectively. If it cannot be done by changing your environment variables and rerunning the target that you want, you might want to reconsider what you're doing.

While the Makefiles are not yet fully documented (will add them to our NaturalDocs real soon now), they may be worth perusing. This is issue: https://github.com/kltm/amigo/issues/26

Images

A KVM image of Ubuntu 12.04.2 of the above configuration (created 2013-07-24) for the AmiGO 2 fronted (only) is available at "http://api.berkeleybop.org/images/"; the name of the image is "a2.img". Just paste those together (I don't want to have the direct link here). The md5sum is 6d60b045fc38fdd8d98222367a506c51.

  • it is an 8GB image, which should have no non-AmiGO 2 services or software
  • hostname: amigo
  • core username: amigo
  • pass: ogima
  • in conf/amigo.yaml, the wired the address is 192.168.122.213 for testing--you'll need to change that
  • the GOlr server is the current public one--you'll need to create your own backend and change it
  • ServerName is not set for Apache.