AmiGO 2 Manual: Installation (2.3.x): Difference between revisions

From GO Wiki
Jump to navigation Jump to search
m (Sjcarbon moved page AmiGO 2 Manual: Installation to AmiGO 2 Manual: Installation (2.3.x): Need more granularity.)
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
'''This is preliminary documentation for beta software'''
=Preliminaries=


=Preliminaries=
The current released production version is 2.3.x (pending).


These instructions are intended to illustrate how to install the AmiGO 2 client software on a Ubuntu 12.04 LTS. With careful reading, these instructions should be generalizable to other cases. The installation of this client software only covers AmiGO 2 and some Apache 2 configuration.
These instructions are intended to illustrate how to install the AmiGO 2 client software on a Ubuntu 14.04 LTS. With careful reading, these instructions should be generalizable to other cases. The installation of this client software only covers AmiGO 2 and some Apache 2 configuration.


For information on the data server that AmiGO 2 uses (covering the installation of and loading using the server software: GOlr schema for Solr, OWLTools, etc.), please see the ''Installation'' section of the [[GOlr]] documentation.
While the correct version of OWLTools is now included as a JAR with the AmiGO installation, but the [[GOlr]] documentation is still available separately and should be read for proper GOlr/Solr installation.


Handling permissions and the like for the various servers are not explicitly covered in this document, but I'd like to make the observation that the AmiGO 2 and client stuff can be handled as any user as long as they have read write permissions to a htdocs and cgi-bin directory in a properly configured Apache 2 installation and the Solr server is locally accessible.
Handling permissions and the like for the various servers are not explicitly covered in this document, but I'd like to make the observation that the AmiGO 2 and client stuff can be handled as any user as long as they have read write permissions to a htdocs and cgi-bin directory in a properly configured Apache 2 installation and the Solr server is locally accessible. Similarly, most CGI servers should be able to deliver AmiGO.


==Packages==
==Packages==
Line 13: Line 13:
Also, in addition to anything that you might need for BBOP JS (in the optional case that you are going to develop with it), the following packages are needed:
Also, in addition to anything that you might need for BBOP JS (in the optional case that you are going to develop with it), the following packages are needed:


  sudo apt-get -u install make rsync rhino libconfig-yaml-perl liburi-perl libjson-xs-perl libossp-uuid-perl libfile-type-perl libfreezethaw-perl libwww-mechanize-perl libcgi-application-perl libdbi-perl libdata-formvalidator-perl libcgi-application-plugin-session-perl libcgi-application-plugin-tt-perl libgraph-perl libclone-perl libxml-xpath-perl libdbd-sqlite3-perl git yui-compressor naturaldocs libgraphviz-perl libfile-type-perl libfile-slurp-perl libdbd-mysql-perl kwalify
  sudo apt-get -u install make rsync rhino libconfig-yaml-perl liburi-perl libjson-xs-perl libossp-uuid-perl libfile-type-perl libfreezethaw-perl libwww-mechanize-perl libcgi-application-perl libdbi-perl libdata-formvalidator-perl libcgi-application-plugin-session-perl libcgi-application-plugin-tt-perl libgraph-perl libclone-perl libxml-xpath-perl libdbd-sqlite3-perl git yui-compressor naturaldocs libgraphviz-perl libfile-type-perl libfile-slurp-perl libdbd-mysql-perl kwalify libxml-libxml-perl xvfb python-virtualenv g++ node
 
You will need to also get some semi-modern form of Node.js (node and npm) onto your system. AmiGO currently requires 0.10+, with 0.12+ recommended. Options that have worked for us are ppa:chris-lea/node.js and the
online installer: https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions (recommended, although we haven't tried 4+ yet).


The package libsql-tokenizer-perl is also required (by GOOSE for query checking), but may no longer be available in repositories. If you're interested in using a locally configured GOOSE, you may need to manually install this package along the lines of:
The package libsql-tokenizer-perl is also required (by GOOSE for query checking), but may no longer be available in repositories. If you're interested in using a locally configured GOOSE, you may need to manually install this package along the lines of:
Line 105: Line 108:
  mkdir -p ~/local/src/git
  mkdir -p ~/local/src/git
  cd ~/local/src/git
  cd ~/local/src/git
  git clone git://github.com/kltm/amigo.git
  git clone git://github.com/geneontology/amigo.git
  cd amigo
  cd amigo
git checkout 2.3.x


Next, we have to configure the AmiGO 2 installation given what we have with our BBOP JS and Apache 2 installations. The install script would do this automatically for us while throwing an error, but we'll just do it directly like:
Next, we have to configure the AmiGO 2 installation given what we have with our BBOP JS and Apache 2 installations. The install script would do this automatically for us while throwing an error, but we'll just do it directly like:


  make initialize
  npm install


This step only ever needs to be performed once, from now on you will just modify the conf/amigo.yaml file.
Now, possibly using one of the example configurations (conf/examples/*.yaml), make sure that you have a usable conf/amigo.yaml.


The most important (and fiddly) part of installing AmiGO 2 is this configuration file. For our case, in a local testing environment, the resulting amigo.yaml might look like the following. Please note that:
./node_modules/.bin/gulp install


* variables names have occasionally been changed, so this might not be exactly like your output
This command will need to be re-run every time ther eis a modification to the configuration.
* we are using a random local IP address for testing--you will have to change that
* for the time being we're using a remote GOlr server--again you'll want to setup your own and change this


---
The most important (and fiddly) part of installing AmiGO 2 is this configuration file. For our case, in a local testing environment, the resulting amigo.yaml might look like the following. Please note that:
AMIGO_BALANCER:
  comment: "Mark this AmiGO 2 installation as behind a load balanacer or similar system. The practical effect of this flag is that if AmiGO detects *.error or *.warning files (usually created by the global-message.pl script), it will return a 503 page signalling any upstream system that the instance is not currently ready.\n"
  type: boolean
  value: 0
AMIGO_BETA:
  comment: "Mark the AmiGO site as BETA (triggers some changes in variables and installed features).\n"
  type: boolean
  value: 0
AMIGO_CGI_PARTIAL_PATH:
  comment: Please enter the full path to a usable CGI enabled directory.
  type: directory
  value: /srv/www-amigo/cgi-bin
AMIGO_CGI_PARTIAL_URL:
  comment: Please enter the URL that this path maps to.
  type: url
  value: http://192.168.122.213/cgi-bin
AMIGO_CONFIGURED:
  comment: "This variable must be edited (to 1) before installing AmiGO.\n"
  type: boolean
  value: 1
AMIGO_GOOGLE_ANALYTICS_ID:
  comment: Google analytics ID. Optional.
  type: string
  value: ''
AMIGO_HTDOCS_PARTIAL_PATH:
  comment: Please enter the full path to a usable static document directory.
  type: directory
  value: /srv/www-amigo/htdocs
AMIGO_HTDOCS_PARTIAL_URL:
  comment: Please enter the URL that this path maps to.
  type: url
  value: http://192.168.122.213/
AMIGO_LAYOUT_LANDING:
  comment: A whitespace separated list of class IDs to use for searches on the landing page.
  type: string
  value: bbop_ont bbop_bio
AMIGO_LAYOUT_SEARCH:
  comment: A whitespace separated list of class IDs to use for live search tabs on the search page. NOT CURRENTLY USED.
  type: string
  value: bbop_ann bbop_ont bbop_bio
AMIGO_PROJECT_NAME:
  comment: Please enter the name of this AmiGO installation.
  type: string
  value: amigo2
AMIGO_PUBLIC_CGI_PARTIAL_URL:
  comment: Please enter the partial public URL.
  type: true
  value: http://amigo.geneontology.org/cgi-bin
AMIGO_PUBLIC_GALAXY_URL:
  comment: Please enter the public URL of the Galaxy server.
  type: url
  value: http://galaxy.berkeleybop.org/
AMIGO_PUBLIC_GOLR_URL:
  comment: Please enter the public URL of the GOlr server.
  type: url
  value: http://golr.berkeleybop.org/
AMIGO_ROOT:
  comment: "Please enter the full path to the git repo amigo[/trunk] directory.\n"
  type: directory
  value: /home/amigo/local/src/git/amigo
AMIGO_SERVER_NAME:
  comment: "The name by which your AmiGO instance wishes to known--think load balancer or cache situations. If you would rather this information not be reported, leave blank.\n"
  type: string
  value: amigo-dev
AMIGO_SHOW_GOOSE_LINKS:
  comment: "Do you wish to show links to GOOSE, the GO Online SQL Environment? Note that this is only suitable for if you are installing the GO GOLD database. Use 1 for yes and 0 for no.\n"
  type: boolean
  value: 1
AMIGO_TERM_REGEXP:
  comment: The regular expressions to identify internal ontology ids.
  type: string
  value: 'all|GO:[0-9]{7}'
AMIGO_VERBOSE:
  comment: Log verbose AmiGO debugging messages.
  type: boolean
  value: 1
AMIGO_VERSION:
  comment: Displayed AmiGO version string. This string is also used to create the AmiGO JS versioning.
  type: string
  value: 2.0b1
GOLR_TIMESTAMP_LOCATION:
  comment: "This is the full path to the timestamp file that is optionally produced by OWLTools during GOlr loading. It contains information about download attempt times for the different loaded files. If not found or unusable, AmiGO 2 will gracefully ignore it.\n"
  type: file
  value: ./golr_timestamp.log
GO_BBOPJS_LOCATION:
  comment: "If this is not redefined to a new location (e.g. the BBOP-JS staging/ directory, it will use the bbop.js bundle packaged with AmiGO 2 (the one used in the unit tests) in the _data/ directory.\n"
  type: directory
  value: ./_data/bbop.js
GO_YAML_LOCATION:
  comment: "Please enter the full path to the YAML config files directory. In that directory, all files like *config.yaml will be scanned for use.\n"
  type: directory
  value: /home/amigo/local/src/git/amigo/metadata
 
If everything is correct, installing AmiGO 2 should be as simple as:


make install
* variables names have occasionally been changed, so this might not be exactly like your output
* we are using a random local IP address for testing--you will have to change that
* for the time being we're using a remote GOlr server--again you'll want to setup your own and change this


You may want to check out various AmiGO 2 system scripts referred to in the message at the end of the installation process: global-message.pl, clean-filesystem.pl, and blank-kvetch.pl.
An example configuration can be found here: https://github.com/geneontology/amigo/blob/master/conf/examples/amigo.yaml.tomodachi


=Afterwards=
=Afterwards=


==All Glory to the Makefile (Important!)==
==System scripts==


For some time now, all day-to-day development, handling, distribution, and loading of [[AmiGO_2]] and [[GOlr]] have been handled with the Makefiles in amigo/Makefile and amigo/golr/Makefile respectively. If it cannot be done by changing your environment variables and rerunning the target that you want, you might want to reconsider what you're doing.
You may want to check out various AmiGO 2 system scripts referred to in the message at the end of the installation process: global-message.pl, clean-filesystem.pl, and blank-kvetch.pl. More information can be found in [[AmiGO_2_Manual:_Administration]].


While the Makefiles are not yet fully documented (will add them to our NaturalDocs real soon now), they may be worth perusing. This is issue: https://github.com/kltm/amigo/issues/26
==All Glory to the gulpfile.js==


For some time now, all day-to-day development, handling, distribution, and loading of [[AmiGO_2]] and [[GOlr]] have been handled with gulpfile.js. If it cannot be done by changing your environment variables and rerunning the target that you want, you might want to reconsider what you're doing.


While the gulpfile.js not yet fully documented (real soon now), it may be worth perusing. This is issue: https://github.com/kltm/amigo/issues/26


[[Category:AmiGO_2_Manual]]
[[Category:AmiGO_2_Manual]]
[[Category:AmiGO_2]]
[[Category:AmiGO_2]]
[[Category:Software]]
[[Category:Software]]

Latest revision as of 19:35, 8 August 2016

Preliminaries

The current released production version is 2.3.x (pending).

These instructions are intended to illustrate how to install the AmiGO 2 client software on a Ubuntu 14.04 LTS. With careful reading, these instructions should be generalizable to other cases. The installation of this client software only covers AmiGO 2 and some Apache 2 configuration.

While the correct version of OWLTools is now included as a JAR with the AmiGO installation, but the GOlr documentation is still available separately and should be read for proper GOlr/Solr installation.

Handling permissions and the like for the various servers are not explicitly covered in this document, but I'd like to make the observation that the AmiGO 2 and client stuff can be handled as any user as long as they have read write permissions to a htdocs and cgi-bin directory in a properly configured Apache 2 installation and the Solr server is locally accessible. Similarly, most CGI servers should be able to deliver AmiGO.

Packages

Also, in addition to anything that you might need for BBOP JS (in the optional case that you are going to develop with it), the following packages are needed:

sudo apt-get -u install make rsync rhino libconfig-yaml-perl liburi-perl libjson-xs-perl libossp-uuid-perl libfile-type-perl libfreezethaw-perl libwww-mechanize-perl libcgi-application-perl libdbi-perl libdata-formvalidator-perl libcgi-application-plugin-session-perl libcgi-application-plugin-tt-perl libgraph-perl libclone-perl libxml-xpath-perl libdbd-sqlite3-perl git yui-compressor naturaldocs libgraphviz-perl libfile-type-perl libfile-slurp-perl libdbd-mysql-perl kwalify libxml-libxml-perl xvfb python-virtualenv g++ node

You will need to also get some semi-modern form of Node.js (node and npm) onto your system. AmiGO currently requires 0.10+, with 0.12+ recommended. Options that have worked for us are ppa:chris-lea/node.js and the online installer: https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions (recommended, although we haven't tried 4+ yet).

The package libsql-tokenizer-perl is also required (by GOOSE for query checking), but may no longer be available in repositories. If you're interested in using a locally configured GOOSE, you may need to manually install this package along the lines of:

sudo apt-get -u install dh-make-perl
cd /tmp
dh-make-perl --build --cpan SQL::Tokenizer
sudo dpkg -i libsql-tokenizer-perl_???_all.deb

Depending on what you're doing with CPAN in this case, you may also need to install a helper library first:

sudo apt-get -u install liblocal-lib-perl

Configuring the Web Server

While we are not officially supporting any particular web server or configuration, we make available here some notes about setting up a server on a new installation.

AmiGO 2 was developed with Apache 2 (see preliminaries above for instructions), but I'd suspect that any CGI-capable web server would be sufficient. Configuring and running a web server is outside the scope of this document.

Prepare the amigo web directories:

sudo mkdir -p /srv/www-amigo
sudo chown amigo /srv/www-amigo
sudo chgrp amigo /srv/www-amigo
mkdir /srv/www-amigo/cgi-bin
mkdir /srv/www-amigo/htdocs

The minimal configuration we're using is:

<VirtualHost *:80>
 ServerAdmin webmaster@localhost

 DocumentRoot /srv/www-amigo/htdocs
 <Directory />
  Options FollowSymLinks
  AllowOverride None
 </Directory>
 <Directory /srv/www-amigo/htdocs/>
  Options Indexes FollowSymLinks MultiViews
  AllowOverride None
  Order allow,deny
  allow from all
 </Directory>

 ScriptAlias /cgi-bin/ /srv/www-amigo/cgi-bin/
 <Directory "/srv/www-amigo/cgi-bin">
  AllowOverride None
  Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
  Order allow,deny
  Allow from all
 </Directory>

 ErrorLog ${APACHE_LOG_DIR}/amigo-error.log

 # Possible values include: debug, info, notice, warn, error, crit,
 # alert, emerg.
 LogLevel warn

 CustomLog ${APACHE_LOG_DIR}/amigo-access.log combined

</VirtualHost>

Install Apache:

sudo apt-get -u install apache2
sudo a2dissite
(default)
sudo emacs -nw /etc/apache2/sites-available/amigo
sudo touch /var/log/apache2/amigo-error.log
sudo touch /var/log/apache2/amigo-access.log
sudo chgrp adm /var/log/apache2/amigo-*.log
sudo chmod 640 /var/log/apache2/amigo-*.log
sudo a2ensite
(amigo)
sudo service apache2 reload

AmiGO 2: Client software installation

The AmiGO 2 client software can be used completely separately from the server software as long as a Solr server with a compliant schema is available somewhere.

BBOP JS: Not Usually Needed

AmiGO 2 no longer depends on an additional download of BBOP JS, it now will use the bundled copy by default. Most users can skip the next paragraph.

For widget developers, since most of the JavaScript in AmiGO 2 relies on this library, the first step in installing AmiGO 2 is to get BBOP JS somehow available. While AmiGO 2 repository (below) comes with a packaged version of bbop.js in the _data/ directory (which is also the default when installing AmiGO 2), if you're interested in hacking around BBOP JS in conjunction with AmiGO 2, you'll have to build it yourself and change the BBOP_JS variable in the Makefile to the appropriate location so you can use the "make refresh" command. Since we don't want to repeat ourselves here, full downloading and building instructions can be found on its homepage at https://github.com/kltm/bbop-js . Scroll down a bit to "Local (building)" to get to the pertinent information.

AmiGO 2

This software contains the GO-specific things that BBOP JS does not support--the client (web server) code for AmiGO 2. For the sake of convenience (and using a similar directory layout to the developers), we'll get it like:

mkdir -p ~/local/src/git
cd ~/local/src/git
git clone git://github.com/geneontology/amigo.git
cd amigo
git checkout 2.3.x

Next, we have to configure the AmiGO 2 installation given what we have with our BBOP JS and Apache 2 installations. The install script would do this automatically for us while throwing an error, but we'll just do it directly like:

npm install

Now, possibly using one of the example configurations (conf/examples/*.yaml), make sure that you have a usable conf/amigo.yaml.

./node_modules/.bin/gulp install

This command will need to be re-run every time ther eis a modification to the configuration.

The most important (and fiddly) part of installing AmiGO 2 is this configuration file. For our case, in a local testing environment, the resulting amigo.yaml might look like the following. Please note that:

  • variables names have occasionally been changed, so this might not be exactly like your output
  • we are using a random local IP address for testing--you will have to change that
  • for the time being we're using a remote GOlr server--again you'll want to setup your own and change this

An example configuration can be found here: https://github.com/geneontology/amigo/blob/master/conf/examples/amigo.yaml.tomodachi

Afterwards

System scripts

You may want to check out various AmiGO 2 system scripts referred to in the message at the end of the installation process: global-message.pl, clean-filesystem.pl, and blank-kvetch.pl. More information can be found in AmiGO_2_Manual:_Administration.

All Glory to the gulpfile.js

For some time now, all day-to-day development, handling, distribution, and loading of AmiGO_2 and GOlr have been handled with gulpfile.js. If it cannot be done by changing your environment variables and rerunning the target that you want, you might want to reconsider what you're doing.

While the gulpfile.js not yet fully documented (real soon now), it may be worth perusing. This is issue: https://github.com/kltm/amigo/issues/26