AmiGO 2 Manual: Installation (2.4.x): Difference between revisions
m (Sjcarbon moved page AmiGO 2 Manual: Installation (new deployment) to AmiGO 2 Manual: Installation (2.4.x): Need more granularity until everything gets packaged again into the repo.) |
|||
(16 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
[[Category:AmiGO_2]] | |||
[[Category:Software]] | |||
[[Category:AmiGO_2_Manual]] | |||
=Preliminaries= | =Preliminaries= | ||
These instructions are intended to illustrate how to install the AmiGO 2 client software on a Ubuntu | The current released production version is 2.4.x. | ||
These instructions are intended to illustrate how to install the AmiGO 2 client software on a Ubuntu 16.04 LTS. With careful reading, these instructions should be generalizable to other cases. The installation of this client software only covers AmiGO 2 and some Apache 2 configuration. | |||
For information on the data server that AmiGO 2 uses (covering the installation of and loading using the server software: GOlr schema for Solr, OWLTools, etc.), please see the ''Installation'' section of the [[GOlr]] documentation. | For information on the data server that AmiGO 2 uses (covering the installation of and loading using the server software: GOlr schema for Solr, OWLTools, etc.), please see the ''Installation'' section of the [[GOlr]] documentation. | ||
Line 9: | Line 14: | ||
=Packages= | =Packages= | ||
In general, package managers are the preferred way of getting libraries onto a system in a consistent, reversible, and secure manner. However, often you'll find yourself in a situation where you'll need to use CPAN because your preferred | ==Perl== | ||
In general, package managers are the preferred way of getting libraries onto a system in a consistent, reversible, and secure manner. However, often you'll find yourself in a situation where you'll need to use CPAN because your preferred distribution or system does not have a robust set of perl libraries. | |||
For users who need to use CPAN, but don't want to mess too deeply with their system (CPAN can turn into a nightmare when installing globally), we'd suggest using CPAN's local::lib and just putting that into your development and run paths: https://metacpan.org/pod/local::lib#The-bootstrapping-technique . | For users who need to use CPAN, but don't want to mess too deeply with their system (CPAN can turn into a nightmare when installing globally), we'd suggest using CPAN's local::lib and just putting that into your development and run paths: https://metacpan.org/pod/local::lib#The-bootstrapping-technique . | ||
==JavaScript== | |||
Given that, in the time that it has taken you to read this, yet another revolution has occurred in JavaScript, staying on top of packages can be hard and distributions do not reliably track them. For this, assuming that you have a recent version of node we do most of our package management with NPM. | |||
To get a recent version of node, we have been using | |||
deb https://deb.nodesource.com/node_0.4.x xenial main | |||
Which was the end product of following the instructions here: https://nodejs.org/en/download/package-manager/ . As you can see, we're working of of the still supported LTS 4.x. | |||
== Ubuntu/Debian == | == Ubuntu/Debian == | ||
The following packages are needed (this might need some revision): | |||
sudo apt-get -u install git make | sudo apt-get -u install git make kwalify libcgi-application-perl libcgi-application-server-perl libcgi-application-plugin-session-perl libcgi-application-plugin-tt-perl libclone-perl libconfig-yaml-perl libdbi-perl libdbd-sqlite3-perl libdbd-mysql-perl libdata-formvalidator-perl libossp-uuid-perl libfile-type-perl libfile-slurp-perl libfreezethaw-perl libgraph-perl libgraphviz-perl libjson-xs-perl liburi-perl libwww-mechanize-perl libxml-libxml-perl libxml-xpath-perl | ||
The package libsql-tokenizer-perl is also required (by GOOSE for query checking), but may no longer be available in repositories. If you're interested in using a locally configured GOOSE, you may need to manually install this package along the lines of: | The package libsql-tokenizer-perl is also required (by GOOSE for query checking), but may no longer be available in repositories. If you're interested in using a locally configured GOOSE, you may need to manually install this package along the lines of: | ||
Line 30: | Line 47: | ||
sudo apt-get -u install liblocal-lib-perl | sudo apt-get -u install liblocal-lib-perl | ||
== CentOS/RedHat == | == CentOS/RedHat ''example'' == | ||
An ''example'' comparative table of packages installed between Ubuntu and CentOS; in this case, the perl packages were manually installed rather than using CPAN (see CPAN note above). | |||
{| class="wikitable" | {| class="wikitable" | ||
Line 130: | Line 147: | ||
| NaturalDocs-1.52.zip | | NaturalDocs-1.52.zip | ||
| See Note [1] | | See Note [1] | ||
|- | |- | ||
| rsync | | rsync | ||
| rsync-3.0.6-9.el6.x86_64 | | rsync-3.0.6-9.el6.x86_64 | ||
| yum install | | yum install | ||
|- | |- | ||
| [not defined] | | [not defined] | ||
Line 159: | Line 168: | ||
/usr/local/naturaldocs), copy the zip file into the directory, then unzip the zip | /usr/local/naturaldocs), copy the zip file into the directory, then unzip the zip | ||
file there. | file there. | ||
=AmiGO 2: Client software installation= | =AmiGO 2: Client software installation= | ||
The AmiGO 2 client software can be used completely separately from the server software as long as a Solr server with a compliant schema is available ''somewhere''. | The AmiGO 2 client software can be used completely separately from the server software as long as a Solr server with a compliant schema is available ''somewhere''. | ||
==AmiGO 2 code== | ==AmiGO 2 code== | ||
This software contains | This software contains the client (web server) code for AmiGO 2. For the sake of convenience (and using a similar directory layout to the developers), we'll get it like: | ||
mkdir -p ~/local/src/git | mkdir -p ~/local/src/git | ||
Line 194: | Line 184: | ||
== Configuration == | == Configuration == | ||
Next, we have to configure the AmiGO 2 installation given what we have with our | Next, we have to configure the AmiGO 2 installation given what we have with our Apache 2 installations. The file to edit is: | ||
conf/amigo.yaml | conf/amigo.yaml | ||
Line 211: | Line 201: | ||
The configuration file for our test setup looks like: | The configuration file for our test setup looks like: | ||
[https://github.com/geneontology/amigo/blob/master/conf/examples/amigo.yaml.localhost Example configuration] | |||
After "npm install", the "gulp" command can be found in "./node_modules/.bin/gulp". The following instructions assume that this is in your path, but if not, feel free to use the full path of gulp. | |||
If everything is correct, installing a dev instance AmiGO 2 should be as simple as: | If everything is correct, installing a dev instance AmiGO 2 should be as simple as: | ||
npm install | |||
gulp install | |||
This will create configuration and amigo2 JavasScript packages, as well as run unit tests and create API docs. | This will create configuration and amigo2 JavasScript packages, as well as run unit tests and create API docs. | ||
In addition, if you are interested in viewing the data statistics pages within AmiGO, you would then run: | |||
gulp cache | |||
Further more, there will be variations for whether we are developing on the embedded server (all perl) or onto an Apache 2 CGI setup. Both of these are given different subsections. | Further more, there will be variations for whether we are developing on the embedded server (all perl) or onto an Apache 2 CGI setup. Both of these are given different subsections. | ||
Line 301: | Line 226: | ||
Running should now be as simple as a: | Running should now be as simple as a: | ||
gulp run-amigo | |||
The dev server should now be running on port :9999 (given the amigo.yaml above). | The dev server should now be running on port :9999 (given the amigo.yaml above). | ||
Line 350: | Line 275: | ||
You may want to check out various AmiGO 2 system scripts referred to in the message at the end of the installation process: global-message.pl, clean-filesystem.pl, and blank-kvetch.pl. More information can be found in [[AmiGO_2_Manual:_Administration]]. | You may want to check out various AmiGO 2 system scripts referred to in the message at the end of the installation process: global-message.pl, clean-filesystem.pl, and blank-kvetch.pl. More information can be found in [[AmiGO_2_Manual:_Administration]]. | ||
==All Glory to the | ==All Glory to the gulpfile== | ||
For some time now, all day-to-day development, handling, distribution, and loading of [[AmiGO_2]] and [[GOlr]] have been handled with the | For some time now, all day-to-day development, handling, distribution, and loading of [[AmiGO_2]] and [[GOlr]] have been handled with the gulpfile.js in amigo/. If it cannot be done by changing your environment variables and rerunning the target that you want, you might want to reconsider what you're doing. | ||
Latest revision as of 19:34, 8 August 2016
Preliminaries
The current released production version is 2.4.x.
These instructions are intended to illustrate how to install the AmiGO 2 client software on a Ubuntu 16.04 LTS. With careful reading, these instructions should be generalizable to other cases. The installation of this client software only covers AmiGO 2 and some Apache 2 configuration.
For information on the data server that AmiGO 2 uses (covering the installation of and loading using the server software: GOlr schema for Solr, OWLTools, etc.), please see the Installation section of the GOlr documentation.
Handling permissions and the like for the various servers is not explicitly covered in this document, but I'd like to make the observation that AmiGO 2 and client stuff can be handled as any user except for the necessary Apache 2 installation (when not using dev deployment directly) and the Solr/GOlr.
Packages
Perl
In general, package managers are the preferred way of getting libraries onto a system in a consistent, reversible, and secure manner. However, often you'll find yourself in a situation where you'll need to use CPAN because your preferred distribution or system does not have a robust set of perl libraries.
For users who need to use CPAN, but don't want to mess too deeply with their system (CPAN can turn into a nightmare when installing globally), we'd suggest using CPAN's local::lib and just putting that into your development and run paths: https://metacpan.org/pod/local::lib#The-bootstrapping-technique .
JavaScript
Given that, in the time that it has taken you to read this, yet another revolution has occurred in JavaScript, staying on top of packages can be hard and distributions do not reliably track them. For this, assuming that you have a recent version of node we do most of our package management with NPM.
To get a recent version of node, we have been using
deb https://deb.nodesource.com/node_0.4.x xenial main
Which was the end product of following the instructions here: https://nodejs.org/en/download/package-manager/ . As you can see, we're working of of the still supported LTS 4.x.
Ubuntu/Debian
The following packages are needed (this might need some revision):
sudo apt-get -u install git make kwalify libcgi-application-perl libcgi-application-server-perl libcgi-application-plugin-session-perl libcgi-application-plugin-tt-perl libclone-perl libconfig-yaml-perl libdbi-perl libdbd-sqlite3-perl libdbd-mysql-perl libdata-formvalidator-perl libossp-uuid-perl libfile-type-perl libfile-slurp-perl libfreezethaw-perl libgraph-perl libgraphviz-perl libjson-xs-perl liburi-perl libwww-mechanize-perl libxml-libxml-perl libxml-xpath-perl
The package libsql-tokenizer-perl is also required (by GOOSE for query checking), but may no longer be available in repositories. If you're interested in using a locally configured GOOSE, you may need to manually install this package along the lines of:
sudo apt-get -u install dh-make-perl cd /tmp dh-make-perl --build --cpan SQL::Tokenizer sudo dpkg -i libsql-tokenizer-perl_???_all.deb
Depending on what you're doing with CPAN in this case, you may also need to install a helper library first:
sudo apt-get -u install liblocal-lib-perl
CentOS/RedHat example
An example comparative table of packages installed between Ubuntu and CentOS; in this case, the perl packages were manually installed rather than using CPAN (see CPAN note above).
Ubuntu | CentOS | install type |
---|---|---|
git | git | yum install |
kwalify | kwalify | gem install |
libcgi-application-perl | CGI-Application-4.50 CGI-Application-Standard-Config-1.01 CGI-Application-Plugin-ErrorPage-1.21 CGI-Application-Plugin-Redirect-1.00 | rpm -i |
libcgi-application-plugin-session-perl | CGI-Application-Plugin-Session-1.03 | rpm -i |
libcgi-application-plugin-tt-perl | CGI-Application-Plugin-TT-1.05 | rpm -i |
libclone-perl | Clone-0.34 | rpm -i |
libconfig-yaml-perl | Config-YAML-1.42 | rpm -i |
libdata-formvalidator-perl | Data-FormValidator-4.81 | rpm -i |
libdbd-mysql-perl | DBD-mysql-4.023 | rpm -i |
libdbd-sqlite3-perl | DBD-SQLite-1.39 | rpm -i |
libdbi-perl | DBI-1.628 | rpm -i |
libfile-slurp-perl | File-Slurp-9999.19 | rpm -i |
libfile-type-perl | File-Type-0.22 | rpm -i |
libfreezethaw-perl | FreezeThaw-0.5001 | rpm -i |
libgraph-perl | Graph-0.96 | rpm -i |
libgraphviz-perl | GraphViz-2.14 | rpm -i |
libjson-xs-perl | JSON-XS-2.34 | rpm -i |
libossp-uuid-perl | uuid-1.6.2.tar.gz | rpm -i |
liburi-perl | URI-1.60 | rpm -i |
libwww-mechanize-perl | WWW-Mechanize-1.72 | rpm -i |
libxml-xpath-perl | XML-XPath-1.13 | rpm -i |
make | make-3.81-20.el6.x86_64 | yum install |
naturaldocs | NaturalDocs-1.52.zip | See Note [1] |
rsync | rsync-3.0.6-9.el6.x86_64 | yum install |
[not defined] | jline yum | install |
[not defined] | jpackage-utils | yum install |
Note [1]: NaturalDocs is a set of perl modules that is not distributed or installed in the standard fashion. It is available from http://www.naturaldocs.org/ and is distributed as a .zip file. The zip file does not create a subdirectory, so it simply extracts the files into the current directory. It's probably best to create a directory first (in our case, we usr /usr/local/naturaldocs), copy the zip file into the directory, then unzip the zip file there.
AmiGO 2: Client software installation
The AmiGO 2 client software can be used completely separately from the server software as long as a Solr server with a compliant schema is available somewhere.
AmiGO 2 code
This software contains the client (web server) code for AmiGO 2. For the sake of convenience (and using a similar directory layout to the developers), we'll get it like:
mkdir -p ~/local/src/git cd ~/local/src/git git clone git://github.com/kltm/amigo.git cd amigo
Configuration
Next, we have to configure the AmiGO 2 installation given what we have with our Apache 2 installations. The file to edit is:
conf/amigo.yaml
If you do not have a conf/amigo.yaml file, you can start from the default template with:
cp conf/.initial_values.yaml conf/amigo.yaml
The most important (and fiddly) part of installing AmiGO 2 is this configuration file. For our case, in a local testing environment, the resulting amigo.yaml might look like the following. Please note that:
- variables names have occasionally been changed, so this might not be exactly like your output
- we are using a random local IP address for testing--you will have to change that
- we are using the username foo
- for the time being we're using a remote GOlr server--again you'll want to setup your own and change this
The configuration file for our test setup looks like:
After "npm install", the "gulp" command can be found in "./node_modules/.bin/gulp". The following instructions assume that this is in your path, but if not, feel free to use the full path of gulp.
If everything is correct, installing a dev instance AmiGO 2 should be as simple as:
npm install gulp install
This will create configuration and amigo2 JavasScript packages, as well as run unit tests and create API docs.
In addition, if you are interested in viewing the data statistics pages within AmiGO, you would then run:
gulp cache
Further more, there will be variations for whether we are developing on the embedded server (all perl) or onto an Apache 2 CGI setup. Both of these are given different subsections.
Development server (embedded perl)
To run the local embedded development server, you will have to add the following package:
sudo apt-get -u install libcgi-application-server-perl
Running should now be as simple as a:
gulp run-amigo
The dev server should now be running on port :9999 (given the amigo.yaml above).
Production server (Apache 2)
While we are not officially supporting any particular web server or configuration, we make available here some notes about setting up a server on a new installation.
AmiGO 2 was developed with Apache 2 in mind, but I'd suspect that any CGI-capable web server would be sufficient. Configuring and running a web server is outside the scope of this document.
First, we need to install the Apache 2 web server:
sudo apt-get -u install apache2
If you are going to use the suggested ModMacro, you'll also need:
sudo apt-get -u install libapache2-mod-macro sudo a2enmod macro
Great, now we need to get Apache setup to aim at our source/deployment directory. We have a template to work with using conf/amigo2-apache.conf.
sudo cp conf/amigo2-apache.conf /etc/apache2/sites-available/
This is easily changed by changing sjcarbon to foo. Now start the server.
sudo a2ensite amigo2-apache sudo a2enmod rewrite sudo a2enmod cgi sudo service apache2 restart
You may be able to check the server now at: http://localhost:9999
You may run into permission issues for newer versions of Apache 2 like
Forbidden You don't have permission to access /amigo on this server.
In this case, please see these documents:
- https://wiki.apache.org/httpd/ClientDeniedByServerConfiguration
- https://httpd.apache.org/docs/2.4/upgrading.html#access
Afterwards
System scripts
You may want to check out various AmiGO 2 system scripts referred to in the message at the end of the installation process: global-message.pl, clean-filesystem.pl, and blank-kvetch.pl. More information can be found in AmiGO_2_Manual:_Administration.
All Glory to the gulpfile
For some time now, all day-to-day development, handling, distribution, and loading of AmiGO_2 and GOlr have been handled with the gulpfile.js in amigo/. If it cannot be done by changing your environment variables and rerunning the target that you want, you might want to reconsider what you're doing.