Software meeting 2013-07-17

From GO Wiki
Jump to: navigation, search

Agenda

Location: Stanford

  • Noon - 1pm (Over lunch)
    • Uberon discussion (Venkat, Laurence, Chris)
  • 1:00 - 2:30pm AmiGO 2 (Stuart, Kalpana, Shuai, Ben, Gail, Seth, Chris)
    • Installation and set up, including Solr
    • "Clean" urls in AmiGO 2.x
    • How to manage link forwarding/mapping between A1.8 and A2.0
    • Frontend/backend metadata syncing
  • 2:30 - 3:00pm
    • Other production issues (Stuart, Kalpana, Shuai, Ben, Gail, Seth, Chris)
  • Move cronjobs to Jenkins
  • Frequency of Amigo1 loading
  • Location of GOOSE
    • will be packaged with AmiGO 2
  • 3:00 - 3:30pm
    • Drupal training/demo for adding content (Seth, Rama)

Notes

Highlights from the GO/AmiGO 2 meeting at Stanford, July 17, 2013

Uberon / sample ontology discussion

Chris/Venkat/Drew/Laurence/Seth

  • Discussed use of GOlr pattern of caching closure of terms in Solr

AmiGO 2

Installation and set up, including Solr

AmiGO 2 consists of 3 parts:

  • Solr (standard setup)
  • Loader (OWLTools, all in java)
  • Frontend - majority is js. lightweight perl layer that just delivers contents. use of perl modules will be reduced in future.

BBOP uses Ubuntu, SGD CentOS - Stuart was hand-compiling perl modules. Also difficulties installing yui-compressor.

ACTION: Seth is going to generate a KVM VM of the AmiGO 2 Ubuntu installation at Berkeley and send it to Stuart. At least initially Stanford will try running the AmiGO 2 frontend on a VM. Seth recommends that the backend be located on a physical server with lots of memory (>70GB). He found that massively parallel jobs causes significant performance issues on a VM. The backend "loading" is all done in memory. No intermediate database necessary. The frontend consists of 3 parts: the UI, Solr, and GOlr (GO specific modules used with Solr).

Before AmiGO 2 is put into production, Seth wants to rewrite/improve the UI.

Solr config - the more memory the better.

We discussed optimal setup - separate machines for Solr and AmiGO. Client talks directly to Solr, light layer (nginx) prevents admin operations.

For testing, we discussed selenium (AmiGO 2 has a number of tests included). Discussed using Laurence's module.

Clean urls in AmiGO 2

Everyone agreed that clean ("restful") URLs should be used with AmiGO 2 in the format: http://geneontology.org/term/GO:XXXXXXX. This means that every URL will need to be mapped in apache. Seth is going to work on simplifying the URLs in the beta release of AmiGO 2 slated for October. He will get Stuart a list of old and new URLs. (Note: I don't remember talking about sending an explicit map, but attempting something similar on our end first--the actual apache file would be pretty different given our different setups. -Seth)

Proposed:

How to manage link forwarding/mapping between A1.8 and A2.0

Via apache. See above discussion.

Frontend/backend metadata syncing

Other production issues

Move cronjobs to Jenkins

A reasonable idea. It would be good to clean-up/remove old cronjobs no longer needed after moving to AmiGO 2.

We need a list of jobs that run on Mike's machine that update content within the www directory

Frequency of AmiGO 1.x loading

Keep the AmiGO 1 loading the same through 2013. Revisit in 2014. AmiGO 2 at Berkeley is being loaded 2x/week. Takes about 7 hours to do the equivalent of a golite load. Need more discussion about frequency.

Location of GOOSE

GOOSE is packaged with AmiGO 2, so it makes sense that the GOOSE software is run out of Stanford. The GOOSE mysql database will remain at Berkeley. (Note: One of the mirrors will remain at Berkeley; the initial mirror is chosen at random. -Seth)

Drupal training/demo for adding content

Seth gave Rama a quick demo of the Drupal installation for the new Gene Ontology web site (beta.genontology.org). Many similar features as WordPress. Seth suggested/recommended that we use a Drupal hosting company to maintain the software and be responsible for keeping it upgraded. Some possibilities are: Pantheon (https.www.getpantheon.com/), A2 Hosting (www.a2hosting.come/drupal-hosting).

We agreed on a plan for migrating:

  • ACTION: Seth will generate "de-htmlified" versions of all our current content - done
  • ACTION: Web team (Rachael, Rama and Kalpana) will create Drupal pages for each of these, possibly manually editing formatting etc,

Tentative Timeline for AmiGO 2 deployment testing

  • July 26 - Seth provide Stuart with AmiGO 2 frontend VM - done!
  • Aug 1-15 - Seth on vacation - done!
  • Aug 23 - Seth evaluate the effort to rewrite the UI in time for the October GO consortium meeting -done!
    • looks good, prototype done; moving forward
  • Aug 30 - Stanford has a running version of AmiGO 2
  • Oct 1 - Candidate release of AmiGO 2