Manager Call 2019-09-04: Difference between revisions

From GO Wiki
Jump to navigation Jump to search
Line 30: Line 30:


== Need for a IT/Cloud strategy ==
== Need for a IT/Cloud strategy ==
While not yet critical / urgent, it is important to have a IT/Cloud strategy for the project to ease the sharing of resources, virtual machines, protocols, loggings etc. LPA created a document for the USC AWS account in April, this could serve as a basis of discussion. Ticket example: https://github.com/geneontology/operations/issues/30
While not yet critical / urgent, it is important to have a IT/Cloud strategy for the project to ease the sharing of resources, virtual machines, protocols, loggings etc. Laurent-Philippe created a document for the USC AWS account in April. It could serve as a basis for discussion. Example of ticket affected by the lack of IT/Cloud strategy: https://github.com/geneontology/operations/issues/30
 
Problem to solve: GO servers and resources are scattered across multiple IT and multiple AWS accounts which are neither listed nor shared. Aside from inconsistency, lack of transparency, being cost ineffective and problems to maintain the architecture in the long term, sharing resources both for development and production is complicated and susceptible to create non standard applications and pipelines. A shared IT/Cloud strategy would also allow developers to quickly share and launch any GO tool for deep and quick iterative testing.
 
Proposal: allocate some time for the next period (October-May 2020) to:
 
1. list all currently used IT & cloud resources (servers, storage, DNS, cloud accounts, etc)
 
2. define together a group strategy and SOP for those IT & cloud resources
 
3. define and share base images and docker containers to launch all GO tools for deep testing and rapid development
 
 
Examples of what this SOP could define:
 
* how many on-site servers and AWS accounts we have and how they are organized to share resources, security permissions and costs (e.g. maybe we still want 1 LBNL and 1 USC account, but then both should be linked to a parent GO account giving permissions to each organization)
 
* who are the current admins of these servers and AWS accounts
 
* who to ask to access more restricted resources (e.g. Seth is the only one to know where the geneontology.org DNS is hosted and to have permission to alter any URL *.geneontology.org ; same thing for release.geneontology.org)
 
* where and how is administered help@geneontology.org ?
 
* how to create new AWS accounts for GO developers (e.g. Laurent-Philippe, Ben and Anushya had to use their own accounts, hence not sharing any standard, resource, images or security permissions)
 
* what default security permissions are granted to GO developers (e.g. ability to launch a test server, to write in some locations, etc)
 
* list all images and docker containers to launch and test in a standard and reproducible way any given GO tool (e.g. Noctua stack, GOLr, NEO, GO API, etc)


==Cross references on GO website/AmiGO page==
==Cross references on GO website/AmiGO page==

Revision as of 08:15, 4 September 2019

Agenda

  • Agenda: David
  • Minutes: Laurent-Philippe

Review action items from last week

  • Seth: Set up a wiki page for GO users meeting, and start announcing. (DONE)
  • PIs/managers: Start on the GO meeting agenda (this is a reminder. Normally this is in the Google drive)
    • Current projects
    • Ask if people want to present anything.
  • Kimberly: talk to Pankaj to see whether they can clean up their file, and correct annotations
    • We will ask if we can handle any annotation edits in P2GO for now.


GO Meeting general

Agenda first draft (by Pascale) https://docs.google.com/document/d/16ZCV1vyTHSEf20z_PO4F8S-uqD2ZvnyExV0McMWnFI0/edit

Review of project process

Aims GO meeting

  • Look at priority tasks for GO meeting.
    • GO-CAM specifications
    • Whole genome annotations from GAF to GO-CAM

Pipeline progress

Collaborators Webpage

We need to decide who we put as collaborators and as contributing groups. There is some redundancy. https://github.com/geneontology/geneontology.github.io/issues/130

Need for a IT/Cloud strategy

While not yet critical / urgent, it is important to have a IT/Cloud strategy for the project to ease the sharing of resources, virtual machines, protocols, loggings etc. Laurent-Philippe created a document for the USC AWS account in April. It could serve as a basis for discussion. Example of ticket affected by the lack of IT/Cloud strategy: https://github.com/geneontology/operations/issues/30

Problem to solve: GO servers and resources are scattered across multiple IT and multiple AWS accounts which are neither listed nor shared. Aside from inconsistency, lack of transparency, being cost ineffective and problems to maintain the architecture in the long term, sharing resources both for development and production is complicated and susceptible to create non standard applications and pipelines. A shared IT/Cloud strategy would also allow developers to quickly share and launch any GO tool for deep and quick iterative testing.

Proposal: allocate some time for the next period (October-May 2020) to:

1. list all currently used IT & cloud resources (servers, storage, DNS, cloud accounts, etc)

2. define together a group strategy and SOP for those IT & cloud resources

3. define and share base images and docker containers to launch all GO tools for deep testing and rapid development


Examples of what this SOP could define:

  • how many on-site servers and AWS accounts we have and how they are organized to share resources, security permissions and costs (e.g. maybe we still want 1 LBNL and 1 USC account, but then both should be linked to a parent GO account giving permissions to each organization)
  • who are the current admins of these servers and AWS accounts
  • who to ask to access more restricted resources (e.g. Seth is the only one to know where the geneontology.org DNS is hosted and to have permission to alter any URL *.geneontology.org ; same thing for release.geneontology.org)
  • where and how is administered help@geneontology.org ?
  • how to create new AWS accounts for GO developers (e.g. Laurent-Philippe, Ben and Anushya had to use their own accounts, hence not sharing any standard, resource, images or security permissions)
  • what default security permissions are granted to GO developers (e.g. ability to launch a test server, to write in some locations, etc)
  • list all images and docker containers to launch and test in a standard and reproducible way any given GO tool (e.g. Noctua stack, GOLr, NEO, GO API, etc)

Cross references on GO website/AmiGO page


Present