GO 18th Consortium Meeting Minutes Day 1

Introductions chronologically:

2007 dimitri seth carbon bop 2006 jim hu ecoli, susan tweedie, trudi pamgo Donghue tair 2005 ben sgd, 2004 doug zfin, ruth ucl, victori rgd 2003 jen Emily, alex deihl mgi, mary dolan mgi, Karen eilbeck, petra fey, ranjana Caltech, pascale ddb, Kimberley, 2002 jdr bop, eurie sgd, Tanya Amelia 2001 rama sgd, mgwin tigr, Harold mgi, 2000 rolf ebi, rex ddb, cm bop, 1999 midori, kara dolinski PU, david hill mgi, 1998 suzi lewis bop, MA, mike cherry, judy

GO Consortium Meeting Minutes

Sunday morning, September 23, 2007

  1. Introductions, especially new people

Introductions chronologically:

2007 dimitri seth carbon bop 2006 jim hu ecoli, susan tweedie, trudi pamgo Donghue tair 2005 ben sgd, 2004 doug zfin, ruth ucl, victori rgd 2003 jen Emily, alex deihl mgi, mary dolan mgi, Karen eilbeck, petra fey, ranjana Caltech, pascale ddb, Kimberley, 2002 jdr bop, eurie sgd, Tanya Amelia 2001 rama sgd, mgwin tigr, Harold mgi, 2000 rolf ebi, rex ddb, cm bop, 1999 midori, kara dolinski PU, david hill mgi, 1998 suzi lewis bop, MA, mike cherry, judy

  2. Quick review of any outstanding Action Items from Jan 2007GOC meeting held in Cambridge, UK http://www.geneontology.org/GO.meetings.shtml?all#consort 

delayed until end of morning before lunch

[edit] Progress Reports

2007 Progress Report for NHGRI due Jan. 1, 2008

suzi – intro of reports. Isa complete done-ish still some work.

       These reports will review accomplishments to date. 
       We are using the itemized list of sub-aims from the grant to organize these 

Aim1: We will maintain comprehensive, logically rigorous and biologically accurate ontologies.

Midori, David. Using wiki, everything they do on phone etc minuted on wiki (partially an experiment):

http://gocwiki.geneontology.org/index.php/Ontology_Development

Barely finished isa complete at last meeting, but is now done and a system is in place to make sure it remains so.

David will speak on his collab with Harvard and MIT: Visualization theory, working on structure improvements and specificity of parentage.

Want 3 high level terms disjoint – cellular process, ??? process and ??? – Details on wiki.

Terms applicable to only one taxa… we are moving away from the sensu system.

Midori generally discussing the Ontology Development wiki pages linked above.

IMG mapping. How much hanging now Jane gone on maternity? Not fixed!

JB/SL wiki can get muddled sometimes – managers should keep on top of this. Alex – if you add new large section you should send out a general email.

SL Action – tutorial on wiki. Rex – wiki help contact group.

David:

Taxon and sensu: Sensu confused users. Asked experts in different areas to review their sensu terms. 60 left to differentiate.

Function - Process links. Not much done, waiting for new OBO (because waiting for cross products) and mappings not as straightforward as we first thought.

Regulation. Add new relationship – ‘regulates’.

David Hill speaks too fast but Emily doing great job on this one. Chris went through ontology and told them were things did not make sense. All analyses in CVS in /go/scratch directory regulation-of-non-process.txt and on the wiki they have been split into 3 distinct categories detailed here:

http://gocwiki.geneontology.org/index.php/Regulation_Worksheet

Part 2:

How are we going to relate these terms to the rest of the ontology?

Action Item: need to trash through regulation.

Part 3 are problem cases. Including typos! Search on this page for ‘?’ for proposals by David where he needs resolution. Going through this list created links between function and process ☺

Negative and positive regulation question – were subtypes. Blood pressure it made no sense. Thought better to be part_of relationships. Keep it as subtypes for BP.

Collaboration with MIT/Harvard group.

Terms in wrong place? Shown through with their content analysis tool. Information Content Analysis. Can we place these terms in a better place – most of the time the answer was, yes. For example, pilus retraction moved further down.

Possible to put this analysis into GOC tools? CM – analysis already in database – check with Chris what he means.

Ontology Structure – CJM

Wiki should be merged with Ontology Development

http://gocwiki.geneontology.org/index.php/Ontology_Structure

Mining Reactome links to link process to function – more after lunch.

Internal cross products can start to be created and maintained in the ontology. OBO-Edit 2.0 will make it easier to maintain these cross products. Features ontology repair tool for links that don’t exist or are broken. Need consistent rules for regulation terms where the regulated stand in part_of relations.

New cross product guide on wiki. Links to ongoing work on BP – CP cross products;

http://gocwiki.geneontology.org/index.php/Cell_cross-products

Karen Eilbeck SO progress

Development : March – august joined J Thornton group - Gabi reeves for Biosapiens project. 96 new terms to SO.

Mark Hathon (with Barry)– ongoing work.

Content meeting in June, HLA immunology community – looking for terms to describe variants. New terms, rearranging of SO – very productive.

Collaboration with phyGo.

Working on synonyms with Colin Batchelor.

Release SO every 2 months.

Karen dropping down to 60%.

COFFEE BREAK

Aim2: We will comprehensively annotate reference genomes in as complete detail as possible.

Reference Genome Project

Rex/Pascale

Provide comprehensive annotation for 12 genomes. MOD, genome DB, curators. Complete means breadth and depth. Breadth – every gene. Depth – to the highest possible knowledge. If small amount of papers then read all. If extensive then summary of reviews. Metrics. GET REX’s PRESENTATION.

250 genes identified for curation. Gene when mutated should contribute to a disease (OMIM).

Per MOD – curators responsible for identifying orthologs using the commonly available tools.

S/W – Google spreadsheets – erratic. Not robust. Anxious to work with the SW group to develop a database – requirements have been written up. Merchant (left in July) wrote prototype. A new member of staff is starting at the end of September to continue development.

Annotation Progress – see slide.

Display approaches – comparing annotations to generic GOSlim branches.

Number of sourceforge requests from reference genome group in the hundreds over 16 months. Average of 10-12 requests per month. GO editorial group doing a good job at keeping up with these. Existing requests are problematic. 411 terms.

Aim3: We will support annotation across all organisms.

Annotation Outreach – Jen Deegan

Keeping track of new groups annotating and writing documentation.

ASK JEN FOR HER SLIDES.

People going to meeting – report back gossip from willing people to Jen.

The SOPs have been tricky but are now on the public GOC website:

http://www.geneontology.org/GO.annotation.SOP.shtml

Michelle created nice ISS guideline SOP.

Action Jen: A reference to these pages should go in next newsletter.

There has been much progress on grants.

Attending many regular conferences.

Less cold calling, it wasn’t very successful. More luck tracking down the right person at conferences. Responding to invitations.

Eurie - User Advocacy

Focusing on lines of communication, newsletter and mailing list. Rota of mailing list monitors.

Newsletters archived. Future news items page on wiki. Michael A wants ISSN for the newsletter. Action Michael! Michael sent URL to Eurie – Action Eurie!

Users meetings, we have a page of potential meetings on wiki.

Tools standards.

Production Systems – Ben Hitz

Deployed 4 nesw linux machines 1 for loading, 2 for amigo poduction, 1 amigo development.

ASK BEN FOR SLIDES

Production Amigo now more fault resistant.

Go Database loading speeded up and now in testing.

Godb sequences – using gp2protein files. If possible do all sequences in your DB, not just annotated.

Assocdb fasta file – Header line massive – can be slimmed down?

Association file cleaning – All IEAs must have a with field.

Amigo – Amelia ASK AMELIA FOR SLIDES

Term enrichment.

Go slimmer.

LUNCH BREAK

Action Items Review

4. IN PROGRESS Action item for Michelle – get this info [i.e., on TIGR course attendance and whether any participants use GO afterwards -mah] We just got our Annotation Engine grant funded and as part of that we will be surveying people who attended the classes.

Check on attendees if they use go afterwards.

8. Action Item: GO-top: Talk with each manager and bring summary reports to the next meeting.

Managers phone call

10. NOT DONE. Action Item for John: Add a term creation date to the .obo file. It didn't get added to the feature request tracker so it was overlooked. Now added:

added to tracker on plane. DONE

12.In progress. Action item for Midori and David (from 2): Find a way to communicate responsibilities and availability viz ontology working groups. Devise a systematic way to bring closure to outstanding term request items. We've made some plans, but not yet acted upon. We have begun implementing our plans (and refining them, as needed); a respectable number of SF items have been assigned, though for others we still need to find expertise. I think of this as an ongoing ontology development effort rather than a discrete one-off action item. [mah]

Done for purpose of meeting. Ongoing inherent part of ontology development,

13. NOT DONE: Action item for AmiGO Working Group (from 4): The AmiGO Working Group will implement a strategy to incorporate and display the contents of the GO references. Request is on the AmiGO tracker (thanks Jane) http://sourceforge.net/tracker/index.php?func=detail&aid=1667315&group_id=36855&atid=494390

Done in next release. Amelia. On the tracker still. IN PROGRESS

15. Action Item for GOA (from 1): Talk to Reactome about getting non-TAS evidence. TAS is no longer considered a useful evidence code and will not be used in any consistency measures of reference genome annotation. Since part of the idea of the reference genomes is to provide a source of IEA annotations for other groups, we strongly encourage reference genome annotators to not use TAS, and instead use experimental evidence codes whenever possible. The GO documentation should also state this in a clear fashion. [annotation camp participants]

IN PROGESS NEXT ON AGENDA

16. Action Item for Judy, Harold, Amelia, Eurie, John Day-Richter (from 3,4,5): Resolve communication issues around obsoletes.

Midori to look into but probably done – tags applied to obsolete terms in OBO edit file.

17. NOT DONE (more discussion required) Action Item for Chris and Jane (Revisit 7): Remove the word 'activity' from the molecular function terms, and consider renaming the molecular function ontology. [note that later on in the meeting after action item 73 notes state that more discussion on this proposal is needed]

Strike activity from the function terms but some people very opposed.

18. [In Progress] Action Item for Jen & Chris (from 8): Assign priorities for contents changes needed to implement sensu plan. Discuss the different aspects of changes to our use of sensu, write documentation on this, and implement the new strategy. This change will then be announced to the community. Comment: There is more discussion to be had on this and a slot has been assigned for this in the meeting.

Jen to update on sensu later in meeting. IN PROGRESS

20. Action Item for Karen C. (from 11): Add to the documentation that it is okay to use experimental evidence codes for perfectly identical gene products from different strains of the same species.

IN PROGRESS.

21. Action Item for Eurie (Revisit 12): No conclusion about how to distinguish large- vs small-scale experiments was reached. People are encouraged to keep thinking about this issue, which clearly needs more discussion.

To be discussed at this meeting.

26. Action Item for Karen E.(?) & Rex: –Work out kinks in getting this data [i.e., GFF3 files] and make it readily available from the GO site.

IN PROGRESS.

28. Action item (Suzi, Tanya): Continue developing [protein family-based annotation] tool.

Put aside for the moment. Need to revisit. EVOLVING

29. Action item (Midori and Rex): Do it [i.e., new column in annotation files], add the column and document the formalism. It's not at all clear why this has my name (and I don't remember); it seems more for the software group and annotators. [mah]

TO BE DISCUSSED. Column 16. Structured Notes Field AKA SLOTS ☺

31. Action Item (Jane): Keep researching CARO integration. IN PROGRESS

32. Action Item (AWG): Explore whether this [Mary's graphs] would be a useful addition to AmiGO 2.0.

STILL EXPLORING. Use logs? More discussion.

33. Action Item (AWG): There should be more tracking of what users are doing in the next AmiGO; explore the options.

Adding dates to file done but nothing further happened.

35. Action Item (John & Jane): Come up with a proposal for a history tracking tool, including timeframe and priority, and send out email.

36. Action item (David, Jane): Organize first training meeting.(postponed due to the amount happening at this meeting)

37. Action Item (Nicole, Jane, John, Mark): Create proposal to group for term submission software, including the “minimum standards for each term”.

JOHN not done – proposal considered.

38. NOT DONE! Action Item (AmiGO Working Group): Change AmiGO to hide (by default) structured comments of certain types. Obsolete comments will not be shown. Structured comments that don’t belong will not be shown. Have option to hide comments. [different types of comment not specified so no action]

Need more specification. Amelia.

39. Action Item (Karen E.): Send the information [microarray tool evaluation] to the group

Send results but hasn’t. Probably obsolete.

40. Action Item (User Advocacy): Create a Wiki page for each tool on GO tools. GO's official tool recommendations (if we have them) can be posted on the Wiki. Link to a Google user group for tool discussions

41. Action Item (User Advocacy): Publish minimum standards for tools on GO tools. This can just extend the existing questionnaire.

IP

42. Action Item (Everyone): Email suggestions for meetings at which we'll do outreach to Eurie & Jane & Jen. [Ongoing - working well]

IP

44. Action Item (Eurie & Jane): Create Wiki of meetings gathered from emails.

IP

45. Action Items (Managers): Revisit metrics & priorities.

ONGOING

46. Action Item (Managers): Create structure within working group Wikis to allow people to announce what they are working on. Ontology development started, and ongoing; as "done" as it can be. [mah]

ONGOING

47. Action Item (Everyone): Create list of projects they are working on, post on Wiki. Ontology development done. [mah]

ONGOING

49. Action Item (Chris, Karen C.): Publish a succinct paragraph explaining why unknowns were removed. Karen C. will write documentation targeting biologists; Chris writes an explanation targeting software engineers. Publish on website, possibly in newsletter, wherever necessary.

done Items 50-52 are about the WITH field for the IPI code. 50. Action Item (MGI, SGD): Change all pipes to commas.

DONE – DAN AND DAVID TO WRITE ON WIKI ACTION ME

51. Action Item (Chris, Ben): Work through any database implications.

???

52. Action Item (User Advocacy): Announce in newsletter, “what's new” section of GO site.

????

53. Action Item (Everyone) – Tell code extension advocates to piss off. Attendees voiced strong opinions against extending evidence codes.

ONGOING

55. Action item (Karen C): makes these statements very explicit in the documentation.

DONE

56. Action Item (Rex, Eurie): Create a gene page for each reference gene on Wiki. Use Alex's pages as a template. Link Wiki pages to existing annotations.

DONE

60. Action Item (Evidence Code Working Group): Develop a decision tree for choosing an evidence code. Present results at next consortium meeting. (large scale/small scale, reviewed/un-reviewed. Similarity/not-similarity etc).

TO BE DISCUSSED AT THIS MEETING

61. Action Item (Rex): Assign programmer to check integrity of WITH field for annotations.

NEED NEW PROGRAMMER

62. Action Item (Everyone): Add SwissProt keyword ids to WITH column when appropriate.(MGI-DONE, GOA-DONE, ZFIN-DONE)

DONE

63. Action Item (Evidence Code Working Group): Send RCA code back to committee.

TO BE DISCUSSED AT THIS MEETING

74. Action Item (Chris, Jane, Harold): Create concrete proposal for linking between all GO ontologies, get relationship ontology up to speed, do pilot project in metabolism.

ONGOING

76. Action Item (Suzi): Come up with a structure for reporting our time commitments.

ONGOING

78. Action Item (User Advocacy): Draft an official GO brochure.

TO BE DONE

79. NOT DONE, can do this when needed. Action Item (Mike): Create GO banners.

ONGOING

81. Action Item (Everyone): Use the word cellular to refer to things related to a cell but aren't necessarily cells or things contained within a cell. Use the word cell to refer to cells themselves or things contained within a cell. (Change cell signaling to cellular signaling, for example).

ONGOING

82. Action Item (Jane): Tweak cellular component definition to clearly distinguish cellular component from cell part.

ONGOING

Afternoon, Sunday, Sept 23, 2007 Discuss protein complexes and the intersection with Reactome annotations

Peter D'Eustachio (Reactome) will be able to join us only for the afternoon.

GET SLIDES FROM PETER

These are some other topics that Ewan Birney brought up at the Interactome meeting:

   * Reactome has a lot of function annotations with the connections to the complex that has the function. These are for very small complexes like homodimers. He says can we accept these? Also he says that he understands that this is a bit granular for us and that we have a lot of similar information for larger complexes. He suggests that there might be better way to store all the data for the functions of all sizes of complexes if we pooled out data and thought of a good system to make it available. 

   * Ewan Birney and the people at the Interactome meeting felt strongly that the more complicated annotations such as those with a NOT or colocalizes_with qualifier, and those with annotations to the root should be in a separate file from the straightforward annotations. The users felt that many people do not know about these more complicated annotations and it would be better to make them an added extra that people must specifically go to download. They were particularly concerned that users may not know that it is important to parse the qualifier column and so may miss vital information. 

• Reactome would like to have a new evidence code to show where an IEA has been transferred from a known orthologue. Such transfers tend to produce more granular annotations. (This has been discussed before but Ewan asked me to bring it up while Peter is there to back the proposal.)

Action: Get Reactome data into GOC before next meeting.

Jen – taxon and GO

JEN IS DOING THE MINUTES FOR THIS ONE

Comments:

DH: useful for ipro2go annotations

VW: ditto

EH: helps annotators, not necessarily good for users, but helpful for new groups?

RL: does it really need to be that complicated?

JB: could be accomplished by triggers, a rulebase. Therefore by definition this would only be seen in a taxa. Efforts should be on annotation of literature. One off triggers.

Sue Rhee: Should explore using goslims? Possibly create a file of cross products to taxa on terms, therefore multiple taxa per terms.

SL: Nice to have an evaluation. Take existing annotations and say that this term has been used in this species.

JDR: large scale derivation of annotation data BAD idea – propagating the garbage. Do something on a higher level – say 10 and see what filth it uncovers.

MC: Too much work. Another piece of SW to be built.

VW: relevant_to and not_releavant_to would be a minimum

RC: no objection to relevant_to but opposed to universals – i.e. is_only_relevant_to.

Missed judy and Michael here…. BH: what fraction of problems would be solved if there were CPs to anatomy ontology missing?

Consensus: using this for error checking would be very useful

RL: taxa isn’t a be all an end all for help in term definition

Consensus: Some of us have doubts and it will cost so some sleep time needed.

Action: as creating new terms – see if ths is the way to go?

PG: Would not use restricted GO set to curate – what would you be missing?

JDR Proposal,

Objections – unmaintainable useless and a waste of time. jen to do pilot pass to see what happens, then she presents it at next meeting

JB proposal – get another group to do it.

JH: issues with true path – when going up levels, start to see untruths

ACTION Jen do experiment

[edit] Cross-products

Background: Cross_Product_Guide

  1. Collaboration and cross-products with Cell Ontology
  2. Taxon-GO links
  3. The intersection of SO and CC (Chris and Karen E) 

[edit] Discussions

  1. Incorporation or synchronization with the SAO (sub-cellular anatomical ontology)
  2. How can we confer micro-credit to experts who contribute to the content?
  3. Creating Complex terms in the component ontology-when do two interacting proteins become a complex? How long should the interaction last for it to be called a complex? (Rama)
  4. Delineating, and isolating, the different categories of "slims" and their usage. See Taxon_Main_Page for more discussion.
  5. The scope of 'binding' terms in the function ontology:
        1. How granular should terms be?
        2. Should there be terms for substrate origin (e.g. "viral protein binding")? See SF 1674020. 
  6. Discuss whether GO can add a has_part relationship (mah, krc) 

[edit] Morning, Monday, September 24, 2007 [edit] Topics related to Annotation

  1. Annotation of protein complexes
  2. Finalizing the format for cross-product information for the new column 16 of the gene association file - could this column also include information on the 'target' of a protein's activity?
  3. Issue with taxon/dual taxon usage to capture strains and cultivars etc, for which taxon IDs do not exist in NCBI.
  4. Ontology versioning proposal for keeping annotations up-to-date (John and Chris) 

[edit] Annotation Evidence Codes

  1. Evidence code proposals from Evidence Code Committee
        1. Finalizing Proposed evidence code documentation (need someone to oversee completion of final details; see also summary email and Proposed evidence code documentation)
        2. Boundary between ISS & RCA and scope of RCA evidence code (Michelle and Rama (filling in for KarenC); see Evidence Code Committee discussion on ISS vs RCA and Email sum up on RCA (contains link to RCA example papers)).
        3. Use of with column for ISS, pending decision on ISS vs RCA boundary (?; see summary of ISS with (blank) proposal (contains link to ISS with '____' sample papers) and some further comments on filling with field for ISS).
        4. Boundary between IMP and IGI (Val; Email synopsis and further comment)
        5. Use of with column for NAS evidence code (David; see summary Email thread on NAS).
        6. Annotations to root nodes (Rama; see Email sum up on ND and probable conclusion of use of ND for root annotations).
        7. Thoughts from MGI See this document and some responses to MGI from Karen
        8. Karen's draft of an Evidence Code Decision tree 
  2. Use cases for evidence codes
  3. "use" of IPI protein binding to track physical interactions 

[edit] Gene Association file

(Rama and MikeC)

   * We have previously discussed the interest of some groups in having two gene association files. This will allow users to easily download a subset of the annotations. The GO database will load all annotations files. We propose a discussion on the naming convention that should be used for these second (or third) files. The HTML table would have all GA files listed. This proposal comes from SGD as we have now incorporated all S. cerevisiae annotations from the GOA UniProt file. The manual annotations from this file are included in our gene_association.sgd file with UniProt (or MGI) as source. We would like to use a second file for the IEA annotations. This is to allow users to easily use the curated annotations or the computationally predicted set. The worry is that many do not look inside the file before they use it for analysis or as a training set. If developers include the computationally predicted annotations within their training sets the overall quality of there annotations will suffer. 

   * The HTML annotation table (http://www.geneontology.org/GO.current.annotations.shtml) should be sorted by kingdom. The HTML table is currently sorted by genus name. 

[edit] Afternoon, Monday, September 24, 2007 [edit] Our Public Face [edit] AmiGO

  1. AmiGO Architecture
  2. ORB and AmiGO
  3. GOOSE
  4. Changes to AmiGO to improve visualization when there are multiple relationships linking terms 

[edit] And the rest

  1. Wiki
  2. Suggestion by PAMGO community to propose the use of GO terms as keywords in journals. 

[edit] GOC and Staff issues (this is largely go-top stuff, perhaps a breakfast?)

   * Considering sending in a proposal to ESF Research Conferences 2007 RFA
   * The PIs should consider options for the future of the GO Editorial Office (GOEO). Affiliation with EBI has numerous advantages (for both GO and EBI), but all four GOEO staff will reach the 9-year limit by the end of the current NIH grant period. 

   (added at Michael's request) 

   * Currently we have several working groups including:
        1. Ontology content - to work on the content of the ontology
        2. Annotation-Outreach - to encourage new groups to start annotating
        3. User Advocacy - to help users to learn how to use the GO and to represent user's need to the consortium.
        4. Reference Genomes - to choose genes lists for everyone to work on. 

So the questions are:

   *
        1. Would it be a good idea to merge user advocacy and annotation outreach as these have a lot in common?
        2. Would it be possible to make a general annotation working group to deal with all issues surrounding annotation? This group could include sub-groups such as the reference genomes group, the evidence code groups and the annotation standard operating procedure group. The main annotation group could have two annotators to chair, or perhaps a rolling chair.