Ontology Editors Daily Workflow

From GO Wiki
Revision as of 09:25, 10 August 2018 by Pascale (talk | contribs)
Jump to navigation Jump to search

Updating the local copy of the ontology with ‘git pull’

  •  Navigate to the ontology directory of go-ontology: cd repos/go-ontology/src/ontology.
  • If the terminal window is not configured to display the branch name, type git status

You will see:

 * On branch [master] [or the name of the branch you are on]
   Your branch is up-to-date with 'origin/master'.
  • If you’re not in the master branch, type: git checkout master
  • From the master branch, type: git pull . This will update your master branch, with the files that are most current on GitHub, bringing in and merging any changes that were made since you last pulled the repository using the command git pull. You will see something like this:
  ~/repos/go-ontology(master) $ git pull
  remote: Counting objects: 26, done.
  remote: Compressing objects: 100% (26/26), done.
  remote: Total 26 (delta 12), reused 0 (delta 0), pack-reused 0
  Unpacking objects: 100% (26/26), done.
  From https://github.com/geneontology/go-ontology
  580c01d..7225e89  master     -> origin/master
  * [new branch]     issue#13029 -> origin/issue#13029
  Updating 580c01d..7225e89
  Fast-forward
  src/ontology/go-edit.obo | 39 ++++++++++++++++++++++++---------------
  1 file changed, 24 insertions(+), 15 deletions(-)
  ~/repos/go-ontology(master) $

Creating a new working branch with ‘git checkout’

  • You should never work on the master branch. When starting to work on a ticket, you should create a new branch of the repository to edit the ontology file.
  • Make sure you are on the master branch before creating a new branch. If the terminal window is not configured to display the branch name, type git status to check which is the active branch. If necessary, go to master by typing git checkout master.
  • Create a new branch: type: git checkout -b branch_name in the terminal window.
  • Naming branches: we recommend using the string ‘issue-‘ followed by the issue number. For instance, for this issue in the tracker: https://github.com/geneontology/go-ontology/issues/13390, you would create a branch as follows: git checkout -b issue-13390. Typing this command will automatically put you in the new branch. You will see this message in your terminal window:
   ~/repos/go-ontology/src/ontology(master) $ git checkout -b issue-13390
   Switched to a new branch 'issue-13390'
   ~/repos/go-ontology/src/ontology(issue-13390) $

Continuing work on an existing working branch

  • If the terminal window is not configured to display the branch name, type: git status to check which is the active branch. If necessary, go to master by typing git checkout master.
  • Update master by typing git pull
  • To list existing local branches: type git branch -l
  • From the master branch, switch to your branch by typing git checkout [branch name]
  • Update the working branch with respect to the current version of the ontology (OPTIONAL) by typing git pull origin master
    • This step is optional because it is not necessary to work on the current version of the ontology.
    • Usually, changes in the working branch will be synchronized with master when you merge the pull request.


  • When you run the 'git pull origin master' command, 3 things can happen:
    1. All is OK: No action needed. You will get the normal prompt in your terminal window.
    2. Some changes have been imported that need to be commited: You may get a message requesting that you commit changes. Git opens a vi terminal with the following comment:
         Merge branch 'master' of https://github.com/geneontology/go-ontology into issue-16057
         # Please enter a commit message to explain why this merge is necessary,
         # especially if it merges an updated upstream into a topic branch.
         #
         # Lines starting with '#' will be ignored, and an empty message aborts
         # the commit.
      

      In the vi editor:

      1. Type i (for 'insert'). This will put your cursor in the first line ("Merge branch...") and allow you to edit.
      2. Type the comment (for example, 'merged master')
      3. Exit: type esp, then :q! to save changes and quit (q= quit, ! = save)
      • You will get a message in the terminal that looks like this, followed by the normal prompt in your terminal window:
        Auto-merging src/ontology/go-edit.obo
        Merge made by the 'recursive' strategy.
        src/ontology/go-edit.obo                  | 117 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------
        src/ontology/subsets/gosubset_prok.yaml   |   2 +-
        src/ontology/subsets/mf_needs_review.yaml |   5 ++---
        3 files changed, 95 insertions(+), 29 deletions(-)  
      
    3. There are conflicts that prevent git from merging
      • In some cases, in particular when there are inconsistencies between your branch and master, (for example when new terms have been created since your branch was last updated, or terms were merged or obsoleted), this step is necessary. These problems are usually reported by the Travis checks.
      • Sometimes when you do git pull origin master, the branch is found by git to have conflicts with master, which will lead to an error message in the terminal window (see next section).

Dealing with conflicts when updating a local branch to master

  • NOTE: This is a delicate operation. Ask someone to review the changes if you have any doubts.
  • The version of the ontology in the local branch may be in conflict with master. In this case, the terminal will return a message that looks like this (for a branch named 'mybranch'):
  Switched to branch 'mybranch'
  ~/repos/go-ontology/src/ontology (mybranch)]$ git pull origin master 
  From https://github.com/geneontology/go-ontology
  * branch              master     -> FETCH_HEAD
  Auto-merging src/ontology/go-edit.obo
  CONFLICT (content): Merge conflict in src/ontology/go-edit.obo
  Automatic merge failed; fix conflicts and then commit the result.
  ~/repos/go-ontology/src/ontology (mybranch|MERGING)]$
  • The problem with conflicts is that you cannot switch to another branch until they are resolved.
  • To fix conflicts:
    • Open the go-edit.obo file using your favorite text editor.
    • Conflicts will be within tags like
  <<<<<<< HEAD
  ...
  >>>>>>> hash number

For example:

  <<<<<<< HEAD
  =======
  id: GO:0019836
  name: hemolysis by symbiont of host erythrocytes
  namespace: biological_process
  def: "The cytolytic destruction of red blood cells, with the release of intracellular hemoglobin, in the host organism by a symbiont. The host is defined as the larger of the organisms involved in a symbiotic interaction." [GOC:add, UniProtKB-KW:KW-0354]
  synonym: "haemolysis in host" EXACT []
  synonym: "hemolysin activity" RELATED []
  synonym: "hemolysis by symbiont of host RBCs" EXACT [CL:0000232]
  synonym: "hemolysis by symbiont of host red blood cells" EXACT [CL:0000232]
  synonym: "regulation of cytolysis of host cells by symbiont" EXACT [GOC:dph, GOC:tb]
  is_a: GO:0001897 ! cytolysis by symbiont of host cells
  is_a: GO:0052331 ! hemolysis in other organism involved in symbiotic interaction
  [Term]
  >>>>>>> e3d0e5afcd225e02722423ef17d2b2fa669a33f3
  • Usually, what is within the <<<<<<< HEAD and >>>>>>> hash, can be deleted.
  • Save your changes.
  • Commit by typing git commit -i * -m 'fixed conflict' go-edit.obo
  • Do a git diff master to make sure that the changes you have made are correct ('git diff' only will not do because you have already committed).
  • If you have any doubt about the changes, delete the branch and start the work over. To do this, type git checkout master and delete the branch by typing, git branch -d mybranch.
  • If the branch is never pushed to origin, there is no impact of the main go-edit.obo file.
  • If the changes are as expected, you can go ahead with the push and the further steps.

Loading, navigating and saving the Ontology in Protégé

  • Before launching Protégé, make sure you are in the correct branch. To check the active branch, type git status
  • Open the go-edit.obo file: Click on the ‘File’ pulldown. The first time, you will have to navigate to repos/go-ontology/src/ontology. Once you have worked on the file, it will show up in the menu under ‘Open’/’Recent’.
  • Click on the ‘Classes’ tab.
  • Searching: Use the search box on the upper right to search for a term in the ontology. Wait for autocomplete to work in the pop-up window.
  • Viewing a term: Double-click on the term. This will reveal the term in the ‘Class hierarchy’ window after a few seconds.
  • Launching the reasoner: To see the term in the ‘Class hierarchy’ (inferred) window, you will need to run the ‘ELK reasoner’. Go to menu: ‘Reasoner’ > select ELK 0.4.3, then click ‘Start reasoner’. Close the various pop-up warnings about the ELK reasoner. You will now see the terms in the inferred hierarchy.
  • After modification of the ontology, synchronize the reasoner. Go to menu: ‘Reasoner’ > ‘ Synchronize reasoner’
    • NOTE: The only changes that the reasoner will detect are those impacting the ontology structure: changes in equivalence axioms, subclasses, merges, obsoletions, new terms.
  • Saving: Use File > Save to save your changes.

Committing, pushing and merging your changes to the repository

Reviewing changes

Changes made to the ontology can be viewed by typing in the terminal window: git diff. If there are changes that have already been committed, the full changes in the active branch relative to master can be viewed by typing git diff master.

Git diff in the mac terminal window
Git diff in the mac terminal window

Committing changes to the local branch

Changes can be committed by typing git commit -m ‘Meaningful message Fixes #ticketnumber’ go-edit.obo. The message ('-m') must be in either single or double quotes.

  • For example: git commit -m ‘hepatic stellate cell migration and contraction and regulation terms. Fixes #13390’ go-edit.obo
  • This will commit the saved changes to the go-edit.obo file in your active branch. The terminal window will show something like:
  ~/repos/go-ontology/src/ontology(issue-13390) $ git commit -m 'Added hepatic stellate cell migration and contraction and regulation terms. Fixes #13390' go-edit.obo
  [issue-13390 dec9df0] Added hepatic stellate cell migration and contraction and regulation terms. Fixes #13390
  1 file changed, 79 insertions(+)
  ~/repos/go-ontology/src/ontology(issue-13390) $ 
  • NOTES
    • The word ‘fixes’ is a magic word in GitHub; when used in combination with the ticket number, it will automatically close the ticket (in the go-ontology repo; if you are working on a ticket from a different repo the command will still go to the go-ontology repo and close a ticket there). In the above example, when the file is merged in GitHub, it will close issue number 13390. Learn more on this GitHub Help Documentation page about Closing issues using keywords.
    • ‘Fixes’ is case-insensitive.
    • If you don’t want to close the ticket, just refer to the ticket # without the word ‘Fixes’. The commit will be associated with the correct ticket but the ticket will remain open.
    • It is also possible to type a longer message than allowed when using the ‘-m’ argument; to do this, skip the -m, and a vi window (on mac) will open in which an unlimited description may be typed.

Pushing working branch to origin

To add your branch to the remote repository, type: git push origin mynewbranch.

Example: git push origin issue-13390

  • TIP: Once you have pushed your changes to the repository, they are available for everyone to see, so at this stage you can ask for feedback.

Creating a Pull Request for your branch

How to view branches
How to view branches

Compare (last sanity check before pull request)

Click the green button Compare & Pull request on the right (see screenshot below). This will allow you to open a Pull Request, and determine if there are conflicts between your branch and master (if there are no conflicts, there will be a green check mark and the text 'Able to merge'.

How to initiate a pull request
How to initiate a pull request

The diff for your file is at the bottom of the page. Examine it as a sanity check.

Git diff
Git diff

Open the pull request

Click the green button Create pull request (see screenshot below). This will start to compare your branch with master and initiate the Travis checks to find potential errors in your branch (for example: missing IDs, missing references for definitions, is_a incomplete, logical errors).

How to open a pull request
How to open a pull request

Request a review (optional)

You may now add comments and ask a colleague to review your pull request by selecting a reviewer in the ‘Reviewers’ list (e.g. @superuser1). The reviewer will be notified. Since the pull request is also a GitHub issue, the reviewer’s comments will show up in the comments section of the pull request, similarly to any other issue filed on the tracker.

Automated checks

  • Wait for the Travis checks to complete (this can take a few minutes; see screenshot).
Travis checks
Travis checks
  • If the Travis checks fail, you will see a red x at the left of each of the checks. Go back to your local branch, correct the reported errors, commit them to the branch, and push to origin. The Travis checks will then start automatically (since the pull request has already been generated).
Travis Fail
Travis Fail
  • If the Travis checks pass, you will see a green check mark at the left of each of the checks.
Travis success
Travis success


Merging the Pull Request

When (and only when!) the Travis checks are successful and if you're ready to integrate the changes in the main ontology file, merge the pull request, by clicking on the green Merge pull request button.

  • Note that the reviewer should go ahead and merge the pull request if s/he agrees with the changes.
  • Confirming the merge will close the ticket if you have used the word ‘fixes’ in your commit comment.
How to merge a pull request
How to merge a pull request
  • You can now safely delete the branch, both in the repository and in your local repository.

Deleting the branch

  • Delete your branch on the repository using the button on the right of the successful merge message.
  • You may also delete the working branch on your local copy. Note that this step is optional. However, if you wish to delete branches on your local machine, in your terminal window:
    • Go back to the master branch by typing git checkout master.
    • Update your local copy of the repository by typing git pull origin master
    • Delete the branch by typing git branch -d workingbranchname
    • Example: git branch -d issue-13390

Review Status

Last reviewed: August 8, 2018

Back to: Ontology Editing Guide