Ontology Editors Daily Workflow

From GO Wiki
Revision as of 02:28, 11 October 2021 by Pascale (talk | contribs) (→‎Review Status)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Updating the local copy of the ontology with ‘git pull’

  •  Navigate to the ontology directory of go-ontology: cd repos/go-ontology/src/ontology.
  • If the terminal window is not configured to display the branch name, type git status

You will see:

 * On branch [master] [or the name of the branch you are on]
   Your branch is up-to-date with 'origin/master'.
  • If you’re not in the master branch, type: git checkout master
  • From the master branch, type: git pull . This will update your master branch, with the files that are most current on GitHub, bringing in and merging any changes that were made since you last pulled the repository using the command git pull. You will see something like this:
  ~/repos/go-ontology(master) $ git pull
  remote: Counting objects: 26, done.
  remote: Compressing objects: 100% (26/26), done.
  remote: Total 26 (delta 12), reused 0 (delta 0), pack-reused 0
  Unpacking objects: 100% (26/26), done.
  From https://github.com/geneontology/go-ontology
  580c01d..7225e89  master     -> origin/master
  * [new branch]     issue#13029 -> origin/issue#13029
  Updating 580c01d..7225e89
  src/ontology/go-edit.obo | 39 ++++++++++++++++++++++++---------------
  1 file changed, 24 insertions(+), 15 deletions(-)
  ~/repos/go-ontology(master) $

Creating a new working branch with ‘git checkout’

  • You should never work on the master branch. When starting to work on a ticket, you should create a new branch of the repository to edit the ontology file.
  • Make sure you are on the master branch before creating a new branch. If the terminal window is not configured to display the branch name, type git status to check which is the active branch. If necessary, go to master by typing git checkout master.
  • Create a new branch: type: git checkout -b branch_name in the terminal window.
  • Naming branches: we recommend using the string ‘issue-‘ followed by the issue number. For instance, for this issue in the tracker: https://github.com/geneontology/go-ontology/issues/13390, you would create a branch as follows: git checkout -b issue-13390. Typing this command will automatically put you in the new branch. You will see this message in your terminal window:
   ~/repos/go-ontology/src/ontology(master) $ git checkout -b issue-13390
   Switched to a new branch 'issue-13390'
   ~/repos/go-ontology/src/ontology(issue-13390) $

Continuing work on an existing working branch and dealing with conflicts

  • If the terminal window is not configured to display the branch name, type: git status to check which is the active branch. If necessary, go to master by typing git checkout master.
  • Update master by typing git pull
  • To list existing local branches: type git branch -l
  • From the master branch, switch to your branch by typing git checkout [branch name]
  • Update the working branch with respect to the current version of the ontology (OPTIONAL) by typing git pull origin master
    • This step is optional because it is not necessary to work on the current version of the ontology.
    • Usually, changes in the working branch will be synchronized with master when you merge the pull request.
    • When you run the 'git pull origin master' command, 3 things can happen:
    1. All is OK: No action needed. You will get a message that the branch is 'Already up to date', followed by the normal prompt in your terminal window.
    2. ~/repos/go-ontology (issue-14892)]$ git pull origin master From https://github.com/geneontology/go-ontology * branch master -> FETCH_HEAD Already up to date. ~/repos/go-ontology (issue-14892)]$
    3. Some changes have been imported that need to be merged into your branch: You may get a message requesting that you merge the master branch into your branch (note that 'merge' is a type of 'commit'). Git opens a text editor window (dependent on how which text editor is set up by default; instructions here are for the vi Editor; https://www.ccsf.edu/Pub/Fac/vi.html) with the following comment:
         Merge branch 'master' of https://github.com/geneontology/go-ontology into issue-16057
         # Please enter a commit message to explain why this merge is necessary,
         # especially if it merges an updated upstream into a topic branch.
         # Lines starting with '#' will be ignored, and an empty message aborts
         # the commit.

      In the vi editor:

      1. Type i (for 'insert'). This will put your cursor in the first line ("Merge branch...") and allow you to edit.
      2. Type the comment (for example, 'merged master')
      3. Exit: type esp, then :q! to save changes and quit (q= quit, ! = save). You will get a message in the terminal that looks like this, followed by the normal prompt in your terminal window:
         Auto-merging src/ontology/go-edit.obo
         Merge made by the 'recursive' strategy.
         src/ontology/go-edit.obo                  | 117 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------
         src/ontology/subsets/gosubset_prok.yaml   |   2 +-
         src/ontology/subsets/mf_needs_review.yaml |   5 ++---
         3 files changed, 95 insertions(+), 29 deletions(-)  
    4. This merges and commit the merged branch (master + your branch).
      4. Type git push origin mybranch to push your changes to the remote repository.
    5. There are conflicts between the files that prevent git from merging
      • In some cases, in particular when there are inconsistencies between your branch and master, (for example when new terms have been created since your branch was last updated, or terms were merged or obsoleted), this step is necessary. These problems are usually reported by the Travis checks.
      • Sometimes when you do git pull origin master, the branch is found by git to have conflicts with master, which will lead to an error message in the terminal window (see next section).
      • The version of the ontology in the local branch may be in conflict with master. In this case, the terminal will return a message that looks like this (for a branch named 'mybranch'):
         Switched to branch 'mybranch'
         ~/repos/go-ontology/src/ontology (mybranch)]$ git pull origin master 
         From https://github.com/geneontology/go-ontology
         * branch              master     -> FETCH_HEAD
         Auto-merging src/ontology/go-edit.obo
         CONFLICT (content): Merge conflict in src/ontology/go-edit.obo
         Automatic merge failed; fix conflicts and then commit the result.
         ~/repos/go-ontology/src/ontology (mybranch|MERGING)]$
      • The problem with conflicts is that you cannot switch to another branch until they are resolved.
      • NOTE: This is a delicate operation. Ask someone to review the changes if you have any doubts.
      • To fix conflicts:
        • Open the go-edit.obo file using your favorite text editor.
        • Conflicts will be within tags like
         <<<<<<< HEAD
         >>>>>>> hash number
      For example: 
         <<<<<<< HEAD
         id: GO:0019836
         name: hemolysis by symbiont of host erythrocytes
         namespace: biological_process
         def: "The cytolytic destruction of red blood cells, with the release of intracellular hemoglobin, in the host organism by a symbiont. The host is defined as the larger of the organisms involved in a symbiotic interaction." [GOC:add, UniProtKB-KW:KW-0354]
         synonym: "haemolysis in host" EXACT []
         synonym: "hemolysin activity" RELATED []
         synonym: "hemolysis by symbiont of host RBCs" EXACT [CL:0000232]
         synonym: "hemolysis by symbiont of host red blood cells" EXACT [CL:0000232]
         synonym: "regulation of cytolysis of host cells by symbiont" EXACT [GOC:dph, GOC:tb]
         is_a: GO:0001897 ! cytolysis by symbiont of host cells
         is_a: GO:0052331 ! hemolysis in other organism involved in symbiotic interaction
         >>>>>>> e3d0e5afcd225e02722423ef17d2b2fa669a33f3
      • Usually, what is within the <<<<<<< HEAD and >>>>>>> hash, can be deleted.
      • Save your changes.
      • Commit by typing git commit -i * -m 'fixed conflict' go-edit.obo
      • Do a git diff master to make sure that the changes you have made are correct ('git diff' only will not do because you have already committed).
    6. Git thinks your branch is ahead of Master for files you didn't edit
      • Sometime git will detect that there are changes to files that you didn't modify. If that happens when you try to do a git pull, you will get the following message:
         error: Your local changes to the following files would be overwritten by merge:
         Please commit your changes or stash them before you merge.
         /repos/go-ontology/src/ontology(master) $ git status
         On branch master
         Your branch is behind 'origin/master' by 46 commits, and can be fast-forwarded.
          (use "git pull" to update your local branch)
         Changes not staged for commit:
           (use "git add <file>..." to update what will be committed)
           (use "git checkout -- <file>..." to discard changes in working directory)
      	modified:   imports/go_taxon_constraints.owl
      	modified:   imports/ro_import.owl
         no changes added to commit (use "git add" and/or "git commit -a")
      • In this example, git detects that you modified imports/go-taxon_constraints.owl and imports/ro_import.owl
      • If you did not want to modify those files, you can revert them back to the master copies by checking them out again.
      • Type git checkout -- imports/go_taxon_constraints.owl Then hit return and type git checkout -- imports/ro_import.owl
      • When you type these and hit return, nothing will happen.
      • If you type git status, you should now see that your working branch is clean and you can proceed with a git pull.
    7. If you aren't sure that what you are doing has worked
      • If you have any doubt about the changes you have made, delete the branch and start the work over. To do this, type git checkout master and delete the branch by typing, git branch -d mybranch.
      • If the branch is never pushed to origin, there is no impact of the main go-edit.obo file.
      • If the changes are as expected, you can go ahead with the push and the further steps.

      Loading, navigating and saving the Ontology in Protégé

      • Before launching Protégé, make sure you are in the correct branch. To check the active branch, type git status
      • Open the go-edit.obo file: Click on the ‘File’ pulldown. The first time, you will have to navigate to repos/go-ontology/src/ontology. Once you have worked on the file, it will show up in the menu under ‘Open’/’Recent’.
      • Click on the ‘Classes’ tab.
      • Searching: Use the search box on the upper right to search for a term in the ontology. Wait for autocomplete to work in the pop-up window.
      • Viewing a term: Double-click on the term. This will reveal the term in the ‘Class hierarchy’ window after a few seconds.
      • Launching the reasoner: To see the term in the ‘Class hierarchy’ (inferred) window, you will need to run the ‘ELK reasoner’. Go to menu: ‘Reasoner’ > select ELK 0.4.3, then click ‘Start reasoner’. Close the various pop-up warnings about the ELK reasoner. You will now see the terms in the inferred hierarchy.
      • After modification of the ontology, synchronize the reasoner. Go to menu: ‘Reasoner’ > ‘ Synchronize reasoner’
        • NOTE: The only changes that the reasoner will detect are those impacting the ontology structure: changes in equivalence axioms, subclasses, merges, obsoletions, new terms.
      • Saving: Use File > Save to save your changes.

      Committing, pushing and merging your changes to the repository

      Reviewing changes

      Changes made to the ontology can be viewed by typing in the terminal window: git diff. If there are changes that have already been committed, the full changes in the active branch relative to master can be viewed by typing git diff master.

      Git diff in the mac terminal window

      Committing changes to the local branch

      Changes can be committed by typing git commit -m ‘Meaningful message Fixes #ticketnumber’ go-edit.obo. The message ('-m') must be in either single or double quotes.

      • For example: git commit -m ‘hepatic stellate cell migration and contraction and regulation terms. Fixes #13390’ go-edit.obo
      • This will commit the saved changes to the go-edit.obo file in your active branch. The terminal window will show something like:
        ~/repos/go-ontology/src/ontology(issue-13390) $ git commit -m 'Added hepatic stellate cell migration and contraction and regulation terms. Fixes #13390' go-edit.obo
        [issue-13390 dec9df0] Added hepatic stellate cell migration and contraction and regulation terms. Fixes #13390
        1 file changed, 79 insertions(+)
        ~/repos/go-ontology/src/ontology(issue-13390) $ 
      • NOTES
        • The word ‘fixes’ is a magic word in GitHub; when used in combination with the ticket number, it will automatically close the ticket (in the go-ontology repo; if you are working on a ticket from a different repo the command will still go to the go-ontology repo and close a ticket there). In the above example, when the file is merged in GitHub, it will close issue number 13390. Learn more on this GitHub Help Documentation page about Closing issues using keywords.
        • ‘Fixes’ is case-insensitive.
        • If you don’t want to close the ticket, just refer to the ticket # without the word ‘Fixes’. The commit will be associated with the correct ticket but the ticket will remain open.
        • It is also possible to type a longer message than allowed when using the ‘-m’ argument; to do this, skip the -m, and a vi window (on mac) will open in which an unlimited description may be typed.

      Pushing working branch to origin

      To add your branch to the remote repository, type: git push origin mynewbranch.

      Example: git push origin issue-13390

      • TIP: Once you have pushed your changes to the repository, they are available for everyone to see, so at this stage you can ask for feedback.

      Creating a Pull Request for your branch

      How to view branches

      Compare (last sanity check before pull request)

      Click the green button Compare & Pull request on the right (see screenshot below). This will allow you to open a Pull Request, and determine if there are conflicts between your branch and master (if there are no conflicts, there will be a green check mark and the text 'Able to merge'.

      How to initiate a pull request

      The diff for your file is at the bottom of the page. Examine it as a sanity check.

      Git diff

      Open the pull request

      Click the green button Create pull request (see screenshot below). This will start to compare your branch with master and initiate the Travis checks to find potential errors in your branch (for example: missing IDs, missing references for definitions, is_a incomplete, logical errors).

      How to open a pull request

      Request a review (optional)

      You may now add comments and ask a colleague to review your pull request by selecting a reviewer in the ‘Reviewers’ list (e.g. @superuser1). The reviewer will be notified. Since the pull request is also a GitHub issue, the reviewer’s comments will show up in the comments section of the pull request, similarly to any other issue filed on the tracker.

      Automated checks

      • Wait for the Travis checks to complete (this can take a few minutes; see screenshot).
      Travis checks
      • If the Travis checks fail, you will see a red x at the left of each of the checks. Go back to your local branch, correct the reported errors, commit them to the branch, and push to origin. The Travis checks will then start automatically (since the pull request has already been generated).
      Travis Fail
      • If the Travis checks pass, you will see a green check mark at the left of each of the checks.
      Travis success

      Merging the Pull Request

      When (and only when!) the Travis checks are successful and if you're ready to integrate the changes in the main ontology file, merge the pull request, by clicking on the green Merge pull request button.

      • Note that the reviewer should go ahead and merge the pull request if s/he agrees with the changes.
      • Confirming the merge will close the ticket if you have used the word ‘fixes’ in your commit comment.
      How to merge a pull request
      • You can now safely delete the branch, both in the repository and in your local repository.

      Deleting the branch

      • Delete your branch on the repository using the button on the right of the successful merge message.
      • You may also delete the working branch on your local copy. Note that this step is optional. However, if you wish to delete branches on your local machine, in your terminal window:
        • Go back to the master branch by typing git checkout master.
        • Update your local copy of the repository by typing git pull origin master
        • Delete the branch by typing git branch -d workingbranchname
        • Example: git branch -d issue-13390

      Review Status

      Last reviewed: October 11, 2021

      Back to: Ontology Editing Guide