View source for GO survey 2009/10

We need to perform a survey for the next GO grant application. This page is for collecting ideas and recording progress.
<br>
==Personnel==
Jane, Val and Jennifer

==Recipients==
*Send to two groups separately:
*#People who have submitted a q to gohelp
*#A group of random bench biologists
** I've contacted EBI outreach for advice on how to contact a group like this and they're going to try and parse a list of addresses for us
[[Category:Advocacy and Outreach]]

==Ideas==
*The discussion of this at the 2009 Cambridge GOC meeting is in the [[GOC_Meeting_Minutes_September_2009#AmiGO_status_.28Chris.29|meeting minutes]]
*Short - maybe 15 questions or less? [Jane]
*Stick to a small number of topics, namely: [Jane]
*#Their perceptions of GO
*#How they use GO
*#What they'd like wrt tools
*Ask what their priorities would be for GO [Jen]

====Jen's first thoughts:====

 1) Have you used GO?
 
 Yes - go to rest of questionnaire
 No- questionnaire ends (here you should have a stub to find out why they haven't used it, never heard of it or  not relevent to their work-val)
 
 2) Were you entirely satisfied with it?
 
 Yes - end (I would put Q 2 before 3, this way you know what the main use is for the people who are satisfied-val)
 no - continue
 
 3) What do you use GO for?
 
 options and text box (here I would have a list of the most common uses (selct as many as apply) and an "other", easier to collate val)
 
 4) How would you prioritise the work that needs done to make the GO better?

(if this question is aimed at biologists, it will need to be much more explicit, but probably not so detailed, only "superusers" will be able to distiunguish between some of these val)
 
 Add ontology coverage. 
 Fix ontology errors.
 Add annotation coverage.
 Fix annotation errors.
 Improve Consortium tools (AmiGO or OBO-Edit).
 Improve documentation and help guides. 
 Provide easier feedback options.
 
 (Put numbers 1 - 6 alongside to indicate priorities.)
 
 5) Please note any particular areas of biology that you feel need improvement in the ontology structure.
 
 text box (Ask them to supply the GO ID which most closely relates to the area which needs work, eg. translation (GO:0006412), heart development (GO:0007507) 
 
 6) Please note any aspect of annotation coverage or quality that you feel needs attention. 
 
 text box

(I would merge 5&6, a biologist will not be able to distinguish between deficiencies in annotations or ontology at this level)


 
 7) Please note any features that you feel are lacking from AmiGO or OBO-Edit. 

(I doubt that a single biologist has used OBO edit, I think we should only ask about the web tools- val)
 
 text box
 
 8) Which areas of documentation do feel most urgently need improved?
 
 text box
 
 9) Are you aware of the following and their uses:
 
 a) The 'NOT' qualifier
 b) GO slims
 c) [other hard stuff that we want to check awareness on...]
 
 10) Do you use any other ontologies or ontology tools?
 
 text box



====Vals first thoughts====
I think we should first figure out exactly what we want to know from our users. I have a feeling the questions need to be muuch more general.

We also need to know more about who the users are. It might be important to find out what aspect of biology they are working in? are they a student or a PI? are they an informaticaian for a wet lab or working directly at the bench, are they working on high throughput projects or single genes? which organism do they work on mainly? There is a big differnece between someone complaining that the pombe coverage isn't good enough to someone working on "frog". If we don't know who the users are, then we won't be able to interpret the answers.

Might also want to know what their main exposure to GO is through (model organims database, public database (uniprot Ensembl), etc. We can't assume that the users main use of GO is though the comsortium website, and this may affect their perception of the annotation and the coverage, and the tools)


Fewer questions with free text answers (I had 3 in my last survey and that was a nightmare to collate, mainly because people complained about the same thing in every box!). Just give people the opportunity at the end to mention any problem which was not covered by the survey which will give you a good indication where your "holes are"

====Jane's first thoughts====

I agree with Val about being general with the questions - I think for the groups we're thinking of targeting these questions might be a bit in-depth. This is what I came up with:

 What they do, what area of biology, are they a tools developer/bioinformatican or bench researcher? Do they do microarrays/large scale analyses?

 Have they heard of GO? In what context? What do they think it is? What resources does it provide? 

 Have they used GO? If yes, through what tool/interface? Were they able to answer the questions they wanted to? What did they want to do that they couldn't 
 do? Did they try different tools? Which ones? Did they ask for help? How did they find the response? What would have made the process easier? Did they look 
 at the documentation?

Been wondering whether the two groups should get different survey questions...I think yes, with some questions in common so that we can make comparisons.


====Email from Suzi====
[snip]
 > So first, is this targeting of largely biologists sufficient, and consistent with our grant aims?

Yes.

We need to offer something that helps us understand this question -- what do you want to use GO for? (as well as what are you currently using GO for). Our biggest issue is maintaining relevance to the research biologists (a much larger and pragmatic group than say, NLP researchers). I would suggest several complementary approaches. First multiple choice questions (What GO tools do you use for expression analysis: A or B or C or D or Enot applicable). These could be based on what we find in the help messages and on the GO tools page. Complementing this would be an open question about what tools do you use now? and What tools would you like to have?. Similarly we could ask them what frustrates them the most: A. missing annotations, B. software bugs, D. non-intuitive user interface, and other problems found by culling the help list. Does this give you a general idea?
But in addition to the survey I think we also need to carry out a small number of in depth interviews with selected (perceptive, thoughtful) individuals. Both watching people trying to do some task, but also just asking questions and listening. Maybe Mike Tyers might be open to doing this, or he could suggest some names, or we could find some ideas from current publications.

 > With respect to the questions themselves - were there any recommendations from the last grant application that might be applicable? I know they criticized the
 survey - was there anything constructive they said about what they would like to have gleaned from our user community?
 
No, their complaint was more about the sampling itself. By sending it simply to people we already know (i.e. the go-friends list and the MOD mailing list) we had biased the sampling. They wanted us to take a more representative sampling of the broader research community. For example, the full range of people who might be be surveyed if we simply walked into any given biological research building and interviewed absolutely everyone in it from top to bottom (limited to scientists I suppose, the janitor wouldn't be necessary), but PIs at all stages of their careers, post-docs, and graduate students; high-throughput projects and small; and a full spectrum of investigation strategies from ChIP-Seq to biochemical assays. This would include both groups whose experiments result in making GO assignments, and those who simply need to use GO assignments.

In addition, I seem to recall they wanted a larger sampling size. I'm trying to remember how many responses we had... ah, it was 1474 people. I think that is probably as good as we can get, but it does set a minimum for this round.

 > And finally, did the SAB provide any feedback that might be useful for designing the survey from the Cambridge meeting?

We talked a bit about it on the GO top meeting this morning and the gist of it is that we want to do a bit of course correction in order to assure we stay focused on our primary motivation, which is to facilitate scientists and their research in understanding biological functions.