User Acceptance Testing

In the final phase of this project our developed application was assessed by users representing three user groups. The following describes the test users; tasks each user group performed with this application, and the feedback and outcome from the user acceptance test.

Test user (groups)

User 1 is a postdoc from the department of Computing and Information Systems at the University of Melbourne.  This user is a representative of the general Bioinformatician that has  both basic biological knowledge and adept programming skills.

User 2 is a postdoc from the University of Queensland, representing biologists and clinicians who want to easily integrate their own research data with datasets from multiple data sources and analyse the data. This user group are generally researchers who have very limited or no programming skills.

User 3 a system administrator from the University of Queensland. This user represents the GVL Galaxy site administrator who works primarily on the server side, managing the Galaxy workflow system, environment and data libraries.

Tests performed by users

User 1 - the main task for this user group is to design and generate scientific workflows/protocols using the Galaxy workflow system. Then publish these workflows with the   research community so that other researchers can apply it to their own data.

User 2 - the main task or this user group is to reuse existing workflows for their own research, modify existing workflows and create new workflows. Additional important features for this user group include the: 

• ability to publish and cite workflows.
• ability to assign a Digital Object Identifier (DOI) to workflows .
• ability to rerun shared workflows and cite them using the DOI in their publications.
• ability to generate important datasets for the community and share them through Galaxy.

User 3 - the main task or this user group is to manage and update the data libraries shared with the community via the Galaxy workflow system. This user works primarily on the server side on tasks such as configuring user, tool or data repository access.

Testing outcomes

The Collection Manager has been introduced to User 1 via video conference and in person to Users 2 and 3. All functionalities have been presented to all users.

User testing identified some issues from the Collection Manager, the main issues were:

• concurrent access to data records in the Collection manager result in database errors when saving the record
• email notification functionality

All issues have been now been fixed.

Individual User Experiences

User 1 found the Collection Manager interface very user friendly and easy to use. User 1 comments that the application will be another means for them to promote their Galaxy workflows in the near future. This will increase awareness of their workflows further in the research community. An account has been created for this user on the production server and they have published two workflows to RDA using the Collection Manager.

User 2 found the Collection Manager impressive. In their opinion it is a very useful tool to easily share and promote workflows with the research community. An account has been created for them on the production server and this user will spend a couple of days testing the Collection Manager to see how it could be merged with their current working style.

User 2 expressed further interest in the DOI minting option provided by the Collection Manager. This feature was identified as a major advantage to using the application, which is not provided by the Galaxy workflow system. Further discussion with User 2 needs to take place to evaluate: when should a workflow be assigned a DOI during the publishing process.

User 3 who has a background in teaching Web Interface design found the Collection Manager Interface very easy to understand and use. They also found value in using this interface to help the public identify which Galaxy instances are available on the web and what data libraries and workflows they host. Many public users do not know where to go to use specific tools with specific data. Having a general repository indexed by Google will ease this process. In addition, User 3 raised a few points for the web-interface that include providing more help text for fields such as ANZSRC and Rights when users are creating a Data record. These changes  are currently being addressed.