|
Size: 11401
Comment:
|
Size: 7193
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 8: | Line 8: |
| The objective of this phase of the user testing process was to gain feedback on the design of the site from close project collaborators. The rationale being to create a list of changes that can be implemented before the stakeholder meeting in June 2008. The methodology involved using a survey tool to guide participants through six scenarios that focused on the core functionality offered by the WW1 website. Participants were asked to use a project laptop to step through the survey with two researchers. Researcher-1 supported the participant and encouraged them to 'think aloud' while they undertook the tasks outlined in the scenarios. Researcher-2 took notes on the interview by added notes into a separate survey in parallel. When answering the survey questions researcher-1 reminded the participant the comments they'd made during the scenarios so that these could be included in their survey responses. The results from this process are discussed below. The outcome of this process is a set of requirement statements that can be fed into further development work. | The objective of this phase of the user testing process was to gain feedback on the design of the site from close project collaborators. The rationale being to create a list of statements that can be fed into the next phase of the website development process, and be implemented before the stakeholder meeting in June 2008. The methodology involved using a survey tool to guide participants through six scenarios that focused on the core functionality offered by the WW1 website. Participants were asked to use a project laptop to step through the survey with two researchers. Researcher-1 supported the participant and encouraged them to 'think aloud' while they undertook the tasks outlined in the scenarios. Researcher-2 took notes on the interview by added notes into a separate survey in parallel. When answering the survey questions researcher-1 reminded the participant the comments they'd made during the scenarios so that these could be included in their survey responses. The results from this process are discussed below. The outcome of this process is a set of requirement statements that can be fed into further development work. |
| Line 10: | Line 10: |
| attachment:WW1SurveyApril%202008.pdf | [attachment:WW1SurveyApril_2008.pdf WW1SurveyApril 2008.pdf] |
| Line 13: | Line 13: |
| TBC |
|
| Line 16: | Line 14: |
| TBC |
|
| Line 20: | Line 16: |
| === Quantitative === attachment:Q4b.jpeg |
'''Q4. Please rate the system in terms of how visually appealing you find the design.''' attachment:Q4b.jpeg |
| Line 23: | Line 18: |
| === Qualitative === * The banner title is just part of the decoration unless I look for the title. I could see from the list what it was though. * Like the left-nav, and the way it expands. * Found the text in the middle clunky, not aesthetically in keeping with the rest. * Content immediately interesting, and found something straight away. * Looked at homepage and at BROWSE BY DATE, which he did not remember seeing in other websites * Good use of images, which is appealing. * Clicked on THE COLLECTIONS link and browsed a bit. Also checked the FAVORITES page. Found the design good. * Looked at the main page only * In photographs in top margin, nothing that immediately strikes as being 'WW1' * Just looked at the first page. * Did not click on any links. * Images do not strike you as being related to the 1st world war as soon as you seen them * Very visul and attractive * Like the banner but simple text catalogue lets it down a bit * More info about the project on the main page. * Links horizontal and not vertical. * More info: i.e. Rather than "The Collections", we should be more specific === Requirements statements === * Provide pictorial and text that communicate very quickly what the archive is about Note: this question is not regarded as very important at this stage since the project has not got round to implementing the finer details of the site at this stage. |
=== Requirements === ''The banner title is just part of the decoration unless I look for the title. I could see from the list what it was though. '' |
| Line 45: | Line 21: |
| . ''In photographs in top margin, nothing that immediately strikes as being 'WW1' '' Generally initial impressions of the site were reported as being satisfactory. It is recognised however that the site is not finished and many of the comments relate to developments that are already planned e.g. text that explains what the website is about. The following requirements are worth consideration however: * (1) The web site banner is prominent on all screens and takes up a large proportion of space but does not clearly describe what the website is about. It is recommended that the banner be made smaller, and more simple with fewer images and text that states clearly what the website can be used for. * (2) The site needs to make it as easy as possible for users to find the most useful tools. For instance, the search box is likely to be the first port of call for many users yet it is hidden in the banner. Other users will turn straight to browsing yet their are currently no collections. For researchers the killer function is the compare manuscripts facility yet this is hidden in favourites. On the other hand the site emphasizes advanced search in the side bar but this is likely to used much less frequently (also most search engines put 'Advanced search' links next to search e.g. see Google. Similarly while participants thought 'browse by date' was a useful tool it is debatable whether this should be so prominent. In summary the requirement is for the team to create some use case processes for different types of user e.g. researcher, teacher, enthusiast, journalist, student, then map out the process by which they will use the site (a) on first encounter (b) to complete the core tasks that this type of user would undertake. This will help create the right emphasis in terms of site functionality.''' ''' |
|
| Line 46: | Line 27: |
| === Quantitative === Q6. Please rate the system in terms of how easy it is to FIND resources that you are interested in. attachment:Q6.jpeg |
'''Q6. Please rate the system in terms of how easy it is to FIND resources that you are interested in.''' attachment:Q6.jpeg |
| Line 49: | Line 29: |
| Q7. Please rate the system in terms of how easy it is to DOWNLOAD resources that you are interested in. | '''Q7. Please rate the system in terms of how easy it is to DOWNLOAD resources that you are interested in.''' |
| Line 53: | Line 33: |
| === Qualitative === * There was no main heading to search for images - had to go via Collection * Drop-down box w no options seems unnecessary * Clicking on heading returned confusing view (sorted on missing metadata first) * Search box 'hidden' * Download worked fine when just right-clicking (no instruction provided. * Download button didn't download) * Collections - contextual info. 1-2 sentences collections, resource types, people involved/events, immediately 1-1000 items, browse options - helped me to have brief info. * Download icon, expected. * Scroll bottom for download, found it. * I'd missed Download text at top. * Tried to browse the collection and mentioned that there was no contextual info, which was strange (scary was the word Alun used). * He needed info about the collections, resource types, people involved etc.). * Then used ADVANCED SEARCH to search for 'Rosenberg' and 'Poppies' and selected one of the items that came up from the search. * He looked for a download ICON that he would recognize from other sites. * Then noticed the DOWNLOAD text link at the bottom of the page and then finally the one at the TOP of the page, which he mentioned that it was not easy to see. Clicked on DOWNLOAD but then again he was not sure if anything was downloaded so he tried to use FILE>SAVE AS. * Thought that the image had downloaded, but it was not. Not clear. * Used SIMPLE SEARCH to find the poem. * Only person out of the first 4 that noticed the SIMPLE SEARCH box. * Was not sure how to download after clicking on the link. Asked: "is it done" when the image appeared on the screen. * Zooming a bit awkward (too incremental--see WOMDA website for example); downloading awkward. * Searching fine. * Advanced search and searched for Rosenberg (spelled correctly!). * Zoomed in one of the images. Zooming in too small increments. Way to zoom all the way with one click??? * Clicked on download. The was not sure how to download the image (asked if it was already downloaded). * Clicked on browse then immediately went to advanced search. * Typed Rosenburg rather than Rosenburg (Everett did the same) so maybe we should suggest corrections to common mistakes? * Found the document easily by searching for the poem title. * Tried to download by clicking on the image which just zoomed. * The tried to download it by right clicking on the image directly and selecting Save As WITHOUT clicking on the Download link. * Tried to browse the collection and sort by author rather then searching. * After a while he tried the advanced search as probably did not notice the simple search. * Rather than using the download link, the first reaction was to use the FILE > SAVE AS menu or adding the image to the favorites. * After Howard pointed out that he should use the download link he was not sure where the file would be downloaded to (which makes sense as he did not know that he should have Right clicked and Saved the image. * Missed Download button. * Happy with File Save As * Simple search is hidden Would like to browse by poet Advanced search too complicated Download wasn't download it was see image Instructions for right click (ctrl click on Mac needed) Reduce or get ris of banner when on other screens too many scroll downs * Do not have a link directly to the different poets. So browsing would not work. * Searching: Simple text box not easy to notice. Stuart went directly to advanced search. * Advanced search too complicated. (all of words, none of the words etc.) * Move 'Collections' in advanced search next to 'Criteria' and not below so no scrolling is needed. * In browsing, the banner makes the user scroll down all the time. * The RED border around the image thumbnail is a bit confusing. * The user thinks that clicking on that will zoom. * Back button takes you back to previous zoom levels. * We need to make the "back to results" link more obvious are Stuart did not see it until later. * Download link just opens the picture in the window BUT does not download it. * Have instructions "To download this image ...". Include MAC and Windows instructions. |
|
| Line 102: | Line 34: |
| With respect to finding resources the responses were split, with 4 saying further development was needed and 1 saying no extra development is need. We should remember that the participants were all experienced internet researchers, and less experienced users are likely to find the site difficult with respect to search. * (3) Make search box easier to find (e.g. not in banner). Recommendation: provide link to advanced search next to Search box as per commonly used search engines such as Google, then remove advanced search from side bar. * (4) Make it easier for user to download resources e.g. provide instructions on how to use File-Save-As option or right click. The current download link is more-or-less redundant. Also make it clear what will be downloaded e.g. just image, image plus link, image + metadata. * (5) Provide more options for download e.g. download as PDF It should be noted that some users will use 'browse' instead of 'search' but we were unable to observe this approach to finding resources because browse capabilities are currently not implemented. |
|
| Line 103: | Line 42: |
| === Quantitative === attachment:Q9.jpeg |
'''9. Please rate the system in terms of how easy it is to use to obtain prints that can be used in face to face meetings.'''''' attachment:Q9.jpeg ''' |
| Line 106: | Line 44: |
| === Qualitative === * Easy * Without seeing output I don't want to say NO MORE WORK NEEDED. E.g. will metadata and reference URL be printed. * Will it be a clean PDF style, or a page with http:// addresss bar etc. * Went directly to ADVANCED SEARCH and found all the items and found the item he was looking for after he narrowed the search down. * Thought that the image alignment for the Compound object was different than the singe items (because of the additional info on the left side of the image). * Found the printing easy to do but wanted to see how the page would come out of a printer (i.e. would the metadata be printed, what the styling would be etc.). * No problem * Used the SIMPLE SEARCH to find the image quickly. * Printed it easily. * More information about location of mss would be useful. * Used Advanced search and searched for "Daughters of War". * Too many results returned. Not apparent which one to choose. * Asked if possible to also have some notes appearing in the search interface. * Printing ok. * Print sizing; image to be printed appears pixilated. * Where is metadata? Seems like some duplication. * used Advance search ti find the item by using the poem Title. * Opened each of the 2 poems in a different window. * Picture seems pixelated when trying to print (image gets enlarged automatically). Asked where the metadata were. * Again Browsed and then used the Advanced Search. * No BACK link from print page. * Very easy...normal * Change print to printer friendly * Bit surpirsed when hit print to get 100% image * Would like to select multiple images and hit print only once * Banner again too obtrusive * Surprise when hit zoom in lost top of image had to use scroll arrows but they were different from red box navigation earlier * Stuart used the simple search immediately now that he knew it was there. * In search 23 seems the same as 3 (2 page Cover). * Check for double cataloguing or if search results appear twice. * Stuart 'jumped' after the image appeared on the screen after he clicked on PRINT. * No thumbnail when viewing Compound objects so zooming loses top of image and SCROLL ARROWS not easy to locate. * Maybe include 'multiple page' printing in Favorites (print all favorites). * Way to get back from the printing page to where you were before. * Maybe use the terminology to 'Printer Friendly' rather than Print. === Requirements statements === * Copy best practice for printing from other sites e.g. ide * Decrease size of banner so takes let screen real estate * Provide on screen help that makes it clear exactly what will be sent to the printer e.g. if metadata and headings included * Provide facility to print with metadata about manuscript * Provide help that tells user how to get back to the web site once they have printed * Check for double cataloguing * Make scroll keys need to be made more obvious * Include multiple page printing (perhaps off favourites) |
=== Requirements === In general this functionality worked well with only 2 participants saying more development work is needed. * (6) Make it clearer what will be sent to the printer e.g. any additional information that will be included with image such as hyperlinks or metadata |
| Line 152: | Line 49: |
| === Quantitative === === Qualitative === === Requirements statements === |
'''Q11. Please rate the system in terms of how easy it is to use the 'Favourites' functionality.'''''' attachment:Q11.jpeg ''' === Requirements === * Allow user to add items to favourites when search results are ticket over multiple pages * Allow items to be re-ordered through drag and drop which would include re-positioning multiple items at a time * Make it clearer what the 'move' functionality is for e.g. mostly to do with slide-show tool? * BUG: Adding multiple items to favourites where this batch of items includes one already added results in an error. Is there need to throw an error? Definitely do not stop all the other items being added as a result of the error |
| Line 156: | Line 57: |
| === Quantitative === === Qualitative === === Requirements statements === |
'''13. Please rate the system in terms of how easy it is to use the 'Advanced Search' functionality.''' attachment:Q%31%33.jpeg === Requirements === |
| Line 165: | Line 67: |
| ||<tablewidth="858px" tableheight="607px">'''Id''' ||'''Requirement''' ||'''Importance (i)''' ||'''Effort (e)''' ||'''Score (i x e)''' ||'''Notes''' || ||1 || || || || || || ||2 || || || || || || |
* Importance (i): Not important=1...5=Very important * Effort (e): Easy=1...5=Very difficult ||<tablewidth="858px" tableheight="607px">Id''' ''' ||Requirement''' ''' ||Importance (i)''' ''' ||Effort (e)''' ''' ||Score (i x e)''' ''' ||Notes''' ''' || ||1 ||Change website banner. ||4 || || || || ||2 ||Website emphasis ||4 || || || || |
Overview of methodology
Phase 1: May 2008
The objective of this phase of the user testing process was to gain feedback on the design of the site from close project collaborators. The rationale being to create a list of statements that can be fed into the next phase of the website development process, and be implemented before the stakeholder meeting in June 2008. The methodology involved using a survey tool to guide participants through six scenarios that focused on the core functionality offered by the WW1 website. Participants were asked to use a project laptop to step through the survey with two researchers. Researcher-1 supported the participant and encouraged them to 'think aloud' while they undertook the tasks outlined in the scenarios. Researcher-2 took notes on the interview by added notes into a separate survey in parallel. When answering the survey questions researcher-1 reminded the participant the comments they'd made during the scenarios so that these could be included in their survey responses. The results from this process are discussed below. The outcome of this process is a set of requirement statements that can be fed into further development work.
[attachment:WW1SurveyApril_2008.pdf WW1SurveyApril 2008.pdf]
Phase 2: June 2008
Phase 3: October2 2008
Findings: Phase 1
First impressions
Q4. Please rate the system in terms of how visually appealing you find the design. attachment:Q4b.jpeg
Requirements
The banner title is just part of the decoration unless I look for the title. I could see from the list what it was though.
In photographs in top margin, nothing that immediately strikes as being 'WW1'
Generally initial impressions of the site were reported as being satisfactory. It is recognised however that the site is not finished and many of the comments relate to developments that are already planned e.g. text that explains what the website is about. The following requirements are worth consideration however:
- (1) The web site banner is prominent on all screens and takes up a large proportion of space but does not clearly describe what the website is about. It is recommended that the banner be made smaller, and more simple with fewer images and text that states clearly what the website can be used for.
(2) The site needs to make it as easy as possible for users to find the most useful tools. For instance, the search box is likely to be the first port of call for many users yet it is hidden in the banner. Other users will turn straight to browsing yet their are currently no collections. For researchers the killer function is the compare manuscripts facility yet this is hidden in favourites. On the other hand the site emphasizes advanced search in the side bar but this is likely to used much less frequently (also most search engines put 'Advanced search' links next to search e.g. see Google. Similarly while participants thought 'browse by date' was a useful tool it is debatable whether this should be so prominent. In summary the requirement is for the team to create some use case processes for different types of user e.g. researcher, teacher, enthusiast, journalist, student, then map out the process by which they will use the site (a) on first encounter (b) to complete the core tasks that this type of user would undertake. This will help create the right emphasis in terms of site functionality.
Searching for resources
Q6. Please rate the system in terms of how easy it is to FIND resources that you are interested in. attachment:Q6.jpeg
Q7. Please rate the system in terms of how easy it is to DOWNLOAD resources that you are interested in.
attachment:Q7.jpeg
Requirements
With respect to finding resources the responses were split, with 4 saying further development was needed and 1 saying no extra development is need. We should remember that the participants were all experienced internet researchers, and less experienced users are likely to find the site difficult with respect to search.
- (3) Make search box easier to find (e.g. not in banner). Recommendation: provide link to advanced search next to Search box as per commonly used search engines such as Google, then remove advanced search from side bar.
- (4) Make it easier for user to download resources e.g. provide instructions on how to use File-Save-As option or right click. The current download link is more-or-less redundant. Also make it clear what will be downloaded e.g. just image, image plus link, image + metadata.
- (5) Provide more options for download e.g. download as PDF
It should be noted that some users will use 'browse' instead of 'search' but we were unable to observe this approach to finding resources because browse capabilities are currently not implemented.
Printing
9. Please rate the system in terms of how easy it is to use to obtain prints that can be used in face to face meetings. attachment:Q9.jpeg
Requirements
In general this functionality worked well with only 2 participants saying more development work is needed.
- (6) Make it clearer what will be sent to the printer e.g. any additional information that will be included with image such as hyperlinks or metadata
Favourites
Q11. Please rate the system in terms of how easy it is to use the 'Favourites' functionality. attachment:Q11.jpeg
Requirements
- Allow user to add items to favourites when search results are ticket over multiple pages
- Allow items to be re-ordered through drag and drop which would include re-positioning multiple items at a time
- Make it clearer what the 'move' functionality is for e.g. mostly to do with slide-show tool?
- BUG: Adding multiple items to favourites where this batch of items includes one already added results in an error. Is there need to throw an error? Definitely do not stop all the other items being added as a result of the error
Advanced search
13. Please rate the system in terms of how easy it is to use the 'Advanced Search' functionality. attachment:Q%31%33.jpeg
Requirements
Compound objects
Quantitative
Qualitative
Requirements statements
Analysis
Phase 1: May 2008
- Importance (i): Not important=1...5=Very important
- Effort (e): Easy=1...5=Very difficult
Id |
Requirement |
Importance (i) |
Effort (e) |
Score (i x e) |
Notes |
1 |
Change website banner. |
4 |
|
|
|
2 |
Website emphasis |
4 |
|
|
|
3 |
|
|
|
|
|
4 |
|
|
|
|
|
5 |
|
|
|
|
|
6 |
|
|
|
|
|
7 |
|
|
|
|
|
8 |
|
|
|
|
|
9 |
|
|
|
|
|
10 |
|
|
|
|
|
11 |
|
|
|
|
|
12 |
|
|
|
|
|
13 |
|
|
|
|
|
14 |
|
|
|
|
|
15 |
|
|
|
|
|
16 |
|
|
|
|
|
17 |
|
|
|
|
|
18 |
|
|
|
|
|
19 |
|
|
|
|
|