Few weeks ago I posted Shaping the scenario tasks, where I shared a “draft” of the scenario tasks and promised to come back with some improvements as the “final version” of these tasks. And here I am, with quite a lot changes and improvements actually.
We decided to elaborate more the scenarios since they did not cover much of each application. We narrowed down the number of applications to two, and started looking them in more details. I also contacted Photos and Calendar, developers and designers, to ask what they’d be interested in having tested. They were really helpful and suggested some new features to test with like: non-destructive editing in Photos, alarms and new edit dialog in Calendar.
There is a lot to cover in both applications but I tried to combine these suggestions and keep the tasks as short and straightforward as possible since the time is limited.
1. You just got back from your trip to Thailand. You want to show your friends all the pictures that you captured there but you notice that they are all mixed with the other pictures that were previously there. To avoid the confusion you decide to create an album that contains only pictures from your trip and name it “Thailand trip”.
2. As you are showing the pictures of the album you just created, you notice a picture that you’d like to share on your social media accounts. You “favorite” that picture, in order to access it more easily later on.
3.While looking trough all the photos, you notice that two of the photos look very similar, so you decide to delete one of them.
4.After showing your friends all the pictures of the “Thailand trip” album, you revisit your “favorite” picture since you want to edit it before sharing on social media.
5.You start by cropping the picture to make it look smaller.
6.Than you want to play with colors a little bit. You want to make the picture brighter and lower its saturation but leave the contrast as it is.
7.Now you want to enhance your photo. You sharpen it a little bit and leave the denoise as it is.
8.And finally, you choose a filter named “Calistoga” as a final touch.
9.You want to apply these changes, so the photo remains as it is.
10.You really like the way that the picture turned out and you decide to set it as a “Background picture”.
Calendar: (I plan to use GNOME Continuous image to test with Calendar)
1.You want to have all your work related activities in your calendar but you don’t want them to be mixed with other activities, since you have a lot going on lately. In order to keep your events organized you decide to create a calendar only for work related activities. Name the calendar simply “Work” and make the activities appear in purple color.
2.You have a meeting with your boss today. You want to create an event named “Meeting for work”, the meeting starts at 2:00pm until 4:00pm. Make sure to put it in the “Work” calendar that you created earlier.
3.Set a reminder 10 minutes before the meeting starts, just in case you forget that you have a meeting.
4.You plan to throw a big celebration party for your birthday next year. You want to check what day of the week will your birthday be, while wishing for Friday so everyone can show up! Check the calendar and tell me what day of the week your birthday will be in 2017.
5.You have already created an event named “GUADEC” for the GUADEC conference you are attending next month but you forgot the exact date. Can you please search for it and then tell me the date.
6.You don’t want to move across different calendars, creating same events multiple times. So, you try to connect this calendar with google calendar (or any other online account you use).
Deciding the number of testers
Before starting any usability test, we should think about how many testers do we need to get good results?
I read an interesting article regarding this, which summarizes that the best results come from testing no more than 5 users and running as many small tests as you can afford.
According to this chart, as you add more users, you learn less because you will keep seeing the same things again and again.
With five testers, you can uncover most of the usability issues but this can happen only if you are iterative in your usability testing, meaning that you do a design, test it, refine the design, test it again, etc.
Since I am not going to test iteratively, I will need more than five users in order to achieve useful results. The goal is to find ten testers.
Setting up the testing environment
I have set up separate test accounts for testers and I also prepopulated each account with the sample files needed.
For example, I downloaded some pictures so that the tester can accomplish tasks like: “You just got back from your trip to Thailand. You want to show your friends all the pictures that you captured there…”, I also created some events so that the tester doesn’t need to create lots of tasks(that would be time consuming): “1.You want to have all your work related activities in your calendar but you don’t want them to be mixed with other activities…”.
I installed an application to do a “screencast” record of the testing process. As I mentioned earlier, that will come in handy later during my analysis.
I did a “dry run” of the test, to make sure I know what to do, and that everything is there for the test. This is the last opportunity to identify anything that’s missing on the scenario tasks or in the setup.
Ready for testing! I will update you on how it went in the next post…