GNOME Usability Test Results (Part 2)

Here I am with the last part of analysis 🙂
On part 1 I focused more on visualizing the results, demographics and talked more about the methodology I used for testing. Later we took a closer look on how testers performed on every task given, using a heat map.

Heatmap

This time I am coming with more details “pulled” from the heat map. I find it is best if I break down the analysis into two sections: ” What went well?” And” What were the challenges?”. In these sections I’ll include the tasks that performed well and the tasks that were more challenging and give a brief description for each of them.

In the “What went well?” part I am going to analyze the “cool” rows. Basically the green and yellow boxes, which represent the “easy” and the “somewhat difficult” tasks.

In the “What were the challenges?” section I am going to talk about the “hot” rows. That includes the orange, red and black boxes. These colors represent the tasks that were “difficult” to accomplish.

 

What went well?

Photos:

Edit a picture(task 4): After showing your friends all the pictures of the “Thailand trip” album, you revisit your “favorite” picture since you want to edit it before sharing on social media.

This task turned out to be straightforward and intuitive for all the participants. They said that the icon used for editing associated with its function and it is also familiar to them from other photo editing applications that use similar icon.Screenshot from 2016-08-24 20-22-55

Crop the picture(task 5): You start by cropping the picture to make it look smaller.

We can notice some yellow boxes in this task and that is because of the confusion caused by the sidebar  “aspect ratio” options for cropping the picture. Participants thought that they can adjust the size of the picture only using the ratios in the sidebar and seemed to be confused. After that, they tried clicking the picture and adjusting from there.Screenshot from 2016-08-24 20-42-56

Change the colors(task 6): Than you want to play with colors a little bit. You want to make the picture brighter and lower its saturation but leave the contrast as it is.

Enhance the picture(task 7): Now you want to enhance your photo. You sharpen it a little bit and leave the denoise as it is.

Apply filter (task 8): And finally, you choose a filter named “Calistoga” as a final touch.

All participants considered the editing process very simple. All testers managed to complete these three tasks without any difficulties. They were really easy to follow once they’ve found the editing icon.

Apply changes(task 9): You want to apply these changes, so the photo remains as it is.

I think it would be a better idea if I merged this task with of the previous task because most of the testers applied the changes before, so this task was not applicable in most cases.

Regarding the applying the changes process, some of the participants said that they expected an “Apply” or “Save changes” button instead of “Done”. They also expected to find the “Done” button in the bottom of the screen.

Done

Calendar:

Set an alarm(task 3): Set a reminder 10 minutes before the meeting starts, just in case you forget that you have a meeting.

After finding a way to create the new event, this task was really easy to accomplish. No participant encountered difficulties on this task.

Search for an event(task 5): 5.You have already created an event named “GUADEC” for the GUADEC  conference you are attending next month but you forgot the exact date. Can you please search for it and then tell me the date.

This task was also intuitive for all the participants, they found the search icon to be very familiar. Screenshot from 2016-08-26 01-19-40

What were the challenges?

Photos:

Create an album(task 1): You just got back from your trip to Thailand. You want to show your friends all the pictures that you captured there but you notice that they are all mixed with the other pictures that were previously there. To avoid the confusion you decide to create an album that contains only pictures from your trip and name it “Thailand trip”.

This task was definitely the most challenging to accomplish. All participants went through similar steps to complete this one.

First they clicked “Albums” on the view switcher and then tried clicking “Name your first album”, thinking that this option would create a new album.

AddAlbum.png

That didn’t work, so they tried right clicking on the screen which also didn’t work. After that, they looked for a “Create an Album” option in the “Application menu” and could not find it. Some participants even searched for “Create an Album” in the search bar.

Finally, they tried right clicking on “Photos” and found the “Add to Album” option on the bottom of the screen.

Another confusing thing for all testers was the “Organize” window. After they selected the desired pictures and clicked “Add to Album”, the “Organize” window would appear, giving two options: “OK” and “Add”. They immediately clicked “OK” and were not sure if they’ve created the album or not.

Organize

They went back to “Albums” on view switcher and found out that the album was not saved. So they started the process again, using the “Add” option this time. The participants also complained about not having a “Cancel” button on this window.

Favorite a picture(task 2): As you are showing the pictures of the album you just created, you notice a picture that you’d like to share on your social media accounts. You “favorite” that picture, in order to access it more easily later on.

This task was pretty easy to accomplish. The star icon was noticeable for nearly all testers. In some cases participants completed this task by using the selection mode. Although some insisted to right click on the picture and expected a menu where they could click “Favorite”.

Favorite-selection.png
Favorite a picture using selection mode

Favorite.png

Delete a picture(task 3): While looking trough all the photos, you notice that two of the photos look very similar, so you decide to delete one of them.

Set a background picture(task 10): You really like the way that the picture turned out and you decide to set it as a “Background picture”.

On both these tasks, most testers tried clicking on the picture first, then right clicked on it and expected a menu where they could find these options. It made more sense to them to have a special menu that they could use to operate with the picture instead of using the header bar menu.

dl.png

Calendar:

Add a new calendar(task 1): You want to have all your work related activities in your calendar but you don’t want them to be mixed with other activities, since you have a lot going on lately. In order to keep your events organized you decide to create a calendar only for work related activities. Name the calendar simply “Work” and make the activities appear in purple color.

In general, all participants were confused by the term “Add a new Calendar” or “Create a new Calendar”. They would expect for an option like “Add a new Calendar” to be located somewhere in the application menu or in a “more obvious” location and not under the “Calendar Settings”. Some even created a new event instead of adding new calendar. Almost all testers were not sure which option to choose for adding the calendar.

Calendar11

Testers expected feedback after they named the calendar. They complained about not having an “Apply” button since they were not quite sure if they added the new album.

 

Calendar

Create an event & set the time(task 2): You have a meeting with your boss today. You want to create an event named “Meeting for work”, the meeting starts at 2:00pm until 4:00pm. Make sure to put it in the “Work” calendar that you created earlier.

This task is derived from the first task, which was challenging to accomplish for most of participants. Therefore this task also caused a little confusion.

After adding a new calendar, testers thought that in order to create an event they need to somehow switch from the current view of calendar to the new calendar they’ve created, only then they’d have the chance to create a new event.

Two participants tried to use “Manage your calendars” to create the event. They tried leaving checked only the calendar they created and unchecking the other calendars, thinking that would “switch” the calendars.

Calendar3

One participant went back to “Calendar Settings” and clicked on the “Work” calendar he created earlier expecting a “Create an Event” window to appear.

Calendar4

After exploring a little, they found out few ways to perform this task.

Nearly all participants used the “quick add” popup to create the events, and found it very clarifying since they could pick the calendars they created and add the event there.

Calendar5

Just a few of them used the Application menu and the header bar menu to create the events.

Calendar6Calendar7.png

 

Setting a reminder was really easy for all the participants.

Look for a specific date(task 4): You plan to throw a big celebration party for your birthday next year. You want to check what day of the week will your birthday be, while wishing for Friday so everyone can show up! Check the calendar and tell me what day of the week your birthday will be in 2017.

For some reason, not many tester used the view switcher to skip to the next year. They used the arrows to change the current date and some tried to scroll through the months.Calendar 9Add an online account(task 6): You don’t want to move across different calendars, creating same events multiple times. So, you try to connect this calendar with google calendar (or any other online account you use).

The first thing some testers did to is choosing the “synchronize” option on header bar menu.

Calendar9After the testers found a way to add an online account and finished this task successfully, they wanted to remove the account.

Calendar10

The “+” and “-” signs were not that obvious to indicate their meanings. They would rather prefer something like “Add an Account” and “Remove an Account” buttons.

Few suggestions

After putting together these results I was thinking, what would I do to improve the usability of Calendar and Photos. So I thought I’d share a few suggestions 🙂

Photos:

One thing I noticed while the participants started the “Create an Album” task is that as soon as they got into the selection mode the blue header bar grabbed their attention. They expected the “important” options like adding an album, printing or deleting to be located there, so they totally ignored the Action bar.

Photo1.png

I suggest adding a color to the Action bar, that way it would be noticeable enough.

Same problem with the “Organize” window. The “OK” blue button draws too much attention, so all testers would click “OK” instead of “Add”.

Organize
Calendar:

I use Calendar daily for quite some time now, so I can talk from a users perspective for a moment. I’d love to see the events highlighted instead of the dots underneath the numbers. That would give a clear look of the month.

Screenshot from 2016-08-26 15-39-59

 

This is it for my usability testing results and analysis, I hope it will help improve usability of Calendar and Photos!
Thanks for reading ^_^

Advertisements

3 thoughts on “GNOME Usability Test Results (Part 2)

  1. Wow. Software quality assurance in action. It works. Debian and GNOME to all FOSS projects could learn a lot from this exercise. Instead of complaining how come Linux can’t touch OS X [even though GNOME has done well at copying large bits of the front end (not much on the back end)] it’s because they have very little desire to merge the consumer with the developer and make it `just work.’

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s