Usability studies: Word importer and EIP-style module-editing interface

Last week we completed usability tests on the Word importer and the EIP-style module-editing interface with 4 different users (plus Chuck and Elvena as guinea pigs (thanks :-) )), all of which were successful in that they showed us a number of areas where the interface needs improvement that many of us developing it probably hadn't even considered.

Before getting in to the results of the test, or recommendations that come out of it (which Chuck and I will put together after reviewing our notes), here is a list of the actual tasks, with the steps we expected that users might take in order to complete the tasks.

A note: for half of the users, we hid the "Preview" tab, and for the other half, we hid the "Roles" tab, a removal being considered in order to reduce the number of tabs cramming the interface.

------------------

First, we briefly explained Connexions and the concept of modules. We told them modules are written in XML (without explaining what is is), and that we were testing our online editing feature. They were given a MS Word file (about SPAM, the food) and told they were going to create a new module with it. After logging them in with a guest account, we asked them to complete the following tasks:

Create a new module with the title “Cooking with SPAM” from the Word doc.

Expected steps:

1. Go to ''My Workspace''
2. Select ''Module'' and click ''Create New Item'' OR click ''Create New Item'' then ''Add'' under Module
3. Agree to the license
4. Take defaults for language selection
5. Rename the module to ''Cooking with SPAM''
6. Click ''Edit'' tab
7. Click ''Import''
8. Browse to file on Desktop
9. Click ''Import''

Preview the newly created module.

Expected Steps:

1. Click ''Online'' OR ''Preview'' tab then ''Online Preview''

Explain that for editing existing modules there is an XML editor named Authentic.

Export the document for use with Authentic.

Expected Steps:

1. Click the browser ''Back'' button to return to editing
2. Click ''Export''
3. Save file to Desktop

Find the section named The history of SPAM and add a new paragraph immediately before it that says:

SPAM is the greatest food ever!

Expected Steps:

1. Click the ''Add'' button
2. Write some text
3. Click the ''Save'' button

You change your mind and decide to delete the new paragraph

Expected Steps:

1. Click the paragraph to be deleted
2. Click ''Delete''
3. Click the ''OK'' to delete the paragraph

Start writing a new paragraph, then change your mind in the middle of editing it and decide not to include it

Expected Steps:

1. Click the ''Add'' button
2. Write some text
3. Click the ''Cancel'' button

You decide that you would like the recipe instructions to be in one paragraph instead of three. Fix this on-line.

Expected Steps:

1. Click one of the paragraphs for editing
2. Cut the text of the paragraph
3. Click ''Save''
4. Click ''OK'' to delete the now-empty paragraph
5. Click the other paragraph for editing
6. Paste in the text
7. Click ''Save''
8. Repeat.

You decide to add a bullet list of future topics to the bottom of the module.

• More SPAM recipes
• SPAM trivia
• SPAM nutritional facts

Expected Steps:

1. Select ''Bulleted list'' from the dropdown
2. Click the ''Add'' button
3. Edit the first item in the list
4. Add items to the list
5. Click the ''Save'' button

You decide you want to co-author this with me (Max Starkenburg). Add me to the list of authors

Expected Steps:

1. Click the ''Roles'' tab OR Click ''Metadata'' then ''Edit...'' link
2. Enter ''Max Starkenburg'' in the search box
3. Click ''Search'' OR hit ''Enter''
4. Check the box next to ''Max Starkenburg''
5. Click ''Add People''

Ask the user why do they think John Doe showed up in a separate table.

You want to add a picture of a can of SPAM at the top of the document, with a caption reading ''SPAM is delicious''. The image is on the desktop (spam.jpg).

Expected Steps:

1. Select ''Figure'' from the dropdown list
2. Click the ''Add'' button
3. Edit the XML text with correct/valid information
4. Click the ''Save'' button
5. Click the ''Files'' tab
6. Upload the correct file

Remove the figure caption

Expected Steps:

1. Click on the figure
2. Remove the <caption /> tag
3. Click the ''Save'' button

Emphasize the word “delicious” in the first paragraph.

Expected Steps:

1. Click on a paragraph
2. Put <emphasis /> around a portion of text
3. Click the ''Save'' button

Give credit to the “Amazing SPAM Homepage” by providing a link to their web site

http://www.cusd.claremont.edu/~mrosenbl/spam.html

Expected Steps 1:

1. Click the ''Links'' tab
2. Add the link information to the form
3. Click ''Add''

or Expected Steps 2:

1. Click on a paragraph
2. Add the necessary XML for a link
3. Click the ''Save'' button

Add a definition of SPAM after the first paragraph, using the meaning provided by the Amazing Spam Homepage:

“SPAM: a type of canned, processed pork luncheon meat made by Geo. A. Hormel and Company (Hormel).”

Expected Steps:

1. Click the ''Switch to editing full source'' link
2. Add a definition in XML
3. Click ''Save''

Between the list of recipe ingredients and the paragraph containing the newly unified instructions, add the following sentence:

“SPAM is the greatest food ever!”

Expected Steps:

1. Click the ''Switch to editing full source'' link
2. Add a paragraph in XML
3. Click ''Save''

Put a list inside the first paragraph that includes the top suspected ingredients of SPAM:

• chopped pork shoulder meat
• ham meat
• salt

Expected Steps:

1. Click on a paragraph
2. Add the XML for a list
3. Click the ''Save'' button