Summary

Using an external testing service, we conducted usability tests with four users who had never seen the subtitling prototype before. Each user was presented with a script (written by PCF) which instructed them to transcribe, sync, and review subtitles to a common video. Below you will find the raw video sessions and written responses as we received them from each user, as well as our general observations of these sessions. We encourage you to review the material and use the insights you uncover as a basis for your submission!

General observations:
Of the four users, only one (Cynthia) could be said to have had a sufficiently positive user experience. Cynthia, also happened to have formerly transcribed for a living and professed to some familiarity to other transcribing software. The other three users could be labelled as new to the practice. At some point in the session, all three of these users expressed a wish to slow down the video to ease the transcription or syncing steps. Jane and John both spent a great deal of time in the transcription stages, rewinding and trying to understand/hear the video. Cynthia and George had an easier time with transcription, so perhaps this only affects a subset of users. All users encountered difficulties in the Sync and Review steps. There appeared general confusion on where edits should be made. Some users returned to the Sync step to make edits, while others made them in Review. Editing and syncing was sufficiently difficult that three of the users would have benefited from resetting their changes -- yet only one (John) was actually able to find the "Start Over" option.

Review individual sessions:

• John
• Jane
• Cynthia
• George

John

Age: 20
Gender: Male
Tech Experience: Expert
Daily Web Usage: 3-5 hours

Our Observations:

• John implies that the tool assumes that people know what a subtitle is.
• John repeatedly clicks the control instructions, expecting buttons.
• John uses video controls (mouse-based) instead of the keyboard controls most of the time.
• When John rewinds the video during the "Transcribe" step he appears to expect the lines to appear in sync.
• John breaks up lines after writing them. ("No, that's too long.")
• Keyboard controls don't work consistently. (possible bug)
• At the start of "Sync", he expected upcoming lines to scroll up as he pressed spacebar.
• John does not take the time to fix syncing on an individual line because he didn't want to start everything over.
• John thinks the Start Over link is difficult to see.
• John completes a review while still within the "Sync" step.
• John spends a significant amount of time trying to understand/hear uncommon words.

His Written Responses

What would have caused you to leave this site?
* There was nothing too severe or problematic that would cause me to leave the site. I understand I have to be logged in to save my work so I'm wondering how much this service would cost? If it's a high price, that might cause me to look elsewhere for my needs. I know some programs that have editing capabilities such as this one and are at a reasonable price.

What other ideas do you have about how this site could be improved?
* I found a couple of things where some thing could be improved. For example, I consider my technological skills to be pretty high so I could easily follow along with the controls and instructions. I understood exactly what the program was offering and how to execute tasks. But for someone who isn't as proficient, I would suggest to add a video tutorial on how to use the program, with step by step instructions. Or elaborate more on the instructions during the first time use. I also noticed that the "Start Over" button was very faint so I would suggest to make it more visible for the user if one does mess up.

What did you like about the site?
* I really liked how friendly the user interface was and there weren't that many buttons to keep track of. The information for button configurations remained on the left side of the screen in case the user forgot the layout and the instructions also provided how to carry out tasks. The actual layout of the entire site was simplistic but easy to navigate around which is always a good thing for a first time user.

Anything else you would like to say to the owner of the site?
* I was very interested in the program but I feel like there can be a good amount of expansion left. I would like to see the program have more capabilities for editing. Other than that, I would strongly suggest this type of program to some video and effects students in my art college.

Back to Summary

Jane

Age: 64
Gender: Female
Tech Experience: Intermediate
Daily Web Usage: 1-3 hours

Our Observations:

• Jane appears intimidated by the task at hand. ("This could get a little bit hairy.")
• After reading the instructions, Jane clicks "Next Step" as if the first step is merely an introduction.
• Jane clicks the control instructions, expecting buttons.
• Keyboard controls don't work consistently. (possible bug)
• Jane does not understand/hear Asa Dotzler's name. Does not take the time to try and understand/hear it better.
• Jane repeatedly shares the feeling that she feels in a "time crunch." When transcribing in real-time "it all goes too fast...a bit confusing."
• Jane goes to "Sync" step without hitting enter on the last line she transcribed, effectively losing her work.
• The syncing instructions are unclear to Jane until after several minutes and attempts.
• Jane attempts edits in both the Sync and Review steps.
• Jane tries clicking on the line directly on the video to edit the text. Only after some time does she try double-clicking on text in the edit box.
• Jane states that she will start over a couple of times, but it is unclear if she ever sees the "Start Over" link on the Sync step.
• Jane expresses her frustration with the process. Two main concerns: "the speed and the syncing step"

Her Written Responses

What would have caused you to leave this site?
* No specific problems with the site itself. At one point I was prompted to login. Since I didn't have login information I was unsure how to proceed and may have bailed out at that point, but decided to continue by clicking on another part of the process and that seemed to let me move on.

What other ideas do you have about how this site could be improved?
* The addition of a link to more detailed instructions, perhaps even a demonstration that the user could watch would be helpful.
* In this particular exercise where you're trying to transcribe the audio, it was a little challenging because of the speed. Yes, there were buttons to stop and back up, but a button to slow down the video/audio might be helpful also.

What did you like about the site?
* It's fairly easy to use once you get the hang of things.

Anything else you would like to say to the owner of the site?
* I think a short demonstration video would be VERY helpful.

Back to Summary

Cynthia

Age: 52
Gender: Female
Tech Experience: Expert
Daily Web Usage: More than 7 hours

Our Observations:

• Cynthia formerly transcribed as a career.
• During transcription, Cynthia does not break up her lines as instructed. This appears to be more efficient, however.
• Cynthia proceeds to the Sync step without pressing enter on the last line, effectively losing her work.
• Cynthia is confused by the Review instructions. She presses and holds down spacebar while in edit mode.

Her Written Responses

What would have caused you to leave this site?
* I may have left the site if I couldn't figure out how to use it - but tutorials make it a breeze

What other ideas do you have about how this site could be improved?
* The addition of a link to more detailed instructions, perhaps even a demonstration that the user could watch would be helpful.
* Nothing - I've done video transcribing and this is the easiest interface I've ever used.

What did you like about the site?
* The ease of use - anyone can do it.

Anything else you would like to say to the owner of the site?
* Great concept - the easiest transcribing program I've ever seen!

Back to Summary

George

Age: 28
Gender: Male
Tech Experience: Expert
Daily Web Usage: 3-5 hours

Our Observations:

• George observes that commas appear as periods in text box.
• George has no difficulty transcribing the text.
• George observes that pressing "Tab" to play/pause the video is not intuitive as tab typically moves the cursor forward in text fields.
• George does not find the labeling of links intuitive and wishes hovering over them would provide more information.
• George wishes that playback could be slowed down during the syncing phase
• George observes that hitting spacebar to start the video and then again immediately with the start of the first line is awkward.
• George wonders how syncing would handle a long pause between lines.
• George wonders if there is an Undo button anywhere, does not find one.
• George is confused by the Review instructions.
• George holds down spacebar and syncing of lines is severely off.
• George never discovers the Start Over link.
• George discovers if you click on the video itself, then spacebar begins to play/pause video -- something that tab is supposed to do.
• George wishes he could manually edit the times, just as is possible with the text.
• George suggests presenting edit/review modes in timeline view, similar to video editing interfaces.

His Written Responses

What would have caused you to leave this site?
* Nothing, but I could really use an undo button, so I might reload the page.

What other ideas do you have about how this site could be improved?
* The biggest problem was that holding spacebar in the review stage seemed to to the opposite of what I was expecting it to do. It shortend the times instead of lengthening them.
* It would be really nice to be able to fine tune the timing by being able to type in a time.
* I need an undo button.
* Add numbers to the bottom tabs to indicate the progression (i.e., 1. Transcribe, 2. Sync, 3. Review).
* The "Start Over" link should be available on every tab.
* It could use some sort of a timeline view.
* Spacebar seems to conflict with the video playback.

What did you like about the site?
* It had good instructions that made sense. It was pretty easy to use for the most part. It makes a tedious job get done quicker. I like the idea of being able to add subtitles easily to a video.

Anything else you would like to say to the owner of the site?
* Is there a way to slow down the playback of the video?

Back to Summary