Using Qualtrics and Soundcloud to do speech/listening research online

Intro

The internet is making it easier and easier to conduct L2 research. It's really easy to send out questionnaires to language learners or teachers, for example. It's also a great tool for collecting writing samples, or having learners read short texts and answer questions. Importantly, the internet makes finding and interacting with participants much easier than doing everything in person. Although you don't get the same level of control over experimental conditions that are valuable for some kinds of research, I'd argue that the internet makes it much more feasible to get non-undergraduate participants at low or no cost. Essentially, you trade experimental control for better sampling (than you could get otherwise).

One thing that isn't really easy to do on the internet is responding to or recording audio. Getting audio samples from participants over the internet in response to some kind of stimulus is pretty much a nightmare, and one that I would love to see solved (honestly, it seems like internet-based test providers are the only ones with a solid handle on this, in terms of out-of-the-box solutions). However, it's becoming a bit easier to get audio samples to participants, and as long as they're responding with clicks or typing, we're in business.

In this post, I'll show you how I've used Qualtrics and Soundcloud to collect transcriptions for a speech intelligibility project I am currently working on. In this project, I had 30 native speakers of Korean (about half located in Korea, and the others spread around the globe) transcribe 28 utterances produced by L2 Korean learners.

Qualtrics

 Qualtrics is a well-known online survey platform. There's a good chance your institution has deluxe access to it that will allow you to use most available functions AND dress up your surveys with a nice, official-looking stylesheet. If not, you can still use Qualtrics for free as an individual; the free version will let you do many kinds of simple surveys.

For this post, I'm going to assume some basic familiarity with Qualtrics. If you're totally new to it, head over to support.qualtrics.com to read up on the basics (and honestly, it's pretty intuitive- just make an account and start playing around to get a feel for it).

Qualtrics does have built-in audio/video support, letting you upload files to your account and embedding them in survey questions. Going this route, you can very easily implement something like this:


 

To do this, create a new Text/Graphic question, type your direction (e.g., "Click play to listen.") and then click the Rich Content Editor... tab. Then, click the little film icon to upload and insert a media file.



After you have your audio file loaded in, create a new Text Entry question below- this is where a listener can type what they heard. You can also add any other type of question, or multiple questions. This is a nice, simple solution, but there are some potential problems. For one, the audio player is fully controllable by the participant. This means that a participant could listen multiple times, and that participants may all have different numbers of repeated listenings. This is a major trade-off in control! Along the same lines, participants can play/pause at will, and scrub back and forth. For some research tasks, this might be fine (say, if you're just farming out audio corpus transcription). But if you want to measure someone's listening comprehension or the intelligibility of a particular speaker/utterance, well, this doesn't give us enough experimental control to confidently do that.

Another weakness of the Qualtrics media player is that it uses Flash. Flash is a fairly common piece of web software (though waning in popularity and use). In my experience it's not quite ubiquitous and universal- every time you see that broken media link with a little puzzle piece on a webpage, it's because your version of Flash isn't up to date or is for some reason incompatible with what the page is trying to show you. As a researcher, you don't want to lose potential participants because they can't play your stimuli.

Embedded streaming audio with Soundcloud

One workaround is embedding streaming media from an external site. Soundcloud uses HTML5 to stream audio. HTML5 is a nearly universal standard on the web, and almost all contemporary browsers handle it well. And as I'll show you, you can customize how embedded Soundcloud audio displays on your Qualtrics survey to increase your level of experimental control.

Get yourself a Soundcloud account (you get free audio storage for about 2 to 3 hours worth of stuff), and upload a file. When you upload, make sure to set your file to Private if you don't want it to be accessible to just anyone and everyone on the internet. After uploading, click the Share button by your file and then click over to the Embed tab.



 If you want to be able to limit participants' control over audio playback, you'll want to click More Options near the bottom of the embed tab and click the checkbox for Enable automatic play. This might seem like a bad idea, but we'll build our own means of advancing through the survey that won't startle your participants with audio unexpectedly playing.


Finally, click in the Code and preview box to highlight your embed code. Hit ctrl+c to copy the code. Back to Qualtrics!









Over in Qualtrics, we're first going to make a question that allows participants to be ready to hear an audio stimulus. Create a new Descriptive Text question, and type a direction like "Click the >> button to play the next audio file." Next, insert a Page Break to require the participant to manually advance to the next question.

Now, create a new Text Entry question (or Multiple Choice, or whatever you'd like). Click on the HTML tab, and paste (ctrl-v) the embed code from Soundcloud. To get rid of any audio controls, change the value after iframe width to 0%, and change the value after height to 0. If you look carefully at the rest of the embed code, you can see that auto_play is set to true (you'll also notice some blurry stuff in the screenshot below- just keeping my private file private!). For the other options, you could go through and set them all to false, but this is ultimately unnecessary- since you made the Soundcloud embed a 0x0 pixel box, there's nothing that anyone can click on in your survey.



In final form, we get:
1. A screen that gets the participant ready to listen

 2. A screen where audio automatically starts playing, and a question that a participant can answer.

Closing 

I hope this post has been helpful. By inserting an automatically playing, one-time-only streaming audio file into Qualtrics, you can have a wide range of participants respond to audio stimuli while still maintaining control over the number of replays and play/pause functions. With some finesse, you could even add a limited number of chances for participants to replay audio (Qualtrics features some fairly robust logic and sequencing options).

Also, I strongly recommend including a practice item or two that serve as audio hardware checks so that participants can make sure their speakers/headphones are a) on, and b) at a comfortable listening volume. In my experiment, I had them transcribe a speaker saying "I can hear the audio well" as a technical check before going on to a practice item.

While you might not be able to achieve the level of control necessary for any speech or listening research, I think we're getting to the point where a lot can be accomplished with pre-packaged data collection software/platforms... if you're willing to do just a little tweaking.