Excluding Mturk workers from your HITs (not just from your Qualtrics surveys)

My method for excluding Mturk workers from Qualtrics surveys (found here) is rather easy to use, but is pretty much limited to Qualtrics and Survey Gizmo. It provides helpful insight into features of the two systems, but I thought I’d develop a method to help the broader Mturk user (and those who use Survey Monkey). This is a rather simple JavaScript based system for telling workers whether or not you have them on a black list. On the black list means they have done your task before (e.g. taken you survey). Off the list means they are AOK.

The system is flexible; there is plenty about it you can change. I put in a complete HTML simply so people who want to can just copy and paste, changing the necessary parameters. There are plenty of instructions (basically everything within double quotation marks) that you can change if you don’t like how it’s worded. It is also designed for survey use where your worker accepts the HIT and is immediately sent to an outside webpage (i.e. the survey). If you need the user to do something else, you’ll have to change what happens in the last conditional of the check() function.

The logic of the system is:
1) User enters
2) System checks if HIT is accepted (by seeing if there is a workerId query string variable in URL)
3) If not accepted
a. Ask for ID input
b. Check given ID against black list
c. If on list, tell not to accept. If not on list, tell to accept
d. Accepting HIT reloads the page, starting program over again (i.e. start back at 1)
4) If accepted
a. Get workerId variable from query string in URL
b. Check actual ID against black list
c. If on list, ask to return HIT (i.e. they lied about their ID before or did not follow instructions). If not on list, give hyperlink and instructions for going to your webpage (e.g. survey).

There are 4 parameters that need to be changed. The first two are simple HTML edits. Just replace the title and instructions with your own title and instructions. This can be done in the rich text editor if you wish after pasting this code into the Source code area of the HIT template creator. The third parameter is the URL to send users to. Just paste your Survey Monkey, Qualtrics, or some other URL between the double quotation marks. The last parameter is a litter more difficult. You need to create a quotation-mark enclosed, comma-separated list of IDs. Typically you do this in Excel by downloading your previous data files, compiling a list of workerId’s in a column then in the column next to it, use the concatenate function to add quotation marks and commas (e.g. putting =concatenate(‘”‘,A1,'”,’) in cell A2 will do what is needed, then copy the formula down for each cell). Copy and paste that list between the parentheses of the workers variable.

Last, note that this code will send the worker ID to your survey. In Qualtrics you can store the ID by creating an embedded data element and naming it workerId

Here is the commented code. Or you can get it from GitHub: https://gist.github.com/TheBehavioralLab/6770997

1:  <h1>TITLE - change this</h1>  
2:  <p>INSTRUCTIONS - change this</p>  
3:  <div id="idcheck">&nbsp;</div>  
4:  <p><textarea cols="80" name="comment" rows="3"></textarea></p>  
5:  <script type='text/javascript'>  
6:  var url="URL GOES HERE";//The URL that the user will go to after they are checked (i.e. the survey url). Change this.  
7:  var workers=new Array("test1","test1","test3");//This is your black list. Change this. All IDs must be enclosed in quoation marks (single or double), and separated by a comma  
8:  //Below is a utility function that checks if a given value is in an array. This will be used to check if the id (given value) is in the black list (an array)  
9:  Array.prototype.contains = function(k) {  
10:    for(var p in this)  
11:      if(this[p] === k)  
12:        return true;  
13:    return false;  
14:  }  
15:  //HTML instructions and form for ID input  
16:  var naCheck="<p>To see if you have completed this survey already, enter worker ID below and click \'Check ID\'</p><p><textarea rows='1' cols='20' id='idinput'></textarea></p><div id='ok' onclick='check(false)' style='border: 3px solid black;cursor: hand; cursor: pointer; background-color:gray;color:white;width:70px;height:20px;text-align:center;margin-left:10px;'>Check ID</div>";  
17:  var href=window.location.href.toString();//Get URL of mturk webpage which may or may not include the worker ID  
18:  var queryString={};//storage variable for query string variables  
19:  href.replace(new RegExp("([^?=&]+)(=([^&]*))?", "g"),function($0, $1, $2, $3){queryString[$1] = $3;});//Populates variable named queryString (created above) with the actual queryString variable names and values  
20:  //If queryString contains a variable names workerId, check if the black list contains that value of that variable, if not, ask the user to input their ID.  
21:  if(queryString['workerId']!=undefined)  
22:  {  
23:       check(queryString['workerId']);//Check ID against black list  
24:  }  
25:  else  
26:  {  
27:       document.getElementById('idcheck').innerHTML=naCheck;//Add HTML necessary to ask for id input  
28:  }  
29:  //This function will check the ID given against the black list, or if not given will check the ID input into the text box against the black list  
30:  function check(id){  
31:       if(id==false){  
32:            if(workers.contains(document.getElementById('idinput').value))  
33:            {  
34:                 //Input ID found on list -- taken the survey before, ask not to accept  
35:                 document.getElementById("idcheck").innerHTML="Your ID is on the list of workers who have completed this survey already. Please do not accept the HIT. Thank you for your previous participation.";  
36:            }  
37:            else  
38:            {  
39:                 //Input ID not found on list -- ask to accept, when this happens it will start the program over again, this time checking their actual ID not their inputted ID, which should be the same  
40:                 document.getElementById("idcheck").innerHTML="Your ID is not on the list of workers who have completed this survey. Please accept the HIT to continue.";  
41:            }  
42:       }  
43:       else  
44:       {  
45:            if(workers.contains(id))  
46:            {  
47:                 //Actual ID found on list -- ask to return HIT (this should rarely happen, if ever... it basically means the person did not follow instructions or tried to lie before hand  
48:                 document.getElementById("idcheck").innerHTML="Your ID is on the list of workers who have completed this survey already. Please return the HIT. If you previously entered your ID and were told to accept the HIT, you likely did not enter the correct worker ID.";  
49:            }  
50:            else  
51:            {  
52:                 //Actual ID not found on list -- send to survey  
53:                 document.getElementById("idcheck").innerHTML="<p>To proceed to the survey <a href='"+url+'&workerId='+id+"' target='mturksurvey'>click here</a>";  
54:            }  
55:       }  
56:  }  
57:  </script>  

Single Post Navigation

← Older Entry

Newer Entry →