Archive for the month “October, 2013”

Social Consequences of Unforced Compliance: Reluctantly Joining Social Media

Twitter was always for overshare-ers to me. Instagram was wannabe creative-types without effort or talent. Klout? The attention whore’s measuring stick. One week ago, I started using all of those. After 7 or so days of being an oversharing, wannabe creative-type, attention whore I’ve noticed interesting behavior and changes in attitudes.

My social media awakening was less abrupt than I let on. I had a Friendster and LiveJournal since 2003. I was also an early MySpace and Facebook adopter. The purpose of these tools was to connect personally with friends and family. Facebook still serves that purpose to me. I live far from virtually everyone I’ve ever known, and it’s nice to know they exist. Facebook is not an external tool to me though. When a friends account becomes a marketing extension of the company they work for, the prejudice (and pride) run deep.

My current thrust was spurred on by a Professor’s (Pete McGraw) repeated discussions about the importance of external visibility for research. Projects used to be important based on citation counts. In the future, real-world impact is likely to be of much greater importance. This wasn’t entirely new to me, but my thinking previously was, “Well, I’ll be super impactful when I come up with my popular audience book idea.”

In the span of about five minutes my thoughts went from, “I should probably update my blog more regularly” to “Maybe also dust off the dormant Twitter account and broadcast the blog better.” Then the ball started rolling really fast. “I guess I should look at my Klout score and see if improving it leads to more people reading the blog.” “Instagram impacts Klout a lot? I said never, but since the app is free let’s go for it.”

Festinger’s Cognitive Consequences of Forced Compliance basically proposed that being forced to commit an act, given certain conditions, changes attitudes to be in line with the behavior. Since my behavior wasn’t forced, did my attitudes change in those few minutes which changed my behavior or does voluntarily changing behavior lead to the same (or stronger) attitude change compared to unforced compliance?

I don’t really desire to answer that question. All I know is the attitudes certainly changed. I enjoy having additional channels to interact with friends. I enjoy cultivating a professional online persona. I enjoy seeing a larger readership of my blog.

Self-Reflective Things I’ve Learned Thus Far

  • I am old. I realize this is a very cliched joke to make when you in your twenties and comparing yourself to teenagers, so that’s not really what I am doing. Instead, I’m making the slightly less cliched joke of pointing out when you are becoming everything you made fun of your parents about. Learning the norms of social networks is difficult. Sometimes I have these out of body experiences where I’m looking at myself like I’m watching a child learning to ice skate (or a grown 27 year old man who grew up in New England and never learned to skate learn to skate… blog pending). Luckily I’m cognizant of this, so I’m not too grandpa-on-facebook-like.
  • Instagram is not nearly as douchey as I had always thought. I’ve always felt like I don’t document important events enough (note this is nothing like the recent research in over-documenting… I don’t even have pictures from my honeymoon up the California coast), and Instagram provides a way to do that with double the positive reinforcement — friends connect, Klout score increases. Side note: Headlines like this one recent posted on Instagram’s twitter account will always make me roll my eyes: “Top 10 Photographers [really???] on Instagram”
  • Interestingly, after joining Instagram and Klout, I felt the need to make fun of that fact. Internally, I apparently couldn’t accept the statement “I’m joining Instagram.” It had to be something like, “Even I think I’m crazy and stupid, but I’m joining Instagram anyway.” This is probably already documented, but the need for some semblance of attitudinal or behavioral consistency even when you are performing a completely inconsistent action seems like an interesting phenomenon (assuming its not just me).

How to randomize or shuffle an array in Qualtrics

Qualtrics does many things right. However, its vast capabilities sometimes makes me think it can do things that it can’t. Unlike SurveyMonkey where I just assume it can’t do anything, sometimes I think Qualtrics can do everything. Luckily, Qualtrics’ JavaScript capabilities makes it so if you know some coding, you can do a lot of the things you thought were impossible.

Randomizing arrays of numbers is something Qualtrics can’t do (easily) without JavaScript. Technically, you can create a randomizer, inside the randomizer create X branch elements, set each element to automatically occur (previously set an embedded data element named a, set the value to 1, then set the branches to occur if a=1), create an embdedded data element in each branch, and set the randomizer to randomly show X elements evenly. Pretty difficult, and even that won’t do everything. You would need to add in however you implement the random number or use some piped text code to add that number to a different element to you can end up with a full array after the randomizer ends. Regardless it is difficult.

In Javascript the code is this (here it is on Github):

1:  function shuffleArray(array) {  
2:    for (var i = array.length - 1; i > 0; i--) {  
3:      var j = Math.floor(Math.random() * (i + 1));  
4:      var temp = array[i];  
5:      array[i] = array[j];  
6:      array[j] = temp;  
7:    }  
8:    return array;  
9:  }  

In a question in Qualtrics, you simply insert this code into the javascript editor. One way or another create your array (type it in, grab it from the question text, etc…. if you need help doing this let me know in the comments), shuffle it using this code, then one way or another use it. You can set the array to an embedded data element using the SurveyEngine.setEmbeddedData() function that is already available in the Quatlrics API (mentioned here). You can add it as text to a question (I think that function is mentioned in the previous link, if not its another SurveyEngine function). Again, the implementation options are infinite. If you’ve gotten this far, and don’t know how to use your randomized array, again, let me know in the comments.

Changing Mturk submit button functionality & A new way to prevent duplicate workers on separate HITs

Previously, I’ve posted some simple ways to prevent workers from completing your mturk HITs because they already completed an identical one last week or last month, etc. (here, here, and here). Sometimes keeping track of long lists of mturk workers who have completed your previous HITs is difficult. Editing, copying, and pasting a list of 1000 workers that took your survey last time and can’t take it this time takes a while and is prone to mistakes. The best method I can suggest when this is difficult is to start databasing your workers. This is how I manage my Mturk participants, but it requires a decent amount of knowledge of JavaScript, PHP, and MySQL. If that is not your cup of tea, I have been trying to work on an especially simple method using cookies, though it is by definition imperfect.

Cookies are simply small bits of data that a webpage can store on your computer, then later read back. It’s how website “remember” that you are logged in, for instance. Using a small amount of JavaScript, you can set a cookie that is named after your study. The existence of the cookie with that particular name means that the worker completed that study already. When you post a study later and want to exclude people who took your first study, your HIT will have some JavaScript in it that searches for that cookie you created earlier. If it exists, the person took your first study and is excluded, if it doesn’t, they can take your new study. There is no copy and pasting, and no long files full of worker IDs.

However, as I said, it is imperfect. Some users turn off cookies, preventing you from setting the cookie. Some users clear their cookies regularly, deleting the cookie you created. Cookies are computer based, not user based. So a user can go to a different computer and still take your new study. Also, a second person can try taking your study on the same computer and be prevented from doing so because the cookie applies to anyone using that computer, not the specific person. Given the large amount of mturk workers, and the base-rates for the problems I just mentioned, I would guess that this method would prevent about 90% of workers who you don’t want taking your studies from doing so. It may be more or less. The key factors are how many people regularly clear their cookies or complete HITs on multiple computers.

Here is the commented code you can copy and paste into your HIT template window. It first loads jQuery, then adds an event listener to the Submit button, then checks if a cookie exists with the same name as the cookie that gets created when the submit button is clicked. All you need to change is the studyName value to something unique for your study. However, what happens on line 20 is up to you. right now it simply alerts the user that they have taken your study before. Someone more useful would be to prevent any other content from loading and telling them to not accept/return the HIT. Since exactly how it gets implemented can change for HIT to HIT, I didn’t not specify a procedure beyond alerting the user. The cookies are set to last a maximum of 1 year. You can hypothetically make it longer, but likely people have deleted their cookies by then anyway.

Github code here

1:  <script src=""></script>  
2:  <script type="text/javascript">  
3:  //this code assumes jQuery is implemented, though it would not be difficult to do this with straight javascript. .ready() assures the code runs after the submit button is added  
4:  $(document).ready(function () {  
5:    var studyName="dm102513";//replace this value with a unique, acceptable cookie name... no spaces and stick with alphanumerics and underscores  
6:    $("#submitButton").click(function(){  
7:        var exdate=new Date();  
8:        exdate.setDate(exdate.getDate() + 365);  
9:     document.cookie=studyName+"=1; expires="+exdate.toUTCString()+";; path=/";//When the user clicks the Submit button, it creates a cookie that you can read later as having already completed this survey  
10:    });  
11:    var i,x,y,ARRcookies=document.cookie.split(";");//variables necessary to read the cookies. Last variables separates all available cookie names into an array we will loop through to find our cookie  
12:    for (i=0;i<ARRcookies.length;i++){  
13:     x=ARRcookies[i].substr(0,ARRcookies[i].indexOf("="));//x=cookie name  
14:     y=ARRcookies[i].substr(ARRcookies[i].indexOf("=")+1);//y=cookie value  
15:     x=x.replace(/^\s+|\s+$/g,"");//Remove white space  
16:     if (x==studyName && y==1){  
17:             //If your cookie exists AND the value is 1, they have taken your survey before.  
18:             //Here you would put code that somehow prevents them from doing anything for your HIT, like not loading content etc.  
19:             //However, for our purposes here, I just put a simple alert in. Change this to whatever code works for you.  
20:       alert("You are not eligible for this HIT because you have already completed an identical HIT.");  
21:     }  
22:    }  
23:  });  
24:  </script>  

Allow only specified workers to complete your Mturk HITs

The other day I posted some code that allows mturk requesters to exclude workers based on a black list (here). This is typically used when someone already completed your survey, but for one reason or another you can’t just extend your already created HIT. The code in there actually started as the inverse. Instead of wanting to block certain people, a colleague wanted to only include certain people. Rather than dealing with the under-powered qualification system (under-powered for mturk web interface users at least), I figured it would be easy to just create a white list of worker Ids, and exclude everyone else from HIT. This way seemed faster than somehow creating a qualification test, or other method of excluding/including workers.

Again, this is very flexible and easily changeable. I, for instance, don’t rely on client-side programs (i.e. JavaScript) for validating users. Instead of checking the ID against a list declared in JavaScript, I send the data to server-side PHP script which returns the validation term, and if validated the URL to send workers to. The changes that would need to be done for this is to remove the url variable initialization, and add an AJAX request instead of the check() function call for when the HIT is accepted.

This code below is copy-and-pastable. Just change the Title, Instructions, target URL (where to send workers who are allowed), and create a white list of worker IDs. See the post linked at the top for how, but you will need to wrap all ids in quotation marks, and separate them by a comma. Last, change the conditional instructions to tell the worker what to do when they are on/off the list, etc.

Here’s the code (also here):

1:  <h1>TITLE - change this</h1>  
2:  <p>INSTRUCTIONS - change this</p>  
3:  <div id="idcheck">&nbsp;</div>  
4:  <p><textarea cols="80" name="comment" rows="3"></textarea></p>  
5:  <script type='text/javascript'>  
6:  var url="URL GOES HERE";//The URL that the user will go to after they are checked (i.e. the survey url). Change this.  
7:  var workers=new Array("test1","test1","test3");//This is your white list. Change this. All IDs must be enclosed in quoation marks (single or double), and separated by a comma  
8:  //Below is a utility function that checks if a given value is in an array. This will be used to check if the id (given value) is in the black list (an array)  
9:  Array.prototype.contains = function(k) {  
10:    for(var p in this)  
11:      if(this[p] === k)  
12:        return true;  
13:    return false;  
14:  }  
15:  //HTML instructions and form for ID input  
16:  var naCheck="<p>To see if you are qualified for this survey, enter worker ID below and click \'Check ID\'</p><p><textarea rows='1' cols='20' id='idinput'></textarea></p><div id='ok' onclick='check(false)' style='border: 3px solid black;cursor: hand; cursor: pointer; background-color:gray;color:white;width:70px;height:20px;text-align:center;margin-left:10px;'>Check ID</div>";  
17:  var href=window.location.href.toString();//Get URL of mturk webpage which may or may not include the worker ID  
18:  var queryString={};//storage variable for query string variables  
19:  href.replace(new RegExp("([^?=&]+)(=([^&]*))?", "g"),function($0, $1, $2, $3){queryString[$1] = $3;});//Populates variable named queryString (created above) with the actual queryString variable names and values  
20:  //If queryString contains a variable names workerId, check if the black list contains that value of that variable, if not, ask the user to input their ID.  
21:  if(queryString['workerId']!=undefined)  
22:  {  
23:       check(queryString['workerId']);//Check ID against white list  
24:  }  
25:  else  
26:  {  
27:       document.getElementById('idcheck').innerHTML=naCheck;//Add HTML necessary to ask for id input  
28:  }  
29:  //This function will check the ID given against the white list, or if not given will check the ID input into the text box against the white list  
30:  function check(id){  
31:       if(id==false){  
32:            if(workers.contains(document.getElementById('idinput').value))  
33:            {  
34:                 //Input ID found on list -- ask to accept, when this happens the page reloads and the program starts over
35:                 document.getElementById("idcheck").innerHTML="Your ID is on the list of qualified workers. Please accept the HIT to continue.";  
36:            }  
37:            else  
38:            {  
39:                 //Input ID not found on list -- ask not to accept
40:                 document.getElementById("idcheck").innerHTML="Your ID is not on the list of qualified workers. Please do not accept the HIT.";  
41:            }  
42:       }  
43:       else  
44:       {  
45:            if(workers.contains(id))  
46:            {  
47:                 //Actual ID found on list -- send to survey
                   document.getElementById("idcheck").innerHTML="<p>To proceed to the survey <a href='"+url+'&workerId='+id+"' target='mturksurvey'>click here</a>";
49:            }  
50:            else  
51:            {  
52:                 //Actual ID not found on list -- Ask to return HIT
53:                document.getElementById("idcheck").innerHTML="Your ID is not on the list of qualified workers. Please return the HIT. If you previously entered your ID and were told to accept the HIT, you likely did not enter the correct worker ID.";   
54:            }  
55:       }  
56:  }  
57:  </script>  

Post Navigation