the:behavioral:lab

Archive for the month “July, 2012”

Do you feel pre-determined … punk?

During a particularly unfortunate (and melodramatic) time, I once told someone that only bad things happen to me. Anything good thing was not happening to me directly, but merely a by-product of something good happening to someone else. As you can imagine, I was not particularly happy, but don’t worry I got over it quickly. I even had a statistical justification. If, say, a trillion random things happen to a person in their life, on average half might end up good and half might end up bad. But what’s the standard deviation? If chance events are normally distributed, there is a likelihood someone, probably me, has at least 95% of those trillion random things turn out bad.

Obviously I didn’t really believe this, or at least to that extent. However, plenty of people out there have some opinion on their own personal luck. During a study where I asked people to make choices between gambles, I decided to ask them why they chose what they did. Most gave mathematical-type answers like the expected value of one was better than the other. A significant number, however, said they didn’t want to pick the risky gamble because they are they are not lucky.

So I began to think what other areas of decision making are affected by a person’s thoughts on luck. Gambling is an obvious one, and to an extent well-researched. Unlucky people tend to stop gambling after a string of losses, while lucky people tend to keep going. However, is this how people make their investment decisions? Or even their traffic decisions? I used to be able to take either the freeway home from work at UCLA or side-streets. Freeways can be faster, but in traffic its variable and you may hit a dead-stop. Side-streets at least have you moving all the time, that is except for at stop lights. I hear people discuss this same problem all the time, all over the country. Do I personally take side streets only when I think I am unlucky? Do I drink the 2-day-expired milk only when I am feeling good about my cosmic chances?

This is going to be the next branch of research I delve into. Can thoughts on luck be used as an individual difference measure to explain decision making behavior. Do attitudes toward luck change over time? Is the attitude merely conscious or does it affect choices even when a person is not thinking about whether or not they are lucky? In the end I hope to finally answer whether the bank robber in Dirty Harry really wasn’t feeling lucky or whether he was just making a rational choice.

The death of the confirmation code… hopefully

The confirmation code is an interesting psychological experiment in and of itself. We receive them for every digital transaction that we take part in from purchasing a book on Amazon.com to paying for dinner with a credit card. In our minds that number is our guarantee that what we just experienced actually happened and that we did not merely purchase that copy of 50 Shades of Grey (what??? I mean A Song of Ice and Fire!) in our imaginations. Without one, in some instances we can survive. Who looks that confirmation code on their credit card receipts after all? However, in others we seem to be lost, uncertain of the future and whether we can go on. Or so it seems in Amazon Mechanical Turk.

When I helped kickstart that use of MTurk at the Anderson School of Management at UCLA a few years ago, we needed a way to connect the data file on Mturk with the data file for our experiments. I’m sure I’m not the first person to think of this simple solution, but I suggested we assign every participant a unique code and have the participant enter it in the MTurk HIT before submitting it. It caught on there, and from the looks of the landscape its how everyone else does it too. It’s so ubiquitous, in fact, that when I stopped using them in favor of, in my opinion, a better practice, I started receiving 30 emails per study asking where their confirmation code was.

The problem with the confirmation code in MTurk is the error rate. Sometimes as many as 10% of people enter the code wrong. Also, it’s a pain to give each participant a unique code, and it’s annoying using these extra columns in the data files. I wanted to eliminate human copy & paste error, and utilize data already available. To do this I started storing each participant’s Amazon Worker ID in my experiment data files. The ID is automatically stored in the MTurk data file, and sending it and storing it in Qualtrics or other online survey systems is a breeze. The only problem, other than the uncertain feeling of the missing confirmation code, is that the worker has to accept the hit before you can access the ID. This is only a small problem.

What is below is a simple JavaScript that will create a hyperlink that only sends people to a URL if they have accepted the HIT already. Once accepted and clicked, the worker’s workerId is appended to the URL as a query string variable.  The code can be copy and pasted into the Source window of the Mturk HIT template. Then you simply have to replace the surveyUrl with the URL of your survey. The code is purposefully rudimentary (for JavaScript at least) so anyone can use it. If you know JavaScript, you can edit the code how you like, using buttons and events in place of the anchor, etc.

<script type='text/javascript'>
  var surveyUrl="https://thebehaviorallab.wordpress.com";//The url you want to send people to.
  function gotoSurvey()
  {
    var href=window.location.href;//Get the url of loading document (in mturk is the iframe the HIT content is in, not the url of the page itself)
    var queryString={};//Create an empty object to dump query string variables in.
    href.replace(new RegExp("([^?=&]+)(=([^&]*))?", "g"),function($0, $1, $2, $3){queryString[$1] = $3;});//Use a regular expression as well the a nifty alternative second parameter for the String.replace() method to dump all query variables into the queryString object
    if(queryString['workerId']!=undefined)
    {
      window.open(addQueryVar(surveyUrl,'workerId',queryString['workerId']),'survey_window');//add workerId variable to URL and open popup. Edit the link text in case of popup blockers
      document.getElementById('survey_anchor').innerHTML="If your popup blocker prevented the survey window from opening, disable it and click this link again.";
    }
    else
    {
      //No workerId variable. Worker hasn't accepted HIT -> ask to accept
      alert("You have not accepted the HIT yet. Please do so before clicking this link");
      document.getElementById('survey_anchor').innerHTML="Please accept the HIT, then click this link again";
    }
  }
  function addQueryVar(url, name, value)
  {
    //Find anchor in URL since you can't add the query string after an anchor
    var fragmentStart = url.indexOf('#');
    if (fragmentStart < 0) fragmentStart = url.length;//no anchor so add variable to end
    var urlBeforeFragment = url.substring(0, fragmentStart);
    return urlBeforeFragment+(urlBeforeFragment.indexOf('?') < 0 ? '?' : '&')+encodeURIComponent(name)+'='+encodeURIComponent(value)+url.substring(fragmentStart);//if there is already query string variables add new variable with &, if not, add with ?, making sure to URI encode the content.
  }
</script>
<p>
  <a id="survey_anchor" name="survey_anchor" href="javascript:gotoSurvey()">Click here to go to the survey</a>
</p>

You no longer have to have subjects enter anything in your Mturk HITs before submitting them. Mturk does require you have at least one input in your code, so I usually put a comment box: <textarea id="comments"></textarea>, and let the users tell me about typos and stuff that I missed.

Storing the ID in Qualtrics is easy. All you have to do is create an embedded data element in the Survey Flow menu. Name the embedded data field workerId (all lowercase except for the I). You are done (leave the value unset). It will grab the value from the URL and store it in your Qualtrics file. The workerId field is already stored in the Mturk data file, so simply sort them, or use VLOOKUP in Excel, or however else you transfer who to accept and who to reject to Mturk.

While confirmation codes psychologically tell us that something actually happened and there is a documented way to look it up, in MTurk they are fraught with problems. I can only hope that by showing people some code that will make their research lives easier, it will also help me by diminishing the amount of worried email I receive each time I post a study.

Post Navigation