turkserver

Turkserver hosts Amazon Mechanical Turk (AMT)-style surveys on your own server. Does not handle users, payment, etc. Requires an Apache web server with PHP 5.3 or higher.

Installation

1. Download turkserver. Copy the turkserver folder onto your server. The “turkserver” folder name will become part of survey URLs, so change the name of the directory if you like.
2. Point your browser to the test page, at http://server/.../turkserver/index.php. This page will check to make sure that turkserver is installed properly.
3. Start hosting experiments! See “usage” below.

Usage

Basic experiment creation

Experiments are generated by simply copying your experimental items and template into the turkserver’s data directory. Turkserver uses a particular file naming convention. Follow these steps to create an experiment:

1. Choose an experiment name. This will be a code that will become part of the URL for your experiment. It should be a combination of letters and/or numbers, with no spaces.
2. Name the experimental item lists file (in comma-separated-value format; aka “batch file” in AMT terms) <experiment_name>.csv and copy it into the turkserver’s data directory.
3. Name your experiment’s template file <experiment_name>.html and copy it into the turkserver’s data directory.
4. And you’re done! Go to http://server/.../turkserver/<experiment_name> to participate in your experiment.

Retreiving results

Results for your experiment will be saved into turkserver’s data directory, in a file named <experiment_name>.results.csv. The format of the results is designed to be similar to the AMT’s results files.

Note that there is one there is one additional column added by turkserver which is not included by Turk: TurkServerCookieReliable. turkserver uses a browser cookie to provide semi-permanent WorkerIds (similar to AMT). If TurkServerCookieReliable is 1, this means that the participant’s browser had cookies turned on, and therefore this submission’s WorkerId can be trusted. If TurkServerCookieReliable is 0, this means that that the participant’s browser did not accept the cookie.

Ending your experiment

When your experiment is complete, you can either remove the <experiment_name>.* files from the data directory. Alternatively, you can open the experiment_meta.ini file, find the section related to your experiment, beginning with [<experiment_name>], and set status = "inactive".

We suggest that you do not reuse experiment names, or change the experiment data files while an experiment is running.

Advanced experiment settings

Some additional settings can be changed by modifying the experiment_meta.ini file. Find the section of the file related to your experiment, beginning with [<experiment_name>]. There you can change the title that is shown in the browser title bar, the submission “thank you” text, and whether the experiment is active or not.

Additional flags can be set here too. Currently the only such flag is add_extra_turk_fields = true which will add a number of blank columns to the results file, so that its output better matches what is produced by AMT.

Contributing

Contributions are welcome! Bug reports, feedback, documentation improvements, and code changes are all welcome forms of contributions. Thank you (in advance) for making turkserver better for everyone.

turkserver is developed on GitHub. Please submit bug reports and feedback requests on the turkserver issue tracker. Patches can be submitted via pull request.