Turktools

Tools to prepare linguistic surveys for Amazon Mechanical Turk

On crowdsourcing

How to use

How to contribute

Paper (preprint)

Contribute to Turktools

Design goals

turktools is designed with the non-technical user in mind. Therefore, the following principles are adopted in its design and development:

The current development target is Python 2.6 and 2.7. Python 3 support would be great in the future.

Note that an unfortunate consequence of the portability goal is to explicitly eschew a shared code library, forcing code duplication across different scripts, in violation of DRY. The current approach is to at least ensure the integrity of duplicated code across tools, by subjecting them to the exact same tests. (Perhaps a build tool will be used in the future to cut down on such redundancies.) Note also that the no dependencies goal and Python 2.6 target means that nice libraries like argparse cannot be used.

Contributing

Contributions are welcome! Bug reports, feedback, documentation improvements, and code changes are all welcome forms of contributions. Thank you (in advance) for making turktools better for everyone.

Bug reports and feature requests:

New bug reports and feature requests can be added on the turktools issue tracker. Please check whether your issue is already reported by someone else before opening a new issue. You must be logged into GitHub to create an issue.

Code:

turktools is developed on GitHub. The best way to hack on turktools is to open a GitHub account, fork this repository, and modify your own “fork” of the turktools. To submit changes, you can then initiate a pull request. Within reason, pull requests should include new test cases, in order to avoid later regressions.

Contributors should be familiar with the technical design goals above.

Documentation:

The turktools website is where we post additional documentation. The code for this site is on the gh-pages branch of our GitHub repository and therefore can be edited via pull request, described above. Feel free to contribute any materials there that you think may be helpful to a broader audience.

Testing

Test Status

turktools includes unit tests using the Python-standard unittest library. Tests can be run by running python tests.py. With the coverage module installed, run coverage run tests.py and then use coverage report -m to see a code coverage report.