Commit 45ca9020 authored by Björn Barz M.Sc.'s avatar Björn Barz M.Sc.

Initial commit

parent f411f49b
This source diff could not be displayed because it is too large. You can view the blob instead.
This diff is collapsed.
# User Reputation for Crowdsourcing
Source code for creating a user reputation model on the Building Footprints data from the New York Public Library.
\ No newline at end of file
Crowdsourcing is a popular way to obtain annotations for large amounts of data quickly.
However, the quality of the annotations can vary widely between individual workers.
Often, there are several different annotations for the same object, sometimes even contradicting ones.
In the following publication, we proposed a method to automatically construct a user reputation model based on the pair-wise agreement of users, so that you know which of your workers are more trustworthy than others:
> Björn Barz, Thomas C. van Dijk, Bert Spaan, and Joachim Denzler:
> [Putting User Reputation on the Map: Unsupervised Quality Control for Crowdsourced Historical Data.][1]
> *2nd ACM SIGSPATIAL Workshop on Geospatial Humanities, 2018.*
In this repository, we provide exemplary source code for building and analyzing such a user reputation model on building footprint data from the [NYPL Building Inspector][2] project.
## Files in this repository
- [**Building Footprint Reputation.ipynb**](Building%20Footprint%20Reputation.ipynb): Jupyter Notebook loading the data, building the user reputation model, and performing several analyses such as user activity vs. quality, evolution of annotation quality over time etc.
- [**Fetch Building Footprint Annotations.ipynb**](Fetch%20Building%20Footprint%20Annotations.ipynb): This notebook downloads queries the Building Inspector API to fetch all data and combines it with ground-truth annotations to create the combined data file `building-footprints-annotations.pickle`.
- **building-footprints-annotations.pickle:** Pickle file with building footprint data, created by the notebook mentioned before.
- **building-footprints-gt.json:** JSON file with ground-truth annotations for a few buildings, described in the following publication:
> Benedikt Budig, Thomas C. van Dijk, Fabian Feitsch, Mauricio Giraldo Arteaga:
> [Polygon consensus: smart crowdsourcing for extracting building footprints from historical maps.][3]
> *24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, 2016.*
[1]: https://doi.org/10.1145/3282933.3282937
[2]: http://buildinginspector.nypl.org/
[3]: https://doi.org/10.1145/2996913.2996951
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment