Multi-label answer aggregation problem aims to estimate the ground truth labels of samples by aggregating a large number of non-expert annotations collected by crowdsourcing. Due to the high annotation cost, multiple sample categories and uneven distribution of digital cultural heritage data, it brings great challenges to multi-label answer aggregation of datasets. Previous methods mainly focus on single-label problems, ignoring the label relevance of multi-label tasks. To some extent, most multi-label aggregation methods consider label correlations but are sensitive to noises and outliers. To solve these problems, this paper proposed a multi-label answer aggregation method based on adaptive graph regularization and joint low-rank matrix factorization AGR-JMF. Firstly, it divided the input annotation matrix into two parts: pure annotations and noise annotations. Then, it constructed the association matrix between labels by adaptive graph regularization method for pure annotations. Finally, in order to realize the multi-label answer aggregations, it used labeling quality, label relevance, and the behavior attributes similarity between annotators to guide the low-rank matrix factorization. Experiments on real-world datasets and MGF dataset show that AGR-JMF has obvious advantages over existing algorithms in terms of aggregating accuracy and identifying unreliable annotators.

[2022-12-07] Accepted Paper /doc/2022.09.0442/v1

[2023-04-05] Printed Article /doc/2022.09.0442/v2