In the big data environment, the parallel K-means clustering algorithm suffers from poor clustering effect, unbalanced data partition, cluster centroid sensitivity. To solve these problems, this paper proposed a parallel K-means algorithm based on MapReduce and MSSA(MR-MSKCA). Firstly, MR-MSKCA designed a dimensionality reduction strategy(DRKCAE), which used Kendall correlation coefficient and deep sparse autoencoder to weight features and extract features to improve the clustering effect of high-dimensional data. Secondly, it proposed a UPS, which divided the dataset and obtained uniform data partition. Finally, this paper proposed MSSA to get the parallel K-means clustering centroid, which solved the problem of initial centroid sensitivity. Compared with MR-KNMF, MR-PGDLSH and MR-GAPKCA, the running time of MR-MSKCA decreased by 45.1%, 49.1%, 59.8%, and the clustering effect increased by 19.2%, 22.8%, 24%. Experiments show that the MR-MSKCA not only has excellent performance, but also has strong adaptability with large-scale dataset.

[2022-06-21] Accepted Paper /doc/2022.04.0149/v1

[2022-11-05] Printed Article /doc/2022.04.0149/v2