Extractive method extracts sentences from source text, which results in information redundancy. Generative method may generate non-source words, which results in grammatical problems and poor naturalness. BERT as a bidirectional Transformer model, which shows excellent performance in natural language understanding tasks. The application of text generation task remains to be explored. To address the above issues, this paper proposed a three-stage compound text summarization model based on pre-training(TSPT), which combining extractive method and generative method. The bidirectional contextual information word vectors generated by the pre-training of the source text are extracted from the sigmoid function to obtain sentence scores and extract key sentences. Finally, key sentences are rewritten as cloze filling tasks to generate final summarization. The experimental results show that it achieves high scores in CNN/Daily Mail dataset.

[2020-10-05] Printed Article /doc/2019.05.0193/v2