Abstractive summarisation generates summaries by producing new text rather than selecting existing sentences from the source document. An abstractive summary may paraphrase source content using different words, compress multiple sentences into one, fuse information from different parts of the document, and generalise specific details into broader statements. This is how humans typically write summaries, and it produces more concise and coherent output than extractive methods. However, abstractive summarisation is fundamentally harder because it requires both understanding the source and generating fluent, faithful text.
Sequence-to-Sequence Models with Attention
Attention: αᵢ = softmax(score(sₜ, hᵢ))
Context: cₜ = ∑ᵢ αᵢ hᵢ
Copy mechanism: P(w) = p_gen · P_vocab(w) + (1 − p_gen) · ∑_{i:xᵢ=w} αᵢ
allowing the model to copy words directly from the source
Modern abstractive summarisation is built on the sequence-to-sequence (seq2seq) framework with attention. The encoder processes the source document into a sequence of hidden states, and the decoder generates the summary one token at a time, attending to relevant parts of the encoder output at each step. Rush et al. (2015) demonstrated that neural attention-based models could generate abstractive headlines for news articles. The pointer-generator network (See et al., 2017) introduced a copy mechanism that allows the model to either generate a word from the vocabulary or copy a word directly from the source, handling out-of-vocabulary words and improving factual accuracy.
Pretrained Models and Faithfulness
Pretrained sequence-to-sequence models such as BART (Lewis et al., 2020), T5, and PEGASUS (Zhang et al., 2020) have dramatically improved abstractive summarisation quality. PEGASUS was specifically designed for summarisation: during pretraining, important sentences are masked from documents and the model learns to generate them, a pretraining objective that closely mimics the summarisation task. These models produce summaries that are often fluent enough to be mistaken for human-written text, but faithfulness remains a critical concern.
Hallucination — generating information not supported by the source document — is the most pressing problem in abstractive summarisation. Maynez et al. (2020) conducted a large-scale human evaluation and found that over 70% of abstractive summaries contain at least one type of hallucination, including intrinsic hallucinations (contradicting the source) and extrinsic hallucinations (adding information not in the source). Factual consistency checking, constrained decoding, and faithfulness-aware training objectives are active areas of research aimed at producing abstractive summaries that are both fluent and faithful.
Controllable abstractive summarisation allows users to specify attributes of the desired summary, such as length, style, focus topic, or level of detail. Control mechanisms include length-aware training, content planning (generating a plan or outline before generating the summary), and conditional generation with control tokens. Multi-document abstractive summarisation extends the task to generating a single summary from multiple source documents, requiring the additional capabilities of cross-document information fusion, contradiction detection, and temporal ordering of events described across documents. These challenges push the boundaries of current generation models and remain active research frontiers.