# Metadata Source URL:: https://arxiv.org/abs/2304.11158 Topics:: #ai, #llm --- # Emergent and Predictable Memorization in Large Language Models Memorization, or the tendency of large language models (LLMs) to output entire sequences from their training data verbatim, is a key concern for safely deploying language models. In particular, it is vital to minimize a model's memorization of sensitive datapoints such as those containing personal identifiable information (PII). The prevalence of such undesirable memorization can pose issues for model trainers, and may even require discarding an otherwise functional model. We therefore seek to predict which sequences will be memorized before a large model's full train-time by extrapolating the memorization behavior of lower-compute trial runs. We measure memorization of the Pythia model suite, and find that intermediate checkpoints are better predictors of a model's memorization behavior than smaller fully-trained models. We additionally provide further novel discoveries on the distribution of memorization scores across models and data. ## Highlights > [!quote]+ Updated on 240423_175247 > > The paper "Emergent and Predictable Memorization in Large Language Models" by Stella Biderman et al. studies the problem of memorization in large language models and proposes a method to predict which sequences will be memorized before full training of the model, based on extrapolation of memorization behavior from lower-compute trial runs, and provides novel insights on the distribution of memorization scores across models and data. >Key insights and lessons learned from the paper: > >Memorization is a key concern for deploying large language models safely, particularly for sensitive datapoints such as PII. >Intermediate checkpoints are better predictors of memorization behavior than smaller fully-trained models. >Memorization scores follow a power-law distribution across models and data, with some datapoints being more prone to memorization than others. >Fine-tuning can mitigate memorization to some extent, but not completely.