Official repository for paper “Can Language Models Follow Multiple Turns of Entangled Instructions?”

Official repository for paper “Can Language Models Follow Multiple Turns of Entangled Instructions?”

📌 Overview

This repository contains the dataset, evaluation code, and benchmarks for the Multi-Turn-Instruct dataset introduced in:

Can Language Models Follow Multiple Turns of Entangled Instructions?
Chi Han
University of Illinois Urbana-Champaign
📧 Email: chihan3@illinois.edu

Dataset

Despite significant achievements in improving the instruction-following capabilities of large language models (LLMs), the ability to process multiple potentially entangled or conflicting instructions remains a considerable challenge. Real-world scenarios often require consistency across multiple instructions over time, such as secret privacy, personal preferences, and prioritization, which demand sophisticated abilities to integrate multiple turns and carefully balance competing objectives when instructions intersect or conflict. This work presents a systematic investigation of LLMs’ capabilities in handling multiple turns of instructions, covering three levels of difficulty: (1) retrieving information from instructions, (2) tracking and reasoning across turns, and (3) resolving conflicts among instructions. We construct Multi-Turn-Instruct with∼1.1K high-quality multi-turn conversations through the human-in-the-loop approach and result in nine capability categories, including statics and dynamics, reasoning, and multitasking.

📊 Main Results

Our finding reveals an intriguing trade-off between different capabilities. While GPT models demonstrate superior memorization, they show reduced effectiveness in privacy-protection tasks requiring selective information withholding. Larger models exhibit stronger reasoning capabilities but still struggle with resolving conflicting instructions. Importantly, these performance gaps cannot be attributed solely to information loss, as models demonstrate strong BLEU scores on memorization tasks, but their attention mechanisms fail to integrate multiple related instructions effectively. These findings highlight critical areas for improvement in complex real-world tasks involving multi-turn instructions.

Shown below is a histogram comparing model performances in the paper:

An Interesting Correlation Among Tasks and Models

Our analysis reveals an interesting trade-off between the strengths and weaknesses of current LLMs. Despite the expectation that improved intelligence will positively reflect in performance in most tasks, tasks positively and negatively correlate in their performance on LLMs. The tasks falling within the “contradiction resolution” category, namely Privacy Protection, Personalization, and Prioritization, are less correlated with the other tasks.

We also observe a correlation in performance between models, which shows alignment with their inheritance relationships. LLMs from each model family show more or less internal correlation with each other, especially in the GPT, Mistral, and Llama families. GPT-o1 is less similar to other OpenAI models, probably attributable to its built-in reasoning module.

Getting Started

Adding a new model

1. Add the model class script to the models/ directory. For example: models/llama_model.py.
2. Update the get_model function in models/get_model.py to include the new model.
3. Run the evaluation script with the new model to see if it works. See details below.

Evaluate on the dataset

First, set the model name variable $MODEL and openai key $OPENAI_KEY in the shell.

TRIAL=test_trial
mkdir -p $TRIAL
PYTHONPATH=. python evaluation/evaluate.py \
    --model $MODEL \
    --dataset-file $TRIAL/dataset.json \
    --output-dir $TRIAL/ \
    --max-tokens-per-turn 512 \
    --openai-key $OPENAI_KEY

python evaluation/score_breakdown.py \
    --score-file $TRIAL/$MODEL/scores.json \
    --metadata-file $TRIAL/metadata.json

As long as you see the output files $TRIAL/$MODEL/predictions.json, $TRIAL/$MODEL/scores.json, and $TRIAL/summary.json, the evaluation is successfully executed.

Citation

If you find our work useful, please feel free to cite our work:

@article{han2025multiturninstruct,
  author    = {Chi Han},
  title     = {Can Language Models Follow Multiple Turns of Entangled Instructions?},
  journal   = {arXiv preprint},
  volume    = {arXiv:2503.13222},
  year      = {2025},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL},
  url       = {https://arxiv.org/abs/2503.13222}
}