Towards Few-Shot Mixed-Type Dialogue Generation

Artificial Intelligence Computer Science Information Science
few-shot learning mixed-type dialogue dialogue generation modular framework task decomposition

A Breakthrough in Mixed-Type Dialogue Generation: Few-Shot Learning Research

One of the significant goals of Artificial Intelligence (AI) is to build agents capable of conducting multiple types of natural language dialogues. The industry and academia have long awaited the creation of dialogue models that can handle both open-domain dialogues and task-oriented dialogues simultaneously, referred to as Mixed-Type Dialogue. However, despite numerous attempts to solve this issue, most studies rely on constructing large-scale manually annotated datasets. This reliance on annotated data is both costly and severely limits the feasibility of real-world applications. To address this challenge, Zeming Liu and his team have made a significant contribution by being the first to propose the Few-Shot Mixed-Type Dialogue Generation task. They have developed a novel solution to tackle this challenge. This article comprehensively dissects the background, methodology, and findings of this groundbreaking research.

Background: Challenges and Opportunities in Dialogue Models

The authors note the scientific importance of designing dialogue models that seamlessly integrate multiple dialogue skills. Specifically, such a model should realize the following three major functions:

  1. Open-Domain Social Dialogue (Persona-Chat): The agent needs to converse with users while showcasing a personalized persona to enhance user engagement.
  2. Knowledge-Grounded Dialogue: The agent should engage in in-depth knowledge-based conversations on specific topics.
  3. Task-Oriented Dialogue: This includes conversational recommendations and goal-oriented dialogues, such as recommending movies or restaurants or assisting users with booking tickets.

Previous research, such as Andrea’s integrated dialogue skill models, Roller’s end-to-end training models, and modularized frameworks, has propelled progress in mixed-type dialogue generation. However, these approaches often depend on large-scale datasets, exhibit high model complexity, or fail to meet the demands for efficiency and flexibility in practical applications.

Zeming Liu’s team identified these bottlenecks and proposed an innovative solution grounded in few-shot learning to enhance the applicability and generation quality of mixed-type dialogues.

Source and Publication Details

This research was jointly conducted by the Research Center for Social Computing and Information Retrieval at Harbin Institute of Technology and Baidu Inc. The paper, titled “Towards Few-Shot Mixed-Type Dialogue Generation”, was published in the journal Science China Information Sciences in February 2025, Volume 68, Issue 2 (DOI: 10.1007/s11432-023-4069-x).

Research Approach and Methodology: Combining Modular Architecture and Few-Shot Learning

At the core of this research is the proposed PLATO-Prompt framework for mixed-type dialogue generation. The researchers provided detailed documentation of its conceptualization and experimental validation.

1. Task Decomposition

The research team decomposed the mixed-type dialogue task into the following three subtasks: - Natural Language Understanding (NLU): Identifying dialogue context and the current user dialogue act. - Dialogue Act Planning (DAP): Planning the agent’s next action based on the context. - Natural Language Generation (NLG): Generating natural language responses aligned with the planned actions and context.

To streamline the structure, the team unified dialogue acts using a format consisting of three dimensions: dialogue type, dialogue topic, and topic attributes, e.g., (Dialogue Type, Dialogue Topic, Topic Attribute).

2. PLATO-Prompt Framework Design

PLATO-Prompt is an enhanced version of the PLATO-2 model, featuring the following technical characteristics:

  • Modular-Based Architecture: The framework allows independent learning and optimization for each subtask.
  • Prompt-Tuning Technology: Task-specific prompts are introduced at the input stage to distinguish between different dialogue types (e.g., social chat, conversational recommendations) or subtasks (e.g., NLU, DAP, NLG).
  • Pre-training and Fine-tuning: The model is pre-trained on multiple publicly available datasets, such as Dulemon, KDConv, and DuRecDial, as well as the team’s own novel Mixed-FS dataset for mixed-type dialogues.

PLATO-Prompt Workflow: The team first performs post-pretraining on the PLATO-2 model and then fine-tunes it on a limited set of mixed-type dialogue data. Compared to traditional autoregressive or end-to-end methods, this framework significantly improves the model’s coherence and interpretability in dialogue generation.

3. Mixed-Type Dialogue Dataset: Mixed-FS

To enable few-shot mixed-type dialogue generation, the research team designed a novel Mixed-FS dataset with the following features: - The dataset spans multiple dialogue types, including knowledge-grounded dialogue, social chat, conversational recommendations, and goal-oriented dialogues. - It integrates a mechanism for dynamically updating user preferences: each dialogue turn captures and adjusts user preferences, such as favorite movie genres, to optimize future recommendations.

Dataset Statistics: Mixed-FS comprises 10 dialogue domains with 100 dialogues, totaling 3,016 utterances, averaging approximately 30 utterances per dialogue.

Additionally, the team constructed a large knowledge graph dataset, Knowledge Base (KG-FS), spanning 10 domains, containing 154K entities and about 1.155M knowledge triples to support Mixed-FS.

Experimental Design and Results

1. Experimental Setup

Experiments were conducted using Mixed-FS and DuRecDial datasets to evaluate the model’s performance on the three subtasks (NLU, DAP, NLG). Comparisons were made with several strong baselines, including: - BST model (BlendedSkillTalk) - PLATO-2 model - Large-scale pre-trained language models such as Baichuan-7B, ChatGLM-6B, and Qwen-7B.

The researchers evaluated the framework under both few-shot and zero-shot settings.

2. Key Results

PLATO-Prompt outperformed all baselines in the following aspects:

  • NLU Subtask: The model achieved superior accuracy and F1 scores in recognizing dialogue types and predicting topics.
  • DAP Subtask: PLATO-Prompt demonstrated exceptional performance in planning dialogue acts with precision and logical consistency.
  • NLG Subtask: For natural language response generation, PLATO-Prompt significantly outperformed other models, especially in human evaluation metrics such as fluency, informativeness, proactivity, and coherence (e.g., fluency and coherence scores exceeded 1.9 on a 2-point scale).

Training Sample Sensitivity Analysis: The researchers also observed that while even a small number of samples significantly improved model performance, increasing the number of training samples resulted in diminishing returns.

Research Conclusions and Significance

  • Innovative Contribution: The study is the first to identify the challenge of few-shot mixed-type dialogue generation and propose a flexible generation framework, PLATO-Prompt.
  • Scientific Value: PLATO-Prompt advances research in human-computer dialogue, offering novel insights into multi-task optimization and integrating multiple dialogue skills.
  • Practical Value: The few-shot learning approach substantially reduces training costs, paving the way for deployment in low-resource real-world environments.
  • Data Contribution: The release of the Mixed-FS dataset and accompanying KG-FS provides valuable resources for future research.

This study not only introduces a new method for mixed-type dialogue generation but also demonstrates its effectiveness in improving dialogue quality through rigorous experimentation. It lays a solid foundation for tackling even more challenging problems in the future, such as zero-shot learning for mixed-type dialogues.