Pengfei Zhang

I.e Identity Decay in Multi-Turn Human-Centric Image Editing: A Benchmark and a Multimodal Agentic Correction Framework

Under Review

Introduces a benchmark measuring identity and fidelity decay in multi-turn human-centric image editing, and a correction pipeline in which a multimodal agent orchestrates identity correction using pretrained human segmentation and parsing models (Sapiens for body, DML_CSR for face) together with deterministic image operators such as TPS warp and high-frequency preservation transfer, flattening the degradation curve.

I.d KinMo: Kinematic-aware Human Motion Understanding and Generation

ICCV 2025

A VLM-based dataset enhancement pipeline that extends existing text-motion alignment datasets with a body-decomposition motion representation, enabling precise motion descriptions, plus a hierarchical motion-text alignment approach that fuses motion detail into text semantics for finer motion control.

project / preprint / demo

I.c LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey

ACL 2026

project / paper

I.b Contextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation

ACM MM 2025

Improves co-speech gesture video generation through chronological speech-gesture alignment that temporally connects the two modalities, and a contextualized gesture tokenization that folds speech context into motion pattern representation through distillation.

project / preprint

I.a Handformer2T: A Lightweight Regression-based Model for Interacting Hands Pose Estimation from a Single RGB Image

WACV 2024

A lightweight model using hand-level tokenization in a transformer-based architecture for interacting hand pose estimation, where only one token is used per hand.

paper

II.f AP-GRPO: Anchor-Gated Phonetic Alignment with Policy Optimization for Pathological Speech Reconstruction

EMNLP 2026 (accepted)

A GRPO framework with a phonetic reward that aligns speech language models to the original speech signal through audible-anchor preservation and inter-anchor phonetic compatibility, combining an anchor-gated reward that matches anchors in clear regions with an inter-anchor phonetic alignment reward that checks whether recovered content is phonetically supported by the corrupted span.

preprint

II.e DemMA: Dementia Multi-Turn Dialogue Agent with Expert-Guided Reasoning and Action Simulation

ACL 2026

An expert-guided multi-turn dialogue agent that simulates dementia patients with clinically grounded personas by distilling expert-guided reasoning and action simulation.

paper / code / data

II.d MedSpeak: Knowledge Enhanced ASR Error Correction Framework for Spoken Medical Question Answering

ICASSP 2026

A knowledge-enhanced retrieval-augmented generation framework built on a knowledge graph capturing semantic medical relationships and phonetic similarities between medical terms.

preprint / code

II.c AudioLens: Multi-Perspective Speech Clustering with Audio-Language Models

Under Review

Introduces audio multi-perspective clustering, where a model partitions speech recordings according to a natural-language perspective while inferring both the number of clusters and their assignments. Contributes AudioLens-Bench, a benchmark evaluating cross-perspective generalization, and AudioLens-R1, a large audio-language model trained with reasoning distillation and direct preference optimization.

II.b From Sounds to Scenes: A Benchmark for Evaluating Context-Aware Auditory Scene Understanding in Large Audio Language Models

Under Review

A benchmark evaluating whether large audio-language models can jointly interpret speech, events and scene-level information to perform coherent semantic reasoning.

preprint

II.a Hallucination Rates of Speech Language Models on Pathological Speech Recognition: A Pilot Multi-Cohort Audit

BSN 2025

An evaluation of popular speech language models on pathological speech recognition across hallucination types: insertion, fluent completion, entity fabrication and semantic substitution.

III.c Can the Environment Speak for Itself? T2-GRPO: Turn-Trajectory Group Relative Policy Optimization for Caregiver Agents

Under Review

A turn-trajectory optimization scheme for long-horizon caregiver agents that derives dense turn-level rewards from patient-state transitions, normalizes them separately from trajectory-level outcomes via centered-rank group-relative advantages, and adds a binary safety veto to enforce hard behavioral constraints.

III.b DEMENTIA-PLAN: An Agent-Based Framework for Multi-Knowledge Graph Retrieval-Augmented Generation in Dementia Care

AAAI 2025

A retrieval-augmented generation framework that improves conversations with mild-stage dementia patients by integrating multiple knowledge graphs with LLMs.

preprint

III.a Adaptive Constraint Relaxation in Personalized Nutrition Recommendations: An LLM-Driven Knowledge Graph Retrieval Approach

AMIA 2025 (Oral)

An LLM-driven knowledge graph retrieval system for personalized food recommendation, resolving the all-or-nothing limitation of traditional recommendation systems.

preprint