跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-09-30AI 評分44

TauIndianBankBench:評測具流程感知、行動導向能力的工具使用對話智能體

TauIndianBankBench: Procedure-Aware, Action-Oriented Evaluation of Tool-Using Conversational Agents

AI 導讀

TauIndianBankBench 以 1,000 項印度零售銀行任務,評測工具使用對話智能體能否完成交易結果及所有必要步驟,並避免未經要求的更改。只按結果評分時,所有模型中有 6.5–12.0% 的對話雖違反流程,仍被判定成功。

正文

Hi everyone, author here! 👋 Thanks for checking out TauIndianBankBench.

TL;DR: Most agent benchmarks only check the final database state. But many required banking steps, like getting a closure quote before closing a deposit, don't change any records, so an agent can skip them and still be marked successful. TauIndianBankBench has 1,000 Indian retail-banking tasks with a simulated customer. It scores the outcome, and it also checks that the agent completed every required step and made no unrequested changes. With outcome-only scoring, 6.5–12.0% of sessions across all models were counted as successes even though the agent broke the procedure.

📄 Paper: https://arxiv.org/abs/2610.30161
📂 Dataset: https://huggingface.co/datasets/NPCI/tau-indian-banking
💻 Code: https://github.com/npci/TauIndianBankBench

Contributions are welcome, whether that's new tasks, procedure fixes or additional banking flows. Feel free to open a PR on the dataset repo. You're also welcome to evaluate your own models and share the results.

If you find this work useful, please upvote the paper and ❤️ like the dataset. It helps more people discover it!

Happy to answer any questions! 🙏

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co