Teaching LLMs to Learn Tool Trialing and Execution through Environment Interaction
Xingjie Gao, Pengcheng Huang, Zhenghao Liu, Yukun Yan, Shuo Wang, Zulong Chen, Chen Qian, Ge Yu, Yu Gu
arXiv preprint
arXiv · Code · Hugging Face · ORKG · Full text (arXiv HTML)
ToolMaster studies how language-model agents learn to use unfamiliar tools by interacting with an environment. A trial phase lets the agent probe available tools and use feedback before executing a task.
Training for tool trialing and execution
The supervised stage uses teacher trajectories that include explicit tool trials and self-corrections. A subsequent stage uses Group Relative Policy Optimization to train trialing and execution jointly. The paper evaluates whether the learned behavior transfers to previously unseen tools. Source
Learning to call previously unseen tools
| Approach | Where the agent learns about an unfamiliar tool |
|---|---|
| ToolLLM | Collects successful multi-step API solutions with a search procedure, then uses the resulting instruction data to train a tool-using model. |
| Simulated Trial and Error (STE) | Simulates useful situations for a tool, interacts with it, and stores lessons from the execution feedback for later use. |
| ToolMaster | Trains an explicit trial phase before task execution. Supervised trajectories model tool probes and self-correction, followed by reinforcement learning over trial and execution together. |
The table compares how methods acquire tool experience. Each paper uses its own evaluation setting.
Evaluating generalization on ToolHop
ToolHop pairs multi-hop questions with interdependent, locally executable tools. ToolMaster uses it as an out-of-domain test. Its Table 2 reports answer correctness on ToolHop and Solvable Pass Rate on TMDB. The ToolHop dataset card documents the original benchmark and its evaluation settings.
Section 5.4 and Figure 4 compare ToolMaster with an ablation trained on successful tool-use trajectories. The analysis groups ToolHop tools by how similar their documentation is to that of tools seen during training. A query is treated as unfamiliar if any required tool falls into the low-similarity group, and as familiar if every required tool falls into the high-similarity group.
Released trajectories and checkpoint
The released SFT file contains 1,444 query records, each with one conversation. Section 4 reports selecting 1,500 ToolBench queries to construct the SFT dataset. The two counts refer to the released file and the paper’s query selection, respectively. The ToolMaster-7B checkpoint is linked from the official repository.