Skip to content

[voice agent] Improve tool calling and logging ux#15269

Merged
stevehuang52 merged 11 commits intomainfrom
heh/va_ux_010726
Jan 8, 2026
Merged

[voice agent] Improve tool calling and logging ux#15269
stevehuang52 merged 11 commits intomainfrom
heh/va_ux_010726

Conversation

@stevehuang52
Copy link
Copy Markdown
Collaborator

Important

The Update branch button must only be pressed in very rare occassions.
An outdated branch is never blocking the merge of a PR.
Please reach out to the automation team before pressing that button.

What does this PR do ?

Improve tool calling and user experience.

Changelog

  • Updated system prompt for more robust switching between using and not using tools.
  • Updated tool docstrings to prompt LLMs to resume interrupted responses (e.g., caused by changing voice speed)
  • Updated README and STT to include nvidia/nemotron-speech-streaming-en-0.6b.

PR Type:

  • New Feature
  • Bugfix
  • Documentation

Signed-off-by: stevehuang52 <heh@nvidia.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>
@stevehuang52 stevehuang52 requested a review from tango4j January 7, 2026 20:25
Signed-off-by: stevehuang52 <heh@nvidia.com>
Signed-off-by: stevehuang52 <heh@nvidia.com>
tango4j

This comment was marked as off-topic.

Signed-off-by: tango4j <tango4j@users.noreply.github.com>
Copy link
Copy Markdown
Collaborator

@tango4j tango4j left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the test file to reflect the changes in this PR.

@tango4j tango4j added Run CICD and removed Run CICD labels Jan 8, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 8, 2026

[🤖]: Hi @stevehuang52 👋,

We wanted to let you know that a CICD pipeline for this PR just finished successfully.

So it might be time to merge this PR or get some approvals.

//cc @chtruong814 @ko3n1g @pablo-garay @thomasdhc

@github-actions github-actions Bot removed the Run CICD label Jan 8, 2026
@stevehuang52 stevehuang52 merged commit 208cf2d into main Jan 8, 2026
60 checks passed
@stevehuang52 stevehuang52 deleted the heh/va_ux_010726 branch January 8, 2026 22:40
AkCodes23 pushed a commit to AkCodes23/NeMo that referenced this pull request Jan 28, 2026
* refactor

Signed-off-by: stevehuang52 <heh@nvidia.com>

* update tts

Signed-off-by: stevehuang52 <heh@nvidia.com>

* improve ux

Signed-off-by: stevehuang52 <heh@nvidia.com>

* fix linting

Signed-off-by: stevehuang52 <heh@nvidia.com>

* refactor tts tool

Signed-off-by: stevehuang52 <heh@nvidia.com>

* Moving voice agent tests from example to test folder

Signed-off-by: taejinp <tango4j@gmail.com>

* Apply isort and black reformatting

Signed-off-by: tango4j <tango4j@users.noreply.github.com>

* Moving the temporary test file to example folder

Signed-off-by: taejinp <tango4j@gmail.com>

---------

Signed-off-by: stevehuang52 <heh@nvidia.com>
Signed-off-by: taejinp <tango4j@gmail.com>
Signed-off-by: tango4j <tango4j@users.noreply.github.com>
Co-authored-by: taejinp <tango4j@gmail.com>
Co-authored-by: tango4j <tango4j@users.noreply.github.com>
Signed-off-by: Akhil Varanasi <akhilvaranasi23@gmail.com>
nune-tadevosyan pushed a commit to nune-tadevosyan/NeMo that referenced this pull request Mar 13, 2026
* refactor

Signed-off-by: stevehuang52 <heh@nvidia.com>

* update tts

Signed-off-by: stevehuang52 <heh@nvidia.com>

* improve ux

Signed-off-by: stevehuang52 <heh@nvidia.com>

* fix linting

Signed-off-by: stevehuang52 <heh@nvidia.com>

* refactor tts tool

Signed-off-by: stevehuang52 <heh@nvidia.com>

* Moving voice agent tests from example to test folder

Signed-off-by: taejinp <tango4j@gmail.com>

* Apply isort and black reformatting

Signed-off-by: tango4j <tango4j@users.noreply.github.com>

* Moving the temporary test file to example folder

Signed-off-by: taejinp <tango4j@gmail.com>

---------

Signed-off-by: stevehuang52 <heh@nvidia.com>
Signed-off-by: taejinp <tango4j@gmail.com>
Signed-off-by: tango4j <tango4j@users.noreply.github.com>
Co-authored-by: taejinp <tango4j@gmail.com>
Co-authored-by: tango4j <tango4j@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants