Willow Inference Server URI Question

[addrum]

Hi! Just got my box setup and been playing around with it and it works quite nicely!

Just had a quick clarification question:

I noticed the https://infer.tovera.io/api/willow url when in the configuration and the readme specifies you can change it to your own inference server; is this something I point to Home Assistant or is the idea to wait for the Willow Inference server release itself and I then host that locally, and then point to my local inference server?

[kristiankielhofner]

Hi, glad to hear!

That field points to a Willow Inference Server instance. We provide a default/demo Willow Inference Server (releasing this week) on a best effort basis.

Do note for the kind of performance we're providing we do more-or-less require CUDA. However, the performance is excellent and supports cards starting from the GTX 1060 6GB (GTX 1070 appears to be the current "sweet spot" for price/performance on the used market). We will be including pointers and benchmarks for very affordable and available GPUs, sample hardware configurations, etc. With the low cost of a Willow Device you can get a used GPU ($100) and Willow ($50) for the roughly same cost as one Raspberry Pi with microphone, speaker, SD card, enclosure, etc.

Of course if you have multiple Willows the cost savings vs Raspberry Pi gets even better as a GPU can be shared for many, many Willow devices.

[BeardedTek]

I was reading through the docs, is it possible to use home assistant's instance of whisper?

[kristiankielhofner]

The Willow Inference Server is very highly optimized to provide the responsiveness and overall user experience of Willow.

The HA STT implementations are unable to provide this level of performance and are currently unsupported. That may change in the future but it's unlikely.

[BeardedTek]

That does make sense, but it would be nice to have more options. I'm definitely going to try the Willow Inference Server once it's released. It would be nice to share tts output with home assistant, just to have a unified voice over multiple platforms.

[kristiankielhofner]

Our first approach would be to go the other direction - bring WIS support to HA with WIS STT/TTS HA components.

Hopefully the community can contribute those to HA this week with the release of WIS :).

[BeardedTek]

I must also add, what you've accomplished in under 2 months is absolutely astounding!!! I can't wait to see where this project goes.

I'd love to test just about anything you release on this. If you need a guinea pig for the esp-32-box-lite, keep me on speed-dial!!!

[kristiankielhofner]

Thank you! We have access to ESP BOX Lites but I'll keep that in mind!

[kristiankielhofner]

WIS released - discuss on HN