ESP32-S3-BOX-3 deploys Home Assistant localized voice assistant, 3 personalized teaching methods – Home Assistant step-by-step teaching EP 10

Everyone should be familiar with controlling smart homes through voice. Even on mobile phones, we often use Hey Siri or OK Google... to call voice assistants. Continuation of the previous article:

A truly smart home with brains! Home Assistant customized AI voice assistant ChatGPT, Gemini makes your smart home come alive – Home Assistant step-by-step tutorial EP 9

neiltw.com

We replaced the Home Assistant voice assistant with Google Gemini or ChatGPT, so that the response of the Home Assistant voice assistant would not be as stiff as Siri and would be smarter. It could even be assigned a personality according to your own preferences, but at that time it could only interact with typing. .

Today I will show you how to use ESP32-S3-BOX-3 to build your own Home Assistant. This is better than synchronizing the device to HomeKit or Google Home through HomeKit Bridge or Google Home integration, and then using Siri or Google Assistant voice control. It is more direct and you can create a more suitable voice assistant according to your own preferences👍

Let’s take a look at the final effect first:

Great! It’s so easy to experience the life of an ancient emperor🤣

Without further ado, let’s get started!

ESP32-S3-BOX-3 unboxing

Contents:

1. ESP32-S3-BOX-3 Me: This is the host computer. It can be used directly regardless of other accessories being plugged in.
2. DOCK: Base, with interfaces on it to connect other devices in series.
3. SENSOR: It has the same function as DOCK, but it has additional temperature and humidity sensors, infrared transmitter/receiver, radar sensor, rechargeable battery slot, and MicroSD slot. Although I can’t use them today, I use this one because it’s more handsome🤣
4. BRACKET: Adapter accessory, you can use this to install the host on other devices.
5. BREAD: Adapter accessory, used to connect to breadboard.
6. transmission line
7. RGD module, Dupont line

For what we are going to do today, you can actually buy "ESP32-S3-BOX-3B", which includes the host and DOCK, and the others will not be used. Think about it when you want to buy it.

Step 1. Install and launch the ESPHome add-on

Step 1-1. "Settings" on the left side of Home Assistant ➜ "Add-ons"

Step 1-2. Install "ESPHome"

1. Click "Add-on Store" in the lower right corner

2. Find "ESPHome" and click

3. Install "ESPHome"

4. Turn on Watchdog, automatically update, and display in the sidebar
5. Click "Start"

6. "ESPHome" startup completed!

Step 2. Install the openWakeWord add-on

Step 2-1. "Settings" on the left side of Home Assistant ➜ "Add-ons"

Step 2-2. Install "openWakeWord"

1. Click "Add-on Store" in the lower right corner

2. Find "openWakeWord" and click

3. Install "openWakeWord"

4. Turn on Watchdog and automatically update
5. Click "Start"

6. "openWakeWord" startup completed!

Step 3. Add "openWakeWord" integration

Step 3-1. "Settings" on the left side of Home Assistant ➜ "Devices and Services"

Step 3-2. Set up "openWakeWord" integration

1. At this time, "openWakeWord" will appear in the "Scanned" section, click "Settings"
2. The system will ask whether to use Wyoming Protocol, click "Send"
3. Click "Finish" in the success message window

Step 4. Set the voice assistant wake word

Step 4-1. Home Assistant "Settings" on the left ➜ "Voice Assistant"

Step 4-2. Select voice assistant

Step 4-3. Set wake word

1. Wake word engine selects "openwakeword"
2. Choose a wake-up word that you like. Here is an example of choosing "ok nabu"

Now that you have set the wake word, proceed to the installation and settings of ESP32-S3-BOX-3!

Step 5. Install and set up ESP32-S3-BOX-3

⚠️Attention! If you have installed S3 BOX in the past, be sure to go to "Settings > Devices and Services > ESPHome" to find the S3 BOX and delete it⚠️

Step 5-1. Flash ESP32-S3-BOX-3 firmware

1. Go to the ESPHome project website:https://esphome.io/projects/index.html
2. Select "Voice assistant"
3. Select "ESP32-S3-BOX-3"
4. Click "Connect"

5. Connect ESP32-S3-BOX-3 to your computer through the transmission cable
6. Select the ESP32-S3-BOX-3 device
7. Click "Connect"

8. Select "Install ESP32 S3 Box 3"
9. Click "Install"
10. Click "Next"

Step 5-2. Set up WiFi connection

Enter your home WiFi connection information and click "Connect"

Step 5-3. Add the device to Home Assistant

1. Click "Add to Home Assistant"

2. Confirm Home Assistant physical address
   The default is "http://homeassistant.local:8123". If you change it when installing Home Assistant, remember to change it to a URL that can connect to Home Assistant!
3. Click "Open Link"

4. At this time you will automatically be taken to the Home Assistant integration list, select "OK" in the pop-up window
5. Select "ESP32 S3 Box 3" in the Add discovered device menu
6. Select "Send" in the pop-up window

Step 6. Check whether the ESP32-S3-BOX-3 integration is successful

1. Home Assistant left column "Settings" > "Devices and Services"
2. Find ESPHome click device count

At this time, you can see the ESP32-S3-BOX-3 related device information, and you can mute and switch the screen, then you have successfully integrated it!

Try talking to the voice assistant!

Also featured – Personalize your voice assistant

Just switching to a voice assistant with a brain must still be too boring, right? ! Try changing your profile, customizing your display picture, and customizing your wake word!

Change persona

1. Click on the right side of the conversation assistant menu⚙️
2. Add the role you want it to play in the instructions
3. Click "Send"

In this way, you can change it into a variety of characters you like! Share with me what kind of persona do you like to change into? 🤣

Customize voice assistant display picture

Frankly speaking, the default display image is really boring, right? Then try changing the picture! You can even change it to a photo of yourself (if you are narcissistic enough🤣), as if there is another self helping you handle big and small things, cool~

Before starting, take a look at the picture below to understand the 6 states of the Home Assistant voice assistant:

① Enter ESPHome web UI

Usually it will be in the sidebar. If not, go to "Settings > Add-ons" to find it.

② Put ESP32-S3-BOX-3 into management (ADOPT)

This will allow you to adjust various settings. Once completed you will see the status in the upper right corner change from DISCOVERED to ONLINE

③ Select picture material or create picture material

In Home Assistant Github repository There are materials shared for everyone to use. If you are like me and have no artistic talent, you can just find one you like from there.

Of course, you can also draw it yourself, or let AI draw it for you, and refer to the format and specifications to produce relevant picture material files:

1. Size: 320×240. If the provided material is not in 4:3 ratio, the remaining area will be filled with background color.
2. Image format: PNG, JPEG, SVG.
3. In the image, you can use two background colors to roughly distinguish whether the voice assistant is active or inactive:
   • loading, idle state: dark background.
   • listening, thinking, reply status: light background.
   • error status: dark or light background is fine.
4. If the background of your image is left transparent, you can set the background color in the configuration in the following steps.
5. Prepare 6 pictures and put them into a folder
6. Place the folder containing 6 pictures into the folder of config/ESPHome

④ Edit ESP32-S3-BOX-3 configuration file

Find ESP32-S3-BOX-3 in ESPHome web UI and click "EDIT"

At this time you will see the yaml editor, like this:

If you choose Home Assistant Github repository Material library, place the following code into substitutions In the block:
※ Remember to change the path to the material path you want

substitutions:
  loading_illustration_file: https://github.com/jlpouffier/home-assistant-s3-box-community-illustrations/raw/main/jarvis_female/illustrations/loading.png
  idle_illustration_file: https://github.com/jlpouffier/home-assistant-s3-box-community-illustrations/raw/main/jarvis_female/illustrations/idle.png
  listening_illustration_file: https://github.com/jlpouffier/home-assistant-s3-box-community-illustrations/raw/main/jarvis_female/illustrations/listening.png
  thinking_illustration_file: https://github.com/jlpouffier/home-assistant-s3-box-community-illustrations/raw/main/jarvis_female/illustrations/thinking.png
  replying_illustration_file: https://github.com/jlpouffier/home-assistant-s3-box-community-illustrations/raw/main/jarvis_female/illustrations/replying.png
  error_illustration_file: https://github.com/jlpouffier/home-assistant-s3-box-community-illustrations/raw/main/jarvis_female/illustrations/error.png

If you are making your own material, place the following code into substitutions block and set the background color of each state (if the image has a transparent background):
※ Change the example voice_assistant_gfx Replace it with the name of your material folder
※ Remember to change the background color of each status (line 8~13 in the example below)

substitutions:
  loading_illustration_file: voice_assistant_gfx/loading.png
  idle_illustration_file: voice_assistant_gfx/idle.png
  listening_illustration_file: voice_assistant_gfx/listening.png
  thinking_illustration_file: voice_assistant_gfx/thinking.png
  replying_illustration_file: voice_assistant_gfx/replying.png
  error_illustration_file: voice_assistant_gfx/error.png
  loading_illustration_background_color: '000000'
  idle_illustration_background_color: '000000'
  listening_illustration_background_color: 'FFFFFF'
  thinking_illustration_background_color: 'FFFFFF'
  replying_illustration_background_color: 'FFFFFF'
  error_illustration_background_color: '000000'

⑤ Save and install

This step will take different installation times depending on the hardware environment you are using, so please be patient! Once completed, you can see the new display image on the screen! ~

※ I used Raspberry Pi 4 + 8G RAM (Raspberry Pi 4 Model B/8G) and ran it for about 30 minutes

I unexpectedly found that this is a perfect match for my desktop🤣

There is still a lot to play with in ESP32-S3-BOX-3, and there is also self-made firmware by foreign masters to give the BOX’s touch screen more functions. I will find time to try it in the future.

I hope today’s article can be helpful to you who have joined Home Assistant in building a localized voice assistant. You are also welcome to leave a message below to interact with me! 👋