> This is a page from the ElevenLabs documentation. For a complete page index, fetch https://elevenlabs.io/docs/llms.txt. For the full documentation in a single file, fetch https://elevenlabs.io/docs/llms-full.txt. # Build a Voice Assistant with Agents Platform on a Raspberry Pi > **Note** > > **Tutorial** ยท Assumes you have completed the [ElevenAgents quickstart](/docs/eleven-agents/quickstart) and have a Raspberry Pi with Python installed. ## Introduction In this tutorial you will learn how to build a voice assistant with Agents Platform running on a Raspberry Pi. Just like conventional home assistants like Alexa on Amazon Echo, Google Home, or Siri on Apple devices, your Eleven Voice assistant will listen to a hotword, in our case "Hey Eleven", and then initiate an ElevenLabs Agents session to assist the user. ## Requirements * A Raspberry Pi 5 or similar device. * A microphone and speaker. * Python 3.9 or higher installed on your machine. * An ElevenLabs account with an [API key](https://elevenlabs.io/app/settings/api-keys). ## Setup ### Install dependencies On Debian-based systems you can install the dependencies with: ```bash sudo apt-get update sudo apt-get install libportaudio2 libportaudiocpp0 portaudio19-dev libasound-dev libsndfile1-dev -y ``` ### Create the project On your Raspberry Pi, open the terminal and create a new directory for your project. ```bash mkdir eleven-voice-assistant cd eleven-voice-assistant ``` Create a new virtual environment and install the dependencies: ```bash python -m venv .venv # Only required the first time you set up the project source .venv/bin/activate ``` Install the dependencies: ```bash pip install tflite-runtime pip install librosa pip install EfficientWord-Net pip install elevenlabs pip install "elevenlabs[pyaudio]" ``` Now create a new python file called `hotword.py` and add the following code: **`hotword.py`** ```python hotword.py import os import signal import time from eff_word_net.streams import SimpleMicStream from eff_word_net.engine import HotwordDetector from eff_word_net.audio_processing import Resnet50_Arc_loss # from eff_word_net import samples_loc from elevenlabs.client import ElevenLabs from elevenlabs.conversational_ai.conversation import Conversation, ConversationInitiationData from elevenlabs.conversational_ai.default_audio_interface import DefaultAudioInterface convai_active = False elevenlabs = ElevenLabs() agent_id = os.getenv("ELEVENLABS_AGENT_ID") api_key = os.getenv("ELEVENLABS_API_KEY") dynamic_vars = { 'user_name': 'Thor', 'greeting': 'Hey' } config = ConversationInitiationData( dynamic_variables=dynamic_vars ) base_model = Resnet50_Arc_loss() eleven_hw = HotwordDetector( hotword="hey_eleven", model = base_model, reference_file=os.path.join("hotword_refs", "hey_eleven_ref.json"), threshold=0.7, relaxation_time=2 ) def create_conversation(): """Create a new conversation instance""" return Conversation( # API client and agent ID. elevenlabs, agent_id, config=config, # Assume auth is required when API_KEY is set. requires_auth=bool(api_key), # Use the default audio interface. audio_interface=DefaultAudioInterface(), # Simple callbacks that print the conversation to the console. callback_agent_response=lambda response: print(f"Agent: {response}"), callback_agent_response_correction=lambda original, corrected: print(f"Agent: {original} -> {corrected}"), callback_user_transcript=lambda transcript: print(f"User: {transcript}"), # Uncomment if you want to see latency measurements. # callback_latency_measurement=lambda latency: print(f"Latency: {latency}ms"), ) def start_mic_stream(): """Start or restart the microphone stream""" global mic_stream try: # Always create a new stream instance mic_stream = SimpleMicStream( window_length_secs=1.5, sliding_window_secs=0.75, ) mic_stream.start_stream() print("Microphone stream started") except Exception as e: print(f"Error starting microphone stream: {e}") mic_stream = None time.sleep(1) # Wait a bit before retrying def stop_mic_stream(): """Stop the microphone stream safely""" global mic_stream try: if mic_stream: # SimpleMicStream doesn't have a stop_stream method # We'll just set it to None and recreate it next time mic_stream = None print("Microphone stream stopped") except Exception as e: print(f"Error stopping microphone stream: {e}") # Initialize microphone stream mic_stream = None start_mic_stream() print("Say Hey Eleven ") while True: if not convai_active: try: # Make sure we have a valid mic stream if mic_stream is None: start_mic_stream() continue frame = mic_stream.getFrame() result = eleven_hw.scoreFrame(frame) if result == None: #no voice activity continue if result["match"]: print("Wakeword uttered", result["confidence"]) # Stop the microphone stream to avoid conflicts stop_mic_stream() # Start ConvAI Session print("Start ConvAI Session") convai_active = True try: # Create a new conversation instance conversation = create_conversation() # Start the session conversation.start_session() # Set up signal handler for graceful shutdown def signal_handler(sig, frame): print("Received interrupt signal, ending session...") try: conversation.end_session() except Exception as e: print(f"Error ending session: {e}") signal.signal(signal.SIGINT, signal_handler) # Wait for session to end conversation_id = conversation.wait_for_session_end() print(f"Conversation ID: {conversation_id}") except Exception as e: print(f"Error during conversation: {e}") finally: # Cleanup convai_active = False print("Conversation ended, cleaning up...") # Give some time for cleanup time.sleep(1) # Restart microphone stream start_mic_stream() print("Ready for next wake word...") except Exception as e: print(f"Error in wake word detection: {e}") # Try to restart microphone stream if there's an error mic_stream = None time.sleep(1) start_mic_stream() ``` ## Agent configuration #### Sign in to ElevenLabs Go to [elevenlabs.io](https://elevenlabs.io/app/sign-up) and sign in to your account. #### Create a new agent Navigate to [Agents Platform > Agents](https://elevenlabs.io/app/agents/agents) and create a new agent from the blank template. #### Set the first message Set the first message and specify the dynamic variable for the platform. ```txt {{greeting}} {{user_name}}, Eleven here, what's up? ``` #### Set the system prompt Set the system prompt. You can find our best practises docs [here](/docs/eleven-agents/best-practices/prompting-guide). ```txt You are a helpful Agents Platform assistant with access to a weather tool. When users ask about weather conditions, use the get_weather tool to fetch accurate, real-time data. The tool requires a latitude and longitude - use your geographic knowledge to convert location names to coordinates accurately. Never ask users for coordinates - you must determine these yourself. Always report weather information conversationally, referring to locations by name only. For weather requests: 1. Extract the location from the user's message 2. Convert the location to coordinates and call get_weather 3. Present the information naturally and helpfully For non-weather queries, provide friendly assistance within your knowledge boundaries. Always be concise, accurate, and helpful. ``` #### Set up a webhook tool We'll set up a simple webhook tool that will fetch the weather data for us. Follow the setup steps [here](/docs/eleven-agents/customization/tools/webhook-tools#configure-the-weather-tool) to set up the tool. ## Run the app To run the app, first set the required environment variables: ```bash export ELEVENLABS_API_KEY=YOUR_API_KEY export ELEVENLABS_AGENT_ID=YOUR_AGENT_ID ``` Then simply run the following command: ```bash python hotword.py ``` Now say "Hey Eleven" to start the conversation. Happy chattin'! ## \[Optional] Train your custom hotword ### Generate training audio To generate the hotword embeddings, you can use ElevenLabs to generate four training samples. Simply navigate to [Text To Speech](https://elevenlabs.io/app/speech-synthesis/text-to-speech) within your ElevenLabs app, and type in your hotword, e.g. "Hey Eleven". Select a voice and click on the "Generate" button. After the audio has been generated, download the audio file and save them into a folder called `hotword_training_audio` at the root of your project. Repeat this process three more times with different voices. ### Train the hotword In your terminal, with your virtual environment activated, run the following command to train the hotword: ```bash python -m eff_word_net.generate_reference --input-dir hotword_training_audio --output-dir hotword_refs --wakeword hey_eleven --model-type resnet_50_arc ``` This will generate the `hey_eleven_ref.json` file in the `hotword_refs` folder. Now you simply need to update the `reference_file` parameter in the `HotwordDetector` class in `hotword.py` to point to the new reference file and you're good to go! ## Next steps #### [ElevenAgents](/docs/eleven-agents/quickstart) Build a fully managed voice agent without handling audio streams manually. #### [TTS quickstart](/docs/eleven-api/quickstart) Explore more text-to-speech options and voice customisation. > ElevenLabs provides APIs and SDKs for text to speech, voice cloning, speech to text, sound effects, voice isolator, voice changer, and conversational AI agents. Build voice-enabled applications with lifelike audio generation.