使用 Expo React Native 构建跨平台语音智能体

教程 · 假设你已完成 ElevenAgents 快速入门，并已设置 Expo 开发环境。

简介

本教程将介绍如何使用支持 WebRTC 的 Expo React Native 和 ElevenLabs React Native SDK，构建可在 iOS 和 Android 上运行的语音智能体。

要求

拥有 API 密钥的 ElevenLabs 账户。
本机已安装 Node.js v18 或更高版本。

设置

创建新的 Expo 项目

使用 create-expo-app 创建新的空白 Expo 项目：

npx create-expo-app@latest --template blank-typescript

安装依赖项

安装 ElevenLabs React Native SDK 及其依赖项：

npx expo install @elevenlabs/react-native @livekit/react-native @livekit/react-native-webrtc @config-plugins/react-native-webrtc @livekit/react-native-expo-plugin @livekit/react-native-expo-plugin livekit-client

如果遇到对等依赖项问题，请在项目根目录添加 .npmrc 文件，并写入以下内容：legacy-peer-deps=true。

启用麦克风权限并添加 Expo 插件

在 app.json 文件中添加以下权限：

app.json

{
  "expo": {
    "scheme": "elevenlabs",
    // ...
    "ios": {
      "infoPlist": {
        "NSMicrophoneUsageDescription": "This app uses the microphone to record audio."
      },
      "supportsTablet": true,
      "bundleIdentifier": "YOUR.BUNDLE.ID"
    },
    "android": {
      "permissions": [
        "android.permission.RECORD_AUDIO",
        "android.permission.ACCESS_NETWORK_STATE",
        "android.permission.CAMERA",
        "android.permission.INTERNET",
        "android.permission.MODIFY_AUDIO_SETTINGS",
        "android.permission.SYSTEM_ALERT_WINDOW",
        "android.permission.WAKE_LOCK",
        "android.permission.BLUETOOTH"
      ],
      "adaptiveIcon": {
        "foregroundImage": "./assets/adaptive-icon.png",
        "backgroundColor": "#ffffff"
      },
      "package": "YOUR.PACKAGE.ID"
    },
    "plugins": ["@livekit/react-native-expo-plugin", "@config-plugins/react-native-webrtc"]
    // ...
  }
}

这样 React Native 就能在开始对话时请求麦克风权限。

注意

对于 Android 模拟器，需要在模拟器麦克风设置中启用“虚拟麦克风使用主机音频输入”。

将 ElevenLabs Agents 添加到应用

在 ./App.tsx 文件中添加以下代码，将 ElevenLabs Agents 添加到应用：

./App.tsx

import { ConversationProvider, useConversation } from "@elevenlabs/react-native";
import type { ConversationStatus, ConversationEvent, Role } from "@elevenlabs/react-native";
import React, { useState } from "react";
import {
  View,
  Text,
  StyleSheet,
  TouchableOpacity,
  Keyboard,
  TouchableWithoutFeedback,
  Platform,
} from "react-native";
import { TextInput } from "react-native";
import { getBatteryLevel, changeBrightness, flashScreen } from "./utils/tools";
const ConversationScreen = () => {
  const conversation = useConversation({
    clientTools: {
      getBatteryLevel,
      changeBrightness,
      flashScreen,
    },
    onConnect: ({ conversationId }: { conversationId: string }) => {
      console.log("✅ Connected to conversation", conversationId);
    },
    onDisconnect: (details: string) => {
      console.log("❌ Disconnected from conversation", details);
    },
    onError: (message: string, context?: Record<string, unknown>) => {
      console.error("❌ Conversation error:", message, context);
    },
    onMessage: ({ message, source }: { message: ConversationEvent; source: Role }) => {
      console.log(`💬 Message from ${source}:`, message);
    },
    onModeChange: ({ mode }: { mode: "speaking" | "listening" }) => {
      console.log(`🔊 Mode: ${mode}`);
    },
    onStatusChange: ({ status }: { status: ConversationStatus }) => {
      console.log(`📡 Status: ${status}`);
    },
    onCanSendFeedbackChange: ({ canSendFeedback }: { canSendFeedback: boolean }) => {
      console.log(`🔊 Can send feedback: ${canSendFeedback}`);
    },
  });
  const [isStarting, setIsStarting] = useState(false);
  const [textInput, setTextInput] = useState("");
  const handleSubmitText = () => {
    if (textInput.trim()) {
      conversation.sendUserMessage(textInput.trim());
      setTextInput("");
      Keyboard.dismiss();
    }
  };
  const startConversation = async () => {
    if (isStarting) return;
    setIsStarting(true);
    try {
      await conversation.startSession({
        agentId: process.env.EXPO_PUBLIC_AGENT_ID,
        dynamicVariables: {
          platform: Platform.OS,
        },
      });
    } catch (error) {
      console.error("Failed to start conversation:", error);
    } finally {
      setIsStarting(false);
    }
  };
  const endConversation = async () => {
    try {
      await conversation.endSession();
    } catch (error) {
      console.error("Failed to end conversation:", error);
    }
  };
  const getStatusColor = (status: ConversationStatus): string => {
    switch (status) {
      case "connected":
        return "#10B981";
      case "connecting":
        return "#F59E0B";
      case "disconnected":
        return "#EF4444";
      default:
        return "#6B7280";
    }
  };
  const getStatusText = (status: ConversationStatus): string => {
    return status[0].toUpperCase() + status.slice(1);
  };
  const canStart = conversation.status === "disconnected" && !isStarting;
  const canEnd = conversation.status === "connected";
  return (
    <TouchableWithoutFeedback onPress={() => Keyboard.dismiss()}>
      <View style={styles.container}>
        <Text style={styles.title}>ElevenLabs React Native Example</Text>
        <Text style={styles.subtitle}>Remember to set the agentId in the .env file!</Text>
        <View style={styles.statusContainer}>
          <View
            style={[styles.statusDot, { backgroundColor: getStatusColor(conversation.status) }]}
          />
          <Text style={styles.statusText}>{getStatusText(conversation.status)}</Text>
        </View>
        {/* Speaking Indicator */}
        {conversation.status === "connected" && (
          <View style={styles.speakingContainer}>
            <View
              style={[
                styles.speakingDot,
                {
                  backgroundColor: conversation.isSpeaking ? "#8B5CF6" : "#D1D5DB",
                },
              ]}
            />
            <Text
              style={[
                styles.speakingText,
                { color: conversation.isSpeaking ? "#8B5CF6" : "#9CA3AF" },
              ]}
            >
              {conversation.isSpeaking ? "🎤 AI Speaking" : "👂 AI Listening"}
            </Text>
          </View>
        )}
        <View style={styles.buttonContainer}>
          <TouchableOpacity
            style={[styles.button, styles.startButton, !canStart && styles.disabledButton]}
            onPress={startConversation}
            disabled={!canStart}
          >
            <Text style={styles.buttonText}>
              {isStarting ? "Starting..." : "Start Conversation"}
            </Text>
          </TouchableOpacity>
          <TouchableOpacity
            style={[styles.button, styles.endButton, !canEnd && styles.disabledButton]}
            onPress={endConversation}
            disabled={!canEnd}
          >
            <Text style={styles.buttonText}>End Conversation</Text>
          </TouchableOpacity>
        </View>
        {/* Feedback Buttons */}
        {conversation.status === "connected" && conversation.canSendFeedback && (
          <View style={styles.feedbackContainer}>
            <Text style={styles.feedbackLabel}>How was that response?</Text>
            <View style={styles.feedbackButtons}>
              <TouchableOpacity
                style={[styles.button, styles.likeButton]}
                onPress={() => conversation.sendFeedback(true)}
              >
                <Text style={styles.buttonText}>👍 Like</Text>
              </TouchableOpacity>
              <TouchableOpacity
                style={[styles.button, styles.dislikeButton]}
                onPress={() => conversation.sendFeedback(false)}
              >
                <Text style={styles.buttonText}>👎 Dislike</Text>
              </TouchableOpacity>
            </View>
          </View>
        )}
        {/* Text Input and Messaging */}
        {conversation.status === "connected" && (
          <View style={styles.messagingContainer}>
            <Text style={styles.messagingLabel}>Send Text Message</Text>
            <TextInput
              style={styles.textInput}
              value={textInput}
              onChangeText={(text) => {
                setTextInput(text);
                // Prevent agent from interrupting while user is typing
                if (text.length > 0) {
                  conversation.sendUserActivity();
                }
              }}
              placeholder="Type your message or context... (Press Enter to send)"
              multiline
              onSubmitEditing={handleSubmitText}
              returnKeyType="send"
              blurOnSubmit={true}
            />
            <View style={styles.messageButtons}>
              <TouchableOpacity
                style={[styles.button, styles.messageButton]}
                onPress={handleSubmitText}
                disabled={!textInput.trim()}
              >
                <Text style={styles.buttonText}>💬 Send Message</Text>
              </TouchableOpacity>
              <TouchableOpacity
                style={[styles.button, styles.contextButton]}
                onPress={() => {
                  if (textInput.trim()) {
                    conversation.sendContextualUpdate(textInput.trim());
                    setTextInput("");
                    Keyboard.dismiss();
                  }
                }}
                disabled={!textInput.trim()}
              >
                <Text style={styles.buttonText}>📝 Send Context</Text>
              </TouchableOpacity>
            </View>
          </View>
        )}
      </View>
    </TouchableWithoutFeedback>
  );
};
export default function App() {
  return (
    <ConversationProvider>
      <ConversationScreen />
    </ConversationProvider>
  );
}
const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: "center",
    alignItems: "center",
    backgroundColor: "#F3F4F6",
    padding: 20,
  },
  title: {
    fontSize: 24,
    fontWeight: "bold",
    marginBottom: 8,
    color: "#1F2937",
  },
  subtitle: {
    fontSize: 16,
    color: "#6B7280",
    marginBottom: 32,
  },
  statusContainer: {
    flexDirection: "row",
    alignItems: "center",
    marginBottom: 24,
  },
  statusDot: {
    width: 12,
    height: 12,
    borderRadius: 6,
    marginRight: 8,
  },
  statusText: {
    fontSize: 16,
    fontWeight: "500",
    color: "#374151",
  },
  speakingContainer: {
    flexDirection: "row",
    alignItems: "center",
    marginBottom: 24,
  },
  speakingDot: {
    width: 12,
    height: 12,
    borderRadius: 6,
    marginRight: 8,
  },
  speakingText: {
    fontSize: 14,
    fontWeight: "500",
  },
  toolsContainer: {
    backgroundColor: "#E5E7EB",
    padding: 16,
    borderRadius: 8,
    marginBottom: 24,
    width: "100%",
  },
  toolsTitle: {
    fontSize: 14,
    fontWeight: "600",
    color: "#374151",
    marginBottom: 8,
  },
  toolItem: {
    fontSize: 12,
    color: "#6B7280",
    fontFamily: "monospace",
    marginBottom: 4,
  },
  buttonContainer: {
    width: "100%",
    gap: 16,
  },
  button: {
    backgroundColor: "#3B82F6",
    paddingVertical: 16,
    paddingHorizontal: 32,
    borderRadius: 8,
    alignItems: "center",
  },
  startButton: {
    backgroundColor: "#10B981",
  },
  endButton: {
    backgroundColor: "#EF4444",
  },
  disabledButton: {
    backgroundColor: "#9CA3AF",
  },
  buttonText: {
    color: "white",
    fontSize: 16,
    fontWeight: "600",
  },
  instructions: {
    marginTop: 24,
    fontSize: 14,
    color: "#6B7280",
    textAlign: "center",
    lineHeight: 20,
  },
  feedbackContainer: {
    marginTop: 24,
    alignItems: "center",
  },
  feedbackLabel: {
    fontSize: 16,
    fontWeight: "500",
    color: "#374151",
    marginBottom: 12,
  },
  feedbackButtons: {
    flexDirection: "row",
    gap: 16,
  },
  likeButton: {
    backgroundColor: "#10B981",
  },
  dislikeButton: {
    backgroundColor: "#EF4444",
  },
  messagingContainer: {
    marginTop: 24,
    width: "100%",
  },
  messagingLabel: {
    fontSize: 16,
    fontWeight: "500",
    color: "#374151",
    marginBottom: 8,
  },
  textInput: {
    backgroundColor: "#FFFFFF",
    borderRadius: 8,
    padding: 16,
    minHeight: 100,
    textAlignVertical: "top",
    borderWidth: 1,
    borderColor: "#D1D5DB",
    marginBottom: 16,
  },
  messageButtons: {
    flexDirection: "row",
    gap: 16,
  },
  messageButton: {
    backgroundColor: "#3B82F6",
    flex: 1,
  },
  contextButton: {
    backgroundColor: "#4F46E5",
    flex: 1,
  },
  activityContainer: {
    marginTop: 24,
    alignItems: "center",
  },
  activityLabel: {
    fontSize: 14,
    color: "#6B7280",
    marginBottom: 8,
    textAlign: "center",
  },
  activityButton: {
    backgroundColor: "#F59E0B",
  },
});

原生客户端工具

构建 ElevenLabs 智能体的重要部分，是让智能体能够动态访问和执行功能。这可以通过客户端工具实现。

新建一个文件存放客户端工具：./utils/tools.ts，并添加以下代码：

./utils/tools.ts

import * as Battery from "expo-battery";
import * as Brightness from "expo-brightness";
const getBatteryLevel = async () => {
  const batteryLevel = await Battery.getBatteryLevelAsync();
  console.log("batteryLevel", batteryLevel);
  if (batteryLevel === -1) {
    return "Error: Device does not support retrieving the battery level.";
  }
  return batteryLevel;
};
const changeBrightness = ({ brightness }: { brightness: number }) => {
  console.log("changeBrightness", brightness);
  Brightness.setSystemBrightnessAsync(brightness);
  return brightness;
};
const flashScreen = () => {
  Brightness.setSystemBrightnessAsync(1);
  setTimeout(() => {
    Brightness.setSystemBrightnessAsync(0);
  }, 200);
  return "Successfully flashed the screen.";
};
export { getBatteryLevel, changeBrightness, flashScreen };

动态变量

除了客户端工具外，我们还会将平台（web、iOS、Android）作为动态变量注入首条消息和提示词：

./App.tsx

// ...
const startConversation = async () => {
  if (isStarting) return;
  setIsStarting(true);
  try {
    await conversation.startSession({
      agentId: process.env.EXPO_PUBLIC_AGENT_ID,
      dynamicVariables: {
        platform: Platform.OS,
      },
    });
  } catch (error) {
    console.error("Failed to start conversation:", error);
  } finally {
    setIsStarting(false);
  }
};
// ...

智能体配置

登录 ElevenLabs

前往 elevenlabs.io 并登录账户。

创建新智能体

前往 Agents Platform > Agents，从空白模板创建新智能体。

设置首条消息

设置首条消息并指定平台的动态变量。

Hi there, woah, so cool that I'm running on {{platform}}. What can I help you with?

设置系统提示词

设置系统提示词。也可以在此处加入动态变量。

You are a helpful assistant running on {{platform}}. You have access to certain tools that allow you to check the user device battery level and change the display brightness. Use these tools if the user asks about them. Otherwise, just answer the question.

设置客户端工具

设置以下客户端工具：

名称：getBatteryLevel
- 描述：以带小数点的百分比获取设备电池电量。
- 等待响应：true
- 响应超时（秒）：3
名称：changeBrightness
- 描述：更改设备屏幕亮度。
- 等待响应：true
- 响应超时（秒）：3
- 参数：
  - 数据类型：number
  - 标识符：brightness
  - 必填：true
  - 值类型：LLM Prompt
  - 描述：介于 0 和 1 之间（含）的数字，表示所需屏幕亮度。
名称：flashScreen
- 描述：快速开关屏幕闪烁。
- 等待响应：true
- 响应超时（秒）：3

运行应用

此应用需要 Expo Go 不支持的部分原生依赖项，因此需要先预构建应用，再在原生设备上运行。

终端 1：
- 运行 npx expo prebuild --clean

npx expo prebuild --clean

运行 npx expo start --tunnel，通过 https 启动 Expo 开发服务器。

npx expo start --tunnel

终端 2：
- 运行 npx expo run:ios --device，在 iOS 设备上运行应用。

npx expo run:ios --device

后续步骤

ElevenAgents

使用完整的 ElevenAgents 平台构建并部署语音智能体。

ElevenAgents 集成概览

探索所有集成选项，包括 Web 小组件、原生 SDK 和电话系统。