banban/talkingq-url/mysql/mysql.md

TalkingQ MySQL 查询手册

数据库概览

TalkingQ 使用 MySQL 8.0 作为数据库后端，存储设备配置、角色信息和会话数据。本文档提供了数据库结构和常用查询操作的参考。

连接信息

数据库名称: talkingq
用户名: talkingq
密码: 在 .env 或 .env.prod 文件中配置的 MYSQL_PASSWORD
主机: localhost (本地开发) 或 mysql (Docker环境)
端口: 3306

连接数据库

# 本地开发环境
mysql -u talkingq -p talkingq

# Docker环境中
sudo docker compose exec -it mysql mysql -u talkingq -p talkingq

表结构

device_configs - 设备配置表

存储设备的角色选择和语言偏好等配置。

字段名	类型	描述
id	INT	主键，自增
device_id	VARCHAR(64)	设备ID，唯一索引
selected_role_key	VARCHAR(64)	选中的角色键
preferred_language	VARCHAR(10)	首选语言代码
last_update_time	FLOAT	最后更新时间戳
created_at	DATETIME	创建时间
updated_at	DATETIME	更新时间

conversation_histories - 对话历史表

存储设备的对话历史记录和角色信息。

字段名	类型	描述
id	INT	主键，自增
device_id	VARCHAR(64)	设备ID，索引
role_key	VARCHAR(64)	角色键，索引
last_interaction_time	FLOAT	最后交互时间戳
created_at	DATETIME	创建时间
updated_at	DATETIME	更新时间

conversation_messages - 对话消息表

存储对话的具体消息内容。

字段名	类型	描述
id	INT	主键，自增
conversation_id	INT	关联的对话历史ID，外键
is_user	BOOLEAN	是否为用户消息
content	TEXT	消息内容
timestamp	FLOAT	消息时间戳
created_at	DATETIME	创建时间
updated_at	DATETIME	更新时间

roles - 角色表

存储系统中可用的角色定义。

字段名	类型	描述
id	INT	主键，自增
role_key	VARCHAR(64)	角色键名，唯一索引
name	VARCHAR(128)	角色名称
description	TEXT	角色描述
content	TEXT	系统提示内容
default_language	VARCHAR(10)	默认语言代码
asr_provider	VARCHAR(64)	ASR服务提供商
llm_provider	VARCHAR(64)	LLM服务提供商
tts_provider	VARCHAR(64)	TTS服务提供商
competitive_llm_mode	BOOLEAN	是否启用竞争模式
volcano_model_id	VARCHAR(64)	Volcano模型ID
volcano_voice_type	VARCHAR(64)	Volcano音色类型
tencent_voice_type	VARCHAR(64)	腾讯音色类型
aliyun_voice_name	VARCHAR(64)	阿里云音色名称
url	VARCHAR(255)	资源URL路径
homophones	JSON	同音词列表
enabled	BOOLEAN	是否启用
created_at	DATETIME	创建时间
updated_at	DATETIME	更新时间

role_languages - 角色语言表

存储角色的多语言配置。

字段名	类型	描述
id	INT	主键，自增
role_id	INT	关联的角色ID，外键
language_code	VARCHAR(10)	语言代码
name	VARCHAR(128)	该语言下的角色名称
content	TEXT	该语言下的系统提示内容
asr_provider	VARCHAR(64)	该语言的ASR服务提供商
llm_provider	VARCHAR(64)	该语言的LLM服务提供商
tts_provider	VARCHAR(64)	该语言的TTS服务提供商
volcano_voice_type	VARCHAR(64)	该语言的Volcano音色类型
tencent_voice_type	VARCHAR(64)	该语言的腾讯音色类型
aliyun_voice_name	VARCHAR(64)	该语言的阿里云音色名称
url	VARCHAR(255)	该语言的资源URL路径
created_at	DATETIME	创建时间
updated_at	DATETIME	更新时间

device_auth - 设备认证表

存储设备认证信息，用于验证设备身份。

字段名	类型	描述
id	INT	主键，自增
device_id	VARCHAR(64)	设备ID，唯一索引
serial_number	VARCHAR(64)	设备序列号
is_active	BOOLEAN	是否活跃
created_at	DATETIME	创建时间
updated_at	DATETIME	更新时间

常用查询示例

设备配置查询

查询所有设备配置

SELECT * FROM device_configs;

查询特定设备的配置

SELECT * FROM device_configs WHERE device_id = '设备ID';

查询使用特定角色的设备

SELECT * FROM device_configs WHERE selected_role_key = '角色键名';

查询使用特定语言的设备

SELECT * FROM device_configs WHERE preferred_language = '语言代码';

查询最近更新的设备配置

SELECT * FROM device_configs
ORDER BY last_update_time DESC
LIMIT 10;

对话历史查询

查询特定设备的所有对话历史

SELECT * FROM conversation_histories WHERE device_id = '设备ID';

查询特定设备和角色的对话历史

SELECT * FROM conversation_histories 
WHERE device_id = '设备ID' AND role_key = '角色键名';

查询最近交互的对话历史

SELECT * FROM conversation_histories 
ORDER BY last_interaction_time DESC 
LIMIT 10;

查询一段时间内的对话历史

SELECT * FROM conversation_histories 
WHERE last_interaction_time BETWEEN UNIX_TIMESTAMP('2023-01-01') AND UNIX_TIMESTAMP('2023-12-31')
ORDER BY last_interaction_time DESC;

对话消息查询

查询特定对话的所有消息

SELECT * FROM conversation_messages 
WHERE conversation_id = 对话ID
ORDER BY timestamp ASC;

查询特定对话的用户消息

SELECT * FROM conversation_messages 
WHERE conversation_id = 对话ID AND is_user = TRUE
ORDER BY timestamp ASC;

查询特定对话的助手回复

SELECT * FROM conversation_messages 
WHERE conversation_id = 对话ID AND is_user = FALSE
ORDER BY timestamp ASC;

查询包含特定关键词的消息

SELECT * FROM conversation_messages 
WHERE content LIKE '%关键词%'
ORDER BY timestamp DESC;

查询最近的消息记录

SELECT cm.*, ch.device_id, ch.role_key 
FROM conversation_messages cm
JOIN conversation_histories ch ON cm.conversation_id = ch.id
ORDER BY cm.timestamp DESC
LIMIT 20;

角色表查询

查询所有启用的角色

SELECT * FROM roles WHERE enabled = TRUE;

查询特定角色的详细信息

SELECT * FROM roles WHERE role_key = '角色键名';

查询使用特定服务提供商的角色

SELECT * FROM roles WHERE llm_provider = 'Volcano';
-- 或
SELECT * FROM roles WHERE tts_provider = 'Tencent';
-- 或
SELECT * FROM roles WHERE asr_provider = 'Aliyun';

查询角色的同音词列表

SELECT role_key, homophones FROM roles WHERE homophones IS NOT NULL;

查询启用竞争模式的角色

SELECT * FROM roles WHERE competitive_llm_mode = TRUE;

角色语言表查询

查询特定角色的所有语言配置

SELECT rl.* FROM role_languages rl
JOIN roles r ON rl.role_id = r.id
WHERE r.role_key = '角色键名';

查询特定语言的所有角色配置

SELECT r.role_key, r.name, rl.* FROM role_languages rl
JOIN roles r ON rl.role_id = r.id
WHERE rl.language_code = '语言代码';

查询特定角色特定语言的配置

SELECT rl.* FROM role_languages rl
JOIN roles r ON rl.role_id = r.id
WHERE r.role_key = '角色键名' AND rl.language_code = '语言代码';

查询支持多语言的角色列表

SELECT r.role_key, r.name, COUNT(rl.id) as language_count
FROM roles r
JOIN role_languages rl ON r.id = rl.role_id
GROUP BY r.id
HAVING language_count > 1
ORDER BY language_count DESC;

设备认证查询

查询所有已认证设备

SELECT * FROM device_auth WHERE is_active = TRUE;

查询特定设备的认证信息

SELECT * FROM device_auth WHERE device_id = '设备ID';

查询最近添加的设备认证

SELECT * FROM device_auth
ORDER BY created_at DESC
LIMIT 10;

多表联合查询

查询特定设备的所有消息及对应的角色

SELECT cm.*, ch.role_key 
FROM conversation_messages cm 
JOIN conversation_histories ch ON cm.conversation_id = ch.id 
WHERE ch.device_id = '设备ID'
ORDER BY cm.timestamp ASC;

查询特定设备使用特定角色的所有对话内容

SELECT cm.*, ch.role_key, ch.last_interaction_time 
FROM conversation_messages cm 
JOIN conversation_histories ch ON cm.conversation_id = ch.id 
WHERE ch.device_id = '设备ID' AND ch.role_key = '角色键名'
ORDER BY cm.timestamp ASC;

查询所有设备的最新一条消息

SELECT ch.device_id, ch.role_key, cm.content, cm.timestamp 
FROM conversation_histories ch 
JOIN conversation_messages cm ON ch.id = cm.conversation_id 
WHERE cm.id = (
    SELECT MAX(cm2.id) 
    FROM conversation_messages cm2
    WHERE cm2.conversation_id = ch.id
);

查询设备当前使用的角色详细信息

SELECT dc.device_id, r.* 
FROM device_configs dc
JOIN roles r ON dc.selected_role_key = r.role_key
WHERE dc.device_id = '设备ID';

查询设备对话中使用的所有角色

SELECT DISTINCT ch.device_id, ch.role_key, r.name, r.description
FROM conversation_histories ch
JOIN roles r ON ch.role_key = r.role_key
WHERE ch.device_id = '设备ID';

查询设备认证状态与当前配置信息

SELECT da.device_id, da.serial_number, da.is_active, 
       dc.selected_role_key, dc.preferred_language, dc.last_update_time
FROM device_auth da
LEFT JOIN device_configs dc ON da.device_id = dc.device_id
ORDER BY da.created_at DESC;

高级统计查询

统计每个角色的使用情况

SELECT selected_role_key, COUNT(*) as device_count 
FROM device_configs 
GROUP BY selected_role_key
ORDER BY device_count DESC;

统计每种语言的使用情况

SELECT preferred_language, COUNT(*) as language_count 
FROM device_configs 
WHERE preferred_language IS NOT NULL
GROUP BY preferred_language
ORDER BY language_count DESC;

统计每个设备的对话数量

SELECT device_id, COUNT(*) as conversation_count 
FROM conversation_histories 
GROUP BY device_id
ORDER BY conversation_count DESC;

统计每个角色的对话数量

SELECT role_key, COUNT(*) as conversation_count 
FROM conversation_histories 
GROUP BY role_key
ORDER BY conversation_count DESC;

统计消息量最多的对话

SELECT ch.id, ch.device_id, ch.role_key, COUNT(cm.id) as message_count 
FROM conversation_histories ch 
JOIN conversation_messages cm ON ch.id = cm.conversation_id 
GROUP BY ch.id, ch.device_id, ch.role_key
ORDER BY message_count DESC
LIMIT 10;

统计最活跃的设备

SELECT device_id, COUNT(DISTINCT role_key) as roles_used, 
       COUNT(*) as total_conversations,
       MAX(last_interaction_time) as last_active
FROM conversation_histories 
GROUP BY device_id
ORDER BY total_conversations DESC
LIMIT 10;

统计各语言下的角色数量

SELECT rl.language_code, COUNT(DISTINCT r.id) as role_count
FROM role_languages rl
JOIN roles r ON rl.role_id = r.id
GROUP BY rl.language_code
ORDER BY role_count DESC;

统计近期设备活跃度

SELECT 
    DATE(FROM_UNIXTIME(last_interaction_time)) as date,
    COUNT(DISTINCT device_id) as active_devices
FROM conversation_histories
WHERE last_interaction_time > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 30 DAY))
GROUP BY date
ORDER BY date DESC;

统计各TTS提供商的使用情况

SELECT 
    CASE 
        WHEN volcano_voice_type IS NOT NULL THEN 'Volcano'
        WHEN tencent_voice_type IS NOT NULL THEN 'Tencent'
        WHEN aliyun_voice_name IS NOT NULL THEN 'Aliyun'
        ELSE 'Unknown'
    END as tts_provider,
    COUNT(*) as role_count
FROM roles
GROUP BY tts_provider
ORDER BY role_count DESC;

统计设备认证与使用情况

SELECT 
    COUNT(da.id) as total_devices,
    SUM(CASE WHEN da.is_active = TRUE THEN 1 ELSE 0 END) as active_devices,
    SUM(CASE WHEN dc.id IS NOT NULL THEN 1 ELSE 0 END) as configured_devices,
    SUM(CASE WHEN ch.device_id IS NOT NULL THEN 1 ELSE 0 END) as devices_with_conversations
FROM device_auth da
LEFT JOIN device_configs dc ON da.device_id = dc.device_id
LEFT JOIN (SELECT DISTINCT device_id FROM conversation_histories) ch ON da.device_id = ch.device_id;

数据分析查询

对话内容分析

查询用户问题平均长度

SELECT ch.role_key, AVG(LENGTH(cm.content)) as avg_length 
FROM conversation_messages cm 
JOIN conversation_histories ch ON cm.conversation_id = ch.id 
WHERE cm.is_user = TRUE
GROUP BY ch.role_key;

查询助手回答平均长度

SELECT ch.role_key, AVG(LENGTH(cm.content)) as avg_length 
FROM conversation_messages cm 
JOIN conversation_histories ch ON cm.conversation_id = ch.id 
WHERE cm.is_user = FALSE
GROUP BY ch.role_key;

查询用户消息长度分布

SELECT 
    CASE 
        WHEN LENGTH(content) < 50 THEN '极短 (<50)'
        WHEN LENGTH(content) BETWEEN 50 AND 100 THEN '短 (50-100)'
        WHEN LENGTH(content) BETWEEN 101 AND 200 THEN '中 (101-200)'
        WHEN LENGTH(content) BETWEEN 201 AND 500 THEN '长 (201-500)'
        ELSE '极长 (>500)'
    END as message_length_category,
    COUNT(*) as message_count
FROM conversation_messages
WHERE is_user = TRUE
GROUP BY message_length_category
ORDER BY 
    CASE message_length_category
        WHEN '极短 (<50)' THEN 1
        WHEN '短 (50-100)' THEN 2
        WHEN '中 (101-200)' THEN 3
        WHEN '长 (201-500)' THEN 4
        ELSE 5
    END;

查询不同角色对话完成时间统计

SELECT ch.role_key, 
       COUNT(*) as conversation_count,
       AVG(MAX(cm.timestamp) - MIN(cm.timestamp)) as avg_duration
FROM conversation_histories ch
JOIN conversation_messages cm ON ch.id = cm.conversation_id
GROUP BY ch.role_key
HAVING conversation_count > 5
ORDER BY avg_duration DESC;

查询不同语言的平均对话长度

SELECT dc.preferred_language, 
       COUNT(DISTINCT ch.id) as conversation_count,
       AVG(cm_count.msg_count) as avg_messages_per_conversation
FROM device_configs dc
JOIN conversation_histories ch ON dc.device_id = ch.device_id
JOIN (
    SELECT conversation_id, COUNT(*) as msg_count
    FROM conversation_messages
    GROUP BY conversation_id
) cm_count ON ch.id = cm_count.conversation_id
WHERE dc.preferred_language IS NOT NULL
GROUP BY dc.preferred_language
HAVING conversation_count > 3
ORDER BY avg_messages_per_conversation DESC;

查询包含特定关键词的对话

SELECT ch.id, ch.device_id, ch.role_key, 
       MAX(ch.last_interaction_time) as conversation_time,
       GROUP_CONCAT(DISTINCT CASE WHEN cm.is_user = TRUE THEN cm.content ELSE NULL END SEPARATOR '\n') as user_messages,
       GROUP_CONCAT(DISTINCT CASE WHEN cm.is_user = FALSE THEN cm.content ELSE NULL END SEPARATOR '\n') as assistant_messages
FROM conversation_histories ch
JOIN conversation_messages cm ON ch.id = cm.conversation_id
WHERE cm.content LIKE '%关键词%'
GROUP BY ch.id, ch.device_id, ch.role_key
ORDER BY conversation_time DESC;

用户行为分析

查询用户平均会话时长趋势

SELECT 
    DATE(FROM_UNIXTIME(ch.last_interaction_time)) as date,
    AVG(
        MAX(cm.timestamp) - MIN(cm.timestamp)
    ) as avg_session_duration
FROM conversation_histories ch
JOIN conversation_messages cm ON ch.id = cm.conversation_id
WHERE ch.last_interaction_time > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 30 DAY))
GROUP BY date
ORDER BY date;

查询每日活跃用户数和对话量

SELECT 
    DATE(FROM_UNIXTIME(last_interaction_time)) as date,
    COUNT(DISTINCT device_id) as active_users,
    COUNT(*) as conversations
FROM conversation_histories
WHERE last_interaction_time > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 30 DAY))
GROUP BY date
ORDER BY date DESC;

查询设备的首次使用时间和最后使用时间

SELECT 
    device_id,
    MIN(created_at) as first_use,
    FROM_UNIXTIME(MAX(last_interaction_time)) as last_use,
    DATEDIFF(FROM_UNIXTIME(MAX(last_interaction_time)), MIN(created_at)) as days_between
FROM conversation_histories
GROUP BY device_id
ORDER BY days_between DESC;

查询用户连续活跃天数

WITH daily_activity AS (
    SELECT 
        device_id, 
        DATE(FROM_UNIXTIME(last_interaction_time)) as activity_date
    FROM conversation_histories
    GROUP BY device_id, activity_date
)
SELECT 
    device_id,
    COUNT(*) as active_days,
    MIN(activity_date) as first_active_day,
    MAX(activity_date) as last_active_day,
    DATEDIFF(MAX(activity_date), MIN(activity_date)) + 1 as day_span,
    COUNT(*) / (DATEDIFF(MAX(activity_date), MIN(activity_date)) + 1) as activity_ratio
FROM daily_activity
GROUP BY device_id
HAVING day_span > 7
ORDER BY active_days DESC, activity_ratio DESC;

管理操作

备份数据库

# 在Docker环境外执行
docker exec -it talkingq-mysql-1 mysqldump -u root -p talkingq > talkingq_backup.sql

# 或使用本地MySQL客户端
mysqldump -h localhost -u talkingq -p talkingq > talkingq_backup.sql

恢复数据库

# 在Docker环境外执行
cat talkingq_backup.sql | docker exec -i talkingq-mysql-1 mysql -u root -p talkingq

# 或使用本地MySQL客户端
mysql -h localhost -u talkingq -p talkingq < talkingq_backup.sql

备份特定表

# 仅备份设备配置表
docker exec -it talkingq-mysql-1 mysqldump -u root -p talkingq device_configs > device_configs_backup.sql

# 仅备份角色表和角色语言表
docker exec -it talkingq-mysql-1 mysqldump -u root -p talkingq roles role_languages > roles_backup.sql

# 仅备份对话历史和消息表
docker exec -it talkingq-mysql-1 mysqldump -u root -p talkingq conversation_histories conversation_messages > conversations_backup.sql

# 备份设备认证表
docker exec -it talkingq-mysql-1 mysqldump -u root -p talkingq device_auth > device_auth_backup.sql

删除过期记录

-- 删除长期未使用的设备配置
DELETE FROM device_configs 
WHERE last_update_time < UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 90 DAY));

-- 删除3个月前的对话历史
DELETE ch, cm
FROM conversation_histories ch
LEFT JOIN conversation_messages cm ON ch.id = cm.conversation_id
WHERE ch.last_interaction_time < UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 3 MONTH));

-- 将90天内未被任何设备使用的角色标记为禁用
UPDATE roles r
SET enabled = FALSE
WHERE r.role_key NOT IN (
    SELECT DISTINCT selected_role_key 
    FROM device_configs
    WHERE last_update_time > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 90 DAY))
) AND r.role_key NOT IN (
    SELECT DISTINCT role_key 
    FROM conversation_histories
    WHERE last_interaction_time > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 90 DAY))
);

-- 清理已停用设备的认证记录
UPDATE device_auth
SET is_active = FALSE
WHERE device_id NOT IN (
    SELECT DISTINCT device_id 
    FROM conversation_histories
    WHERE last_interaction_time > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 180 DAY))
);

角色管理

-- 添加新角色（基本信息）
INSERT INTO roles (role_key, name, description, content, default_language, enabled)
VALUES ('new_role', '新角色', '新角色的描述', '系统提示内容', 'zh', TRUE);

-- 为角色添加语言配置
INSERT INTO role_languages (role_id, language_code, name, content)
VALUES (
    (SELECT id FROM roles WHERE role_key = 'new_role'),
    'en',
    'New Role',
    'System prompt content in English'
);

-- 复制角色
INSERT INTO roles (
    role_key, name, description, content, default_language, 
    asr_provider, llm_provider, tts_provider, competitive_llm_mode,
    volcano_model_id, volcano_voice_type, tencent_voice_type, aliyun_voice_name,
    url, homophones, enabled
)
SELECT 
    CONCAT(role_key, '_copy'), CONCAT(name, ' (复制)'), description, content, default_language,
    asr_provider, llm_provider, tts_provider, competitive_llm_mode,
    volcano_model_id, volcano_voice_type, tencent_voice_type, aliyun_voice_name,
    url, homophones, enabled
FROM roles
WHERE role_key = '要复制的角色键名';

数据库优化

优化表

OPTIMIZE TABLE device_configs;
OPTIMIZE TABLE conversation_histories;
OPTIMIZE TABLE conversation_messages;
OPTIMIZE TABLE roles;
OPTIMIZE TABLE role_languages;
OPTIMIZE TABLE device_auth;

分析表

ANALYZE TABLE device_configs;
ANALYZE TABLE conversation_histories;
ANALYZE TABLE conversation_messages;
ANALYZE TABLE roles;
ANALYZE TABLE role_languages;
ANALYZE TABLE device_auth;

检查表

CHECK TABLE device_configs;
CHECK TABLE conversation_histories;
CHECK TABLE conversation_messages;
CHECK TABLE roles;
CHECK TABLE role_languages;
CHECK TABLE device_auth;

添加索引

-- 为对话历史表添加联合索引
CREATE INDEX idx_device_role ON conversation_histories(device_id, role_key);

-- 为角色语言表添加联合索引
CREATE INDEX idx_role_language ON role_languages(role_id, language_code);

-- 为消息表添加时间戳索引，优化时间范围查询
CREATE INDEX idx_message_timestamp ON conversation_messages(timestamp);

-- 为设备认证表添加索引
CREATE INDEX idx_device_auth_active ON device_auth(is_active);

监控查询性能

-- 查看慢查询日志状态
SHOW VARIABLES LIKE 'slow_query%';

-- 查看当前运行的查询
SHOW PROCESSLIST;

-- 终止执行时间过长的查询
KILL QUERY 线程ID;

数据库维护脚本示例

以下是一个可以添加到cron任务的脚本示例，用于定期清理过期数据和优化表：

#!/bin/bash
# 文件名: /home/ubuntu/TalkingQ_URL/scripts/db_maintenance.sh

# 数据库连接参数
DB_USER="talkingq"
DB_PASS="您的密码"
DB_NAME="talkingq"
DB_HOST="mysql"

# 日志文件
LOG_FILE="/home/ubuntu/TalkingQ_URL/logs/db_maintenance.log"

echo "===== $(date) 开始数据库维护 =====" >> $LOG_FILE

# 删除3个月前的对话历史
echo "清理旧对话历史..." >> $LOG_FILE
docker exec -i talkingq-mysql-1 mysql -u$DB_USER -p$DB_PASS $DB_NAME << EOF
DELETE ch, cm
FROM conversation_histories ch
LEFT JOIN conversation_messages cm ON ch.id = cm.conversation_id
WHERE ch.last_interaction_time < UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 3 MONTH));
EOF

# 优化表
echo "优化数据库表..." >> $LOG_FILE
docker exec -i talkingq-mysql-1 mysql -u$DB_USER -p$DB_PASS $DB_NAME << EOF
OPTIMIZE TABLE conversation_histories, conversation_messages, device_configs;
EOF

echo "===== $(date) 数据库维护完成 =====" >> $LOG_FILE