chore: 重构迁移脚本
This commit is contained in:
parent
56aabb67bd
commit
8443513dad
108
docs/latest/changelog/migrate_to_v0-5.md
Normal file
108
docs/latest/changelog/migrate_to_v0-5.md
Normal file
@ -0,0 +1,108 @@
|
||||
# v0.5 数据迁移指南
|
||||
|
||||
v0.5 版本将数据存储从 SQLite + JSON 文件迁移到 PostgreSQL。本指南帮助你完成数据迁移。
|
||||
|
||||
::: tip warning
|
||||
迁移脚本可能会存在问题,不建议在生产环境下尝试,生产环境下,请新建或仔细检查迁移脚本,慎重迁移。
|
||||
:::
|
||||
|
||||
## 迁移内容
|
||||
|
||||
| 数据类型 | 源 | 目标 | 存储内容 |
|
||||
|---------|-----|------|---------|
|
||||
| 业务数据 | SQLite (`saves/database/server.db`) | PostgreSQL | 用户、部门、对话、消息、工具调用、MCP 服务器等 |
|
||||
| 知识库元数据 | JSON 文件 (`saves/knowledge_base_data/`) | PostgreSQL | 知识库配置、文件信息、评估数据 |
|
||||
| Tasker 任务记录 | JSON 文件 (`saves/tasks/tasks.json`) | PostgreSQL | 后台任务状态、进度、结果(独立存储) |
|
||||
|
||||
## 迁移前准备
|
||||
|
||||
### 1. 启动服务
|
||||
|
||||
```bash
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
### 2. 备份数据
|
||||
|
||||
**重要:** 迁移前必须备份数据!
|
||||
|
||||
```bash
|
||||
# 备份 saves 目录(包含 SQLite 数据库和知识库元数据)
|
||||
cp -r saves saves_backup_$(date +%Y%m%d)
|
||||
|
||||
# 如果使用外部数据库,也请备份 PostgreSQL
|
||||
pg_dump -U postgres -d yuxi_know > pg_backup_$(date +%Y%m%d).sql
|
||||
```
|
||||
|
||||
### 3. 确保 PostgreSQL 已启动
|
||||
|
||||
```bash
|
||||
docker compose up -d postgres
|
||||
# 等待健康检查通过
|
||||
```
|
||||
|
||||
## 执行迁移
|
||||
|
||||
### 方式一:使用统一迁移脚本(推荐)
|
||||
|
||||
```bash
|
||||
# 1. 预览迁移(不执行)
|
||||
docker compose exec api python scripts/migrate_all.py --dry-run
|
||||
|
||||
# 2. 执行迁移
|
||||
docker compose exec api python scripts/migrate_all.py --execute
|
||||
|
||||
# 3. 验证迁移结果
|
||||
docker compose exec api python scripts/migrate_all.py --verify
|
||||
```
|
||||
|
||||
### 方式二:分阶段迁移
|
||||
|
||||
```bash
|
||||
# 只迁移业务数据
|
||||
docker compose exec api python scripts/migrate_all.py --execute --stage business
|
||||
|
||||
# 只迁移知识库元数据
|
||||
docker compose exec api python scripts/migrate_all.py --execute --stage knowledge
|
||||
|
||||
# 只迁移 Tasker 任务记录
|
||||
docker compose exec api python scripts/migrate_all.py --execute --stage tasker
|
||||
```
|
||||
|
||||
## 重启服务
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## 验证迁移
|
||||
|
||||
### 检查服务状态
|
||||
|
||||
```bash
|
||||
# 查看 API 服务日志
|
||||
docker logs api-dev --tail 50
|
||||
|
||||
# 检查健康状态
|
||||
curl http://localhost:5050/api/system/health
|
||||
```
|
||||
|
||||
### 验证数据
|
||||
|
||||
```bash
|
||||
# 使用迁移脚本验证
|
||||
docker compose exec api python scripts/migrate_all.py --verify
|
||||
```
|
||||
|
||||
预期输出:
|
||||
|
||||
```
|
||||
============================================================
|
||||
📊 验证结果汇总
|
||||
============================================================
|
||||
✅ departments: SQLite=X, PostgreSQL=X
|
||||
✅ users: SQLite=X, PostgreSQL=X
|
||||
✅ conversations: SQLite=X, PostgreSQL=X
|
||||
...
|
||||
全部匹配: ✅ 是
|
||||
```
|
||||
1471
scripts/migrate_all.py
Normal file
1471
scripts/migrate_all.py
Normal file
File diff suppressed because it is too large
Load Diff
@ -384,6 +384,8 @@ class MCPServer(Base):
|
||||
|
||||
def to_mcp_config(self) -> dict:
|
||||
"""转换为 MCP 配置格式(用于加载到 MCP_SERVERS 缓存)"""
|
||||
import json
|
||||
|
||||
config = {
|
||||
"transport": self.transport,
|
||||
}
|
||||
@ -391,10 +393,24 @@ class MCPServer(Base):
|
||||
config["url"] = self.url
|
||||
if self.command:
|
||||
config["command"] = self.command
|
||||
if self.args:
|
||||
config["args"] = self.args
|
||||
if self.headers:
|
||||
config["headers"] = self.headers
|
||||
# args 只用于 stdio 传输类型,必须是列表
|
||||
if self.transport == "stdio" and self.args:
|
||||
if isinstance(self.args, list):
|
||||
config["args"] = self.args
|
||||
elif isinstance(self.args, str):
|
||||
try:
|
||||
config["args"] = json.loads(self.args)
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
# headers 只用于 sse/streamable_http 传输类型
|
||||
if self.transport in ("sse", "streamable_http") and self.headers:
|
||||
if isinstance(self.headers, dict):
|
||||
config["headers"] = self.headers
|
||||
elif isinstance(self.headers, str):
|
||||
try:
|
||||
config["headers"] = json.loads(self.headers)
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
if self.timeout is not None:
|
||||
config["timeout"] = self.timeout
|
||||
if self.sse_read_timeout is not None:
|
||||
|
||||
@ -424,15 +424,31 @@ class MCPServer(Base):
|
||||
|
||||
def to_mcp_config(self) -> dict[str, Any]:
|
||||
"""转换为 MCP 配置格式(用于加载到 MCP_SERVERS 缓存)"""
|
||||
import json
|
||||
|
||||
config = {"transport": self.transport}
|
||||
if self.url:
|
||||
config["url"] = self.url
|
||||
if self.command:
|
||||
config["command"] = self.command
|
||||
if self.args:
|
||||
config["args"] = self.args
|
||||
if self.headers:
|
||||
config["headers"] = self.headers
|
||||
# args 只用于 stdio 传输类型,必须是列表
|
||||
if self.transport == "stdio" and self.args:
|
||||
if isinstance(self.args, list):
|
||||
config["args"] = self.args
|
||||
elif isinstance(self.args, str):
|
||||
try:
|
||||
config["args"] = json.loads(self.args)
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
# headers 只用于 sse/streamable_http 传输类型
|
||||
if self.transport in ("sse", "streamable_http") and self.headers:
|
||||
if isinstance(self.headers, dict):
|
||||
config["headers"] = self.headers
|
||||
elif isinstance(self.headers, str):
|
||||
try:
|
||||
config["headers"] = json.loads(self.headers)
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
if self.timeout is not None:
|
||||
config["timeout"] = self.timeout
|
||||
if self.sse_read_timeout is not None:
|
||||
|
||||
Loading…
Reference in New Issue
Block a user