Kris
00a5577aac
本次提交包含了大量跨模块的功能增强、bug修复与代码优化： 1. 清理冗余空行与导入格式，统一代码风格 2. 新增集成超时异常类，替换原生TimeoutError避免穿透核心层 3. 扩展审计日志DTO与用例，新增变更字段追踪能力 4. 完善各类仓储接口，新增批量操作、时间范围过滤支持 5. 重构健康检查返回格式，统一使用status字段替代冗余的reachable/healthy 6. 扩展仪表盘与各类服务端口，新增待办统计、系统dashboard等能力 7. 优化批量操作与事务处理，新增保存点支持隔离失败操作 8. 修复配额管理阈值计算逻辑，支持自定义告警阈值 9. 统一认证类型错误提示，优化插件注册校验逻辑 10. 扩展SOAP适配器预览能力，兼容bytes类型输入 11. 新增批量恢复、批量硬删除回收站资源的接口与DTO 12. 优化测试用例断言逻辑，兼容前端操作符别名与字段名差异
402 lines
15 KiB
Python
402 lines
15 KiB
Python
"""Slack discover / preview_tools / create_tools handler 实现。
|
||
|
||
handler 签名遵循 ``OperationHandler`` 协议：
|
||
``(source_type: str, system_config: dict[str, Any]) -> Awaitable[Any]``。
|
||
|
||
handler 内部职责：
|
||
1. 从 ``system_config`` 提取 ``base_url`` / ``auth_config`` / ``_resolved_token``
|
||
2. 构建 ``auth_headers``（token 由 use_cases 层注入到 ``system_config["_resolved_token"]``）
|
||
3. 调用 Slack Web API 聚合元数据（conversations.list / users.list / team.info）
|
||
4. 调用 ``SlackConfigGenerator`` 生成工具配置草稿
|
||
|
||
429 退避分层（见设计文档 §6.2）：
|
||
- discover/preview/create 阶段：本模块 ``_request_with_retry`` 实现（解析 Retry-After）
|
||
- 运行时工具执行：通过工具 ``retry_policy.retry_status_codes: [429, 502, 503, 504]`` 配置
|
||
|
||
Slack 错误响应特殊性（见设计文档 §8.1）：
|
||
- HTTP 状态码成功与失败均可能返回 ``200 OK``
|
||
- 响应体必含 ``ok`` 布尔字段；失败时 ``ok=false`` 且包含 ``error`` 字符串字段
|
||
- ``_request_with_retry`` 在 HTTP 2xx 时额外检查 ``ok`` 字段，``ok=false`` 通过
|
||
``map_slack_error`` 转换为 ``ExternalSystemError`` 子类
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import asyncio
|
||
import time
|
||
from typing import Any
|
||
|
||
from yuxi.external_systems.exceptions import (
|
||
AuthError,
|
||
ExecutionError,
|
||
RateLimitExceededError,
|
||
)
|
||
from yuxi.external_systems.integrations.schemas import GeneratedToolsDraft
|
||
from yuxi.external_systems.integrations.slack.constants import (
|
||
_RETRY_BASE_DELAY,
|
||
_RETRY_MAX_ATTEMPTS,
|
||
_SLACK_HEADERS,
|
||
BASE_URL,
|
||
CONVERSATION_TYPES,
|
||
PAGE_SIZE,
|
||
)
|
||
from yuxi.external_systems.integrations.slack.error_extractor import (
|
||
map_slack_error,
|
||
)
|
||
from yuxi.external_systems.integrations.slack.generators import (
|
||
SlackConfigGenerator,
|
||
)
|
||
|
||
# ── system_config 解析辅助 ────────────────────────────────────────────────
|
||
|
||
|
||
def _build_auth_headers(system_config: dict[str, Any]) -> dict[str, str]:
|
||
"""从 system_config 提取 token，构建 Slack 请求头。
|
||
|
||
system_config 中 token 由 use_cases 层通过 ``token_manager.get_token`` 预处理后
|
||
注入到 ``system_config["_resolved_token"]["access_token"]``。handler 不自行换取
|
||
token——token 获取是 use_cases 层的职责。
|
||
"""
|
||
token_info = system_config.get("_resolved_token")
|
||
if not isinstance(token_info, dict) or not token_info.get("access_token"):
|
||
raise AuthError("slack: system_config 缺少 _resolved_token，use_cases 层未注入 token")
|
||
access_token = token_info["access_token"]
|
||
token_type = token_info.get("token_type", "Bearer")
|
||
headers = dict(_SLACK_HEADERS)
|
||
headers["Authorization"] = f"{token_type} {access_token}"
|
||
return headers
|
||
|
||
|
||
def _resolve_base_url(system_config: dict[str, Any]) -> str:
|
||
"""从 system_config 提取 base_url，缺省返回 ``https://slack.com/api``。"""
|
||
connection_config = system_config.get("connection_config") or {}
|
||
base_url = connection_config.get("base_url") or BASE_URL
|
||
return str(base_url).rstrip("/")
|
||
|
||
|
||
def _resolve_selected_tools(system_config: dict[str, Any]) -> list[str] | None:
|
||
"""从 system_config 提取用户选择的工具 slug 列表。"""
|
||
discovery_options = system_config.get("discovery_options") or {}
|
||
selected = discovery_options.get("selected_tools")
|
||
return list(selected) if selected else None
|
||
|
||
|
||
# ── 429 退避 HTTP 请求 ────────────────────────────────────────────────────
|
||
|
||
|
||
async def _request_with_retry(
|
||
base_url: str,
|
||
method: str,
|
||
headers: dict[str, str],
|
||
*,
|
||
method_verb: str = "GET",
|
||
params: dict[str, Any] | None = None,
|
||
json_body: dict[str, Any] | None = None,
|
||
) -> dict[str, Any]:
|
||
"""带 429 退避的 HTTP 请求（discover/preview/create 阶段使用）。
|
||
|
||
解析 ``Retry-After`` 头部（秒数），按指数退避重试，最多 ``_RETRY_MAX_ATTEMPTS`` 次。
|
||
Slack 错误响应特殊性：HTTP 2xx 时 ``ok=false`` 也视为错误，通过
|
||
``map_slack_error`` 转换为 ``ExternalSystemError`` 子类抛出。
|
||
网络层异常（超时 / 连接错误）转换为 ``ExecutionError``。
|
||
|
||
Args:
|
||
base_url: Slack Web API 基础 URL（如 ``https://slack.com/api``）。
|
||
method: Web API 方法名（如 ``conversations.list``）。
|
||
headers: 请求头（含 Authorization）。
|
||
method_verb: HTTP 方法（``GET`` / ``POST``）。
|
||
params: 查询参数（GET）。
|
||
json_body: JSON body（POST 写操作）。
|
||
|
||
Returns:
|
||
解析后的 JSON 响应 dict（``ok=true`` 的成功响应）。
|
||
"""
|
||
import httpx
|
||
|
||
url = f"{base_url}/{method}"
|
||
last_error: Exception | None = None
|
||
|
||
for attempt in range(_RETRY_MAX_ATTEMPTS):
|
||
try:
|
||
async with httpx.AsyncClient() as client:
|
||
if method_verb == "GET":
|
||
resp = await client.get(url, headers=headers, params=params)
|
||
else:
|
||
# POST 写操作使用 JSON body，必须设置 Content-Type: application/json
|
||
post_headers = dict(headers)
|
||
post_headers["Content-Type"] = "application/json"
|
||
resp = await client.request(method_verb, url, headers=post_headers, json=json_body)
|
||
|
||
# 429 限流：优先使用 Retry-After 头退避
|
||
if resp.status_code == 429:
|
||
if attempt == _RETRY_MAX_ATTEMPTS - 1:
|
||
retry_after_header = resp.headers.get("Retry-After", "60")
|
||
try:
|
||
retry_after = int(retry_after_header)
|
||
except ValueError:
|
||
retry_after = 60
|
||
raise RateLimitExceededError(
|
||
f"slack: 限流，已重试 {_RETRY_MAX_ATTEMPTS} 次仍失败: {url}",
|
||
retry_after=retry_after,
|
||
limit_type="qps",
|
||
)
|
||
retry_after_header = resp.headers.get("Retry-After")
|
||
if retry_after_header:
|
||
try:
|
||
delay = float(retry_after_header)
|
||
except ValueError:
|
||
delay = _RETRY_BASE_DELAY * (2**attempt)
|
||
else:
|
||
delay = _RETRY_BASE_DELAY * (2**attempt)
|
||
await asyncio.sleep(delay)
|
||
continue
|
||
|
||
# 非 2xx：解析错误 body 并映射为 ExternalSystemError 子类
|
||
if resp.status_code >= 400:
|
||
try:
|
||
error_body = resp.json()
|
||
except ValueError:
|
||
error_body = {"error": f"http_{resp.status_code}"}
|
||
raise map_slack_error(resp.status_code, error_body)
|
||
|
||
# 2xx：解析 JSON 响应
|
||
try:
|
||
body = resp.json()
|
||
except ValueError as exc:
|
||
raise ExecutionError(f"slack: 响应 JSON 解析失败: {url}: {exc}") from exc
|
||
|
||
# Slack 特殊性：HTTP 200 也可能 ok=false
|
||
if isinstance(body, dict) and body.get("ok") is False:
|
||
raise map_slack_error(resp.status_code, body)
|
||
|
||
return body
|
||
|
||
except httpx.HTTPError as exc:
|
||
last_error = exc
|
||
if attempt == _RETRY_MAX_ATTEMPTS - 1:
|
||
break
|
||
await asyncio.sleep(_RETRY_BASE_DELAY * (2**attempt))
|
||
continue
|
||
|
||
raise ExecutionError(f"slack: 请求 {url} 失败，已重试 {_RETRY_MAX_ATTEMPTS} 次: {last_error}")
|
||
|
||
|
||
# ── cursor 分页自动迭代 ──────────────────────────────────────────────────
|
||
|
||
|
||
async def _paginate(
|
||
base_url: str,
|
||
method: str,
|
||
headers: dict[str, str],
|
||
params: dict[str, Any],
|
||
) -> list[dict[str, Any]]:
|
||
"""cursor 分页自动迭代，聚合所有页结果（GET 请求）。
|
||
|
||
Slack 列表接口（``conversations.list`` / ``users.list`` /
|
||
``conversations.history`` / ``conversations.replies``）均使用 cursor 分页，
|
||
响应中的 ``response_metadata.next_cursor`` 为空字符串时表示结束。
|
||
|
||
Args:
|
||
method: Web API 方法名（如 ``conversations.list``）。
|
||
params: 初始查询参数（不含 ``cursor``）。
|
||
|
||
Returns:
|
||
聚合后的资源列表（``channels`` / ``members`` / ``messages``）。
|
||
"""
|
||
aggregated: list[dict[str, Any]] = []
|
||
cursor: str | None = None
|
||
while True:
|
||
query = {**params}
|
||
if cursor:
|
||
query["cursor"] = cursor
|
||
resp = await _request_with_retry(
|
||
base_url,
|
||
method,
|
||
headers,
|
||
method_verb="GET",
|
||
params=query,
|
||
)
|
||
# 响应体中资源列表的 key 因方法而异
|
||
for key in ("channels", "members", "messages"):
|
||
items = resp.get(key)
|
||
if isinstance(items, list):
|
||
aggregated.extend(items)
|
||
break
|
||
cursor = (resp.get("response_metadata") or {}).get("next_cursor", "")
|
||
if not cursor:
|
||
break
|
||
return aggregated
|
||
|
||
|
||
# ── 三个 OperationHandler 实现 ────────────────────────────────────────────
|
||
|
||
|
||
async def discover_slack(
|
||
source_type: str,
|
||
system_config: dict[str, Any],
|
||
) -> list[dict[str, Any]]:
|
||
"""discover handler：发现 Slack 工作区可用资源（频道 + 用户 + 工作区信息）。
|
||
|
||
通过 ``conversations.list`` + ``users.list`` + ``team.info`` 三个接口聚合返回
|
||
工作区元数据，供前端展示可用频道与用户。
|
||
|
||
Returns:
|
||
资源列表，每项形如：
|
||
- ``{"type": "channel", "id": "C1234567890", "name": "general", "is_private": false}``
|
||
- ``{"type": "user", "id": "U1234567890", "name": "alice", "email": "..."}``
|
||
- ``{"type": "team", "id": "T1234567890", "name": "Acme", "domain": "acme"}``
|
||
"""
|
||
base_url = _resolve_base_url(system_config)
|
||
headers = _build_auth_headers(system_config)
|
||
|
||
channels = await _paginate(
|
||
base_url,
|
||
"conversations.list",
|
||
headers,
|
||
params={"types": CONVERSATION_TYPES, "limit": PAGE_SIZE},
|
||
)
|
||
users = await _paginate(
|
||
base_url,
|
||
"users.list",
|
||
headers,
|
||
params={"limit": PAGE_SIZE},
|
||
)
|
||
team_resp = await _request_with_retry(
|
||
base_url,
|
||
"team.info",
|
||
headers,
|
||
method_verb="GET",
|
||
)
|
||
team = team_resp.get("team") or {}
|
||
|
||
result: list[dict[str, Any]] = [
|
||
*[
|
||
{
|
||
"type": "channel",
|
||
"id": c.get("id", ""),
|
||
"name": c.get("name", ""),
|
||
"is_private": c.get("is_private", False),
|
||
}
|
||
for c in channels
|
||
if isinstance(c, dict)
|
||
],
|
||
*[
|
||
{
|
||
"type": "user",
|
||
"id": u.get("id", ""),
|
||
"name": u.get("name", ""),
|
||
"email": (u.get("profile") or {}).get("email") if isinstance(u.get("profile"), dict) else None,
|
||
}
|
||
for u in users
|
||
if isinstance(u, dict) and not u.get("deleted")
|
||
],
|
||
]
|
||
if team:
|
||
result.append(
|
||
{
|
||
"type": "team",
|
||
"id": team.get("id", ""),
|
||
"name": team.get("name", ""),
|
||
"domain": team.get("domain"),
|
||
}
|
||
)
|
||
return result
|
||
|
||
|
||
async def preview_slack_tools(
|
||
source_type: str,
|
||
system_config: dict[str, Any],
|
||
) -> list[dict[str, Any]]:
|
||
"""preview_tools handler：预览将生成的工具列表（不持久化）。
|
||
|
||
Returns:
|
||
工具预览列表，每项为 ``ExternalToolCreateInput`` 兼容的 dict。
|
||
"""
|
||
payload = _build_generator_payload(system_config)
|
||
generator = SlackConfigGenerator()
|
||
return await generator.generate(payload)
|
||
|
||
|
||
async def create_slack_tools(
|
||
source_type: str,
|
||
system_config: dict[str, Any],
|
||
) -> GeneratedToolsDraft:
|
||
"""create_tools handler：生成工具草稿（不直接持久化）。
|
||
|
||
Returns:
|
||
``GeneratedToolsDraft``，由 use_cases 层统一持久化。
|
||
"""
|
||
payload = _build_generator_payload(system_config)
|
||
generator = SlackConfigGenerator()
|
||
tool_configs = await generator.generate(payload)
|
||
return GeneratedToolsDraft(tool_configs=tool_configs, override_existing=False)
|
||
|
||
|
||
# ── 生成器 payload 构造 ───────────────────────────────────────────────────
|
||
|
||
|
||
def _build_generator_payload(system_config: dict[str, Any]) -> dict[str, Any]:
|
||
"""从 system_config 构造生成器 payload。
|
||
|
||
流程：
|
||
1. 解析 base_url / auth_type / auth_config
|
||
2. 解析用户选择的工具 slug 列表（可选）
|
||
3. 汇总为生成器 payload
|
||
"""
|
||
base_url = _resolve_base_url(system_config)
|
||
auth_config = system_config.get("auth_config") or {}
|
||
auth_type = system_config.get("auth_type", "slack_bot_token")
|
||
selected_tools = _resolve_selected_tools(system_config)
|
||
|
||
payload: dict[str, Any] = {
|
||
"base_url": base_url,
|
||
"auth_type": auth_type,
|
||
"auth_config": auth_config,
|
||
}
|
||
if selected_tools is not None:
|
||
payload["selected_tools"] = selected_tools
|
||
return payload
|
||
|
||
|
||
# ── 健康检查 ─────────────────────────────────────────────────────────────
|
||
|
||
|
||
async def health_check_slack(system_config: dict[str, Any]) -> dict[str, Any]:
|
||
"""Slack 健康检查：调用 ``POST /api/auth.test`` 验证 Token 有效性。
|
||
|
||
供 use_cases 层直接调用，不注册到 ``IntegrationOperationRegistry``。
|
||
|
||
Returns::
|
||
|
||
{
|
||
"status": "healthy" | "unhealthy",
|
||
"team_id": "T1234567890",
|
||
"user_id": "U1234567890",
|
||
"bot_id": "B1234567890",
|
||
"duration_ms": 123,
|
||
}
|
||
"""
|
||
start = time.monotonic()
|
||
try:
|
||
base_url = _resolve_base_url(system_config)
|
||
headers = _build_auth_headers(system_config)
|
||
resp = await _request_with_retry(
|
||
base_url,
|
||
"auth.test",
|
||
headers,
|
||
method_verb="POST",
|
||
)
|
||
return {
|
||
"status": "healthy",
|
||
"team_id": resp.get("team_id"),
|
||
"user_id": resp.get("user_id"),
|
||
"bot_id": resp.get("bot_id"),
|
||
"duration_ms": int((time.monotonic() - start) * 1000),
|
||
}
|
||
except Exception as exc:
|
||
return {
|
||
"status": "unhealthy",
|
||
"detail": str(exc),
|
||
"duration_ms": int((time.monotonic() - start) * 1000),
|
||
}
|