Merge branch 'main' of https://github.com/xerrors/Yuxi-Know
This commit is contained in:
parent
13402235f6
commit
d9e39a7a7d
4
.gitignore
vendored
4
.gitignore
vendored
@ -33,6 +33,8 @@ src/data
|
||||
*/package-lock.json
|
||||
web/package-lock.json
|
||||
saves
|
||||
saves_dev
|
||||
notebooks
|
||||
graphrag
|
||||
docker/volumes
|
||||
docker/volumes
|
||||
.cursorrules
|
||||
|
||||
31
README.md
31
README.md
@ -12,9 +12,8 @@
|
||||
> [!NOTE]
|
||||
> 当前项目还处于开发的早期,还存在一些 BUG,有问题随时提 issue。
|
||||
|
||||
已知问题:
|
||||
|
||||
- [ ] 从 Flask 更换到 Fast API 之后,并行命令还存在问题。
|
||||
- [ ] Ollma Embedding 支持(Open-like Embedding 支持)
|
||||
- [x] DeepSeek-R1 支持,需配置 `DEEPSEEK_API_KEY` 或者 `SILICONFLOW_API_KEY` 使用
|
||||
|
||||
## 概述
|
||||
|
||||
@ -31,12 +30,12 @@ ZHIPUAI_API_KEY=270ea********8bfa97.e3XOMd****Q1Sk
|
||||
OPENAI_API_KEY=sk-*********[可选]
|
||||
```
|
||||
|
||||
本项目的基础对话服务可以在不含显卡的设备上运行,大模型使用在线服务商的接口。但是如果想要完整的知识库对话体验,则需要 8G 以上的显存。因为需要本地运行 embedding 模型和 rerank 模型。
|
||||
本项目的基础对话服务可以在不含显卡的设备上运行,大模型使用在线服务商的接口。但是如果想要完整的知识库对话体验,则需要 8G 以上的显存。因为需要本地运行 embedding 模型和 rerank 模型。如果需要指定本地模型所在路径,需要配置 `MODEL_DIR` 参数。
|
||||
|
||||
**提醒**:下面的脚本会启动开发版本,源代码的修改会自动更新(含前端和后端)。如果生产环境部署,请使用 `docker/docker-compose.yml` 启动。
|
||||
|
||||
```bash
|
||||
docker-compose -f docker/docker-compose.dev.yml up --build
|
||||
docker compose -f docker/docker-compose.dev.yml --env-file src/.env up --build
|
||||
```
|
||||
|
||||
**也可以加上 `-d` 参数,后台运行。*
|
||||
@ -63,7 +62,7 @@ docker-compose -f docker/docker-compose.dev.yml up --build
|
||||
关闭 docker 服务:
|
||||
|
||||
```bash
|
||||
docker-compose -f docker/docker-compose.dev.yml down
|
||||
docker compose -f docker/docker-compose.dev.yml --env-file src/.env down
|
||||
```
|
||||
|
||||
查看日志:
|
||||
@ -72,26 +71,10 @@ docker-compose -f docker/docker-compose.dev.yml down
|
||||
docker logs <CONTAINER_NAME> # 例如:docker logs api-dev
|
||||
```
|
||||
|
||||
如果需要使用到本地模型(不推荐手动指定),比如向量模型或者重排序模型,则需要将环境变量中设置的 `MODEL_ROOT_DIR` 做映射,比如本地模型都是存放在 `/hdd/models` 里面,则需要在 `docker-compose.yml` 和 `docker-compose.dev.yml` 中添加:
|
||||
|
||||
```yml
|
||||
services:
|
||||
api:
|
||||
build:
|
||||
context: ..
|
||||
dockerfile: docker/api.Dockerfile
|
||||
container_name: api-dev
|
||||
working_dir: /app
|
||||
volumes:
|
||||
- ../src:/app/src
|
||||
- ../saves:/app/saves
|
||||
- /hdd/zwj/models:/hdd/zwj/models # <== 修改这一行
|
||||
```
|
||||
|
||||
**生产环境部署**:本项目同时支持使用 Docker 部署生产环境,只需要更换 `docker-compose` 文件就可以了。
|
||||
|
||||
```bash
|
||||
docker-compose -f docker/docker-compose.yml up --build
|
||||
docker compose -f docker/docker-compose.yml --env-file src/.env up --build
|
||||
```
|
||||
|
||||
## 模型支持
|
||||
@ -131,7 +114,7 @@ docker-compose -f docker/docker-compose.yml up --build
|
||||
|
||||
对于**语言模型**,并不支持直接运行本地语言模型,请使用 vllm 或者 ollama 转成 API 服务之后使用。
|
||||
|
||||
对于**向量模型**和**重排序模型**,可以不做修改会自动下载模型,如果下载过程中出现问题,请参考 [HF-Mirror](https://hf-mirror.com/) 配置相关内容。如果想要使用本地已经下载好的模型(不建议),可以在 `saves/config/config.yaml` 配置相关内容。同时注意要在 docker 中做映射,参考 README 中的 `docker/docker-compose.yml`。
|
||||
对于**向量模型**和**重排序模型**,可以不做修改会自动下载模型,如果下载过程中出现问题,请参考 [HF-Mirror](https://hf-mirror.com/) 配置相关内容。如果想要使用本地已经下载好的模型(不建议),可以在网页的 settings 里面做映射。
|
||||
|
||||
例如:
|
||||
|
||||
|
||||
@ -10,7 +10,12 @@ COPY ../requirements.txt /app/requirements.txt
|
||||
# 安装依赖(Docker 会缓存这一步,除非 requirements.txt 发生变化)
|
||||
RUN pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
|
||||
|
||||
RUN pip install gunicorn
|
||||
RUN pip install -U gunicorn -i https://pypi.tuna.tsinghua.edu.cn/simple
|
||||
|
||||
RUN sed -i s@/archive.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list
|
||||
RUN sed -i s@/security.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list
|
||||
RUN apt-get clean
|
||||
RUN apt-get update && apt-get install ffmpeg libsm6 libxext6 -y
|
||||
|
||||
# 复制代码到容器中
|
||||
COPY ../src /app/src
|
||||
|
||||
@ -7,8 +7,9 @@ services:
|
||||
working_dir: /app
|
||||
volumes:
|
||||
- ../src:/app/src
|
||||
- ../saves:/app/saves
|
||||
- /hdd/zwj/models:/hdd/zwj/models
|
||||
- ../saves_dev:/app/saves
|
||||
- ../src/static/config.dev.yaml:/app/src/static/config.yaml
|
||||
- ${MODEL_DIR}:${MODEL_DIR}
|
||||
ports:
|
||||
- "5000:5000"
|
||||
depends_on:
|
||||
@ -21,6 +22,7 @@ services:
|
||||
- NEO4J_USERNAME=neo4j
|
||||
- NEO4J_PASSWORD=0123456789
|
||||
- MILVUS_URI=http://milvus:19530
|
||||
- MODEL_DIR=${MODEL_DIR} # 优先级高于 .env 中的 MODEL_DIR
|
||||
command: uvicorn src.main:app --host 0.0.0.0 --port 5000 --reload
|
||||
|
||||
web:
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
dashscope>=1.20.3
|
||||
PyMuPDF==1.23.26
|
||||
peft>=0.11.1
|
||||
FlagEmbedding>=1.2.11
|
||||
FlagEmbedding>=1.3.2
|
||||
Flask>=3.0.3
|
||||
Flask_Cors>=4.0.1
|
||||
llama_index>=0.11.8
|
||||
@ -24,4 +24,5 @@ opencv-python-headless
|
||||
docx2txt
|
||||
uvicorn[standard]
|
||||
fastapi
|
||||
python-multipart
|
||||
python-multipart
|
||||
tavily-python
|
||||
@ -1,25 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# 检查是否提供了 API_KEY 参数
|
||||
if [ -z "$1" ]; then
|
||||
echo "请提供 API_KEY。"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 获取当前目录路径
|
||||
CURRENT_DIR=$(pwd)
|
||||
|
||||
# 如果 src 目录不存在则创建
|
||||
if [! -d "${CURRENT_DIR}/src" ]; then
|
||||
mkdir -p "${CURRENT_DIR}/src"
|
||||
fi
|
||||
|
||||
# 如果.env 文件不存在,则从.env.template 复制一份创建
|
||||
if [! -f "${CURRENT_DIR}/src/.env" ]; then
|
||||
cp "${CURRENT_DIR}/src/.env.template" "${CURRENT_DIR}/src/.env"
|
||||
fi
|
||||
|
||||
# 将 API_KEY 写入.env 文件
|
||||
echo "ZHIPUAI_API_KEY=$1" > "${CURRENT_DIR}/src/.env"
|
||||
|
||||
echo "API_KEY 已成功写入 src/.env 文件。"
|
||||
@ -6,7 +6,7 @@ from src.utils.logging_config import setup_logger
|
||||
|
||||
logger = setup_logger("Config")
|
||||
|
||||
with open(Path("src/config/models.yaml"), "r") as f:
|
||||
with open(Path("src/static/models.yaml"), "r") as f:
|
||||
_models = yaml.safe_load(f)
|
||||
|
||||
MODEL_NAMES = _models["MODEL_NAMES"]
|
||||
@ -17,7 +17,7 @@ RERANKER_LIST = _models["RERANKER_LIST"]
|
||||
class SimpleConfig(dict):
|
||||
|
||||
def __key(self, key):
|
||||
return "" if key is None else key.lower()
|
||||
return "" if key is None else key.lower() # 目前忘记了这里为什么要 lower 了,只能说配置项最好不要有大写的
|
||||
|
||||
def __str__(self):
|
||||
return json.dumps(self)
|
||||
@ -40,32 +40,33 @@ class SimpleConfig(dict):
|
||||
|
||||
class Config(SimpleConfig):
|
||||
|
||||
def __init__(self, filename=None):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._config_items = {}
|
||||
self.save_dir = "saves"
|
||||
self.filename = str(Path("src/static/config.yaml"))
|
||||
os.makedirs(os.path.dirname(self.filename), exist_ok=True)
|
||||
|
||||
### >>> 默认配置
|
||||
# 可以在 config/base.yaml 中覆盖
|
||||
self.add_item("stream", default=True, des="是否开启流式输出")
|
||||
self.add_item("save_dir", default="saves", des="保存目录")
|
||||
# 功能选项
|
||||
self.add_item("enable_reranker", default=False, des="是否开启重排序")
|
||||
self.add_item("enable_knowledge_base", default=False, des="是否开启知识库")
|
||||
self.add_item("enable_knowledge_graph", default=False, des="是否开启知识图谱")
|
||||
self.add_item("enable_search_engine", default=False, des="是否开启搜索引擎")
|
||||
|
||||
self.add_item("enable_web_search", default=False, des="是否开启网页搜索")
|
||||
# 模型配置
|
||||
## 注意这里是模型名,而不是具体的模型路径,默认使用 HuggingFace 的路径
|
||||
## 如果需要自定义路径,则在 config/base.yaml 中配置 model_local_paths
|
||||
## 如果需要自定义本地模型路径,则在 src/.env 中配置 MODEL_DIR
|
||||
self.add_item("model_provider", default="zhipu", des="模型提供商", choices=list(MODEL_NAMES.keys()))
|
||||
self.add_item("model_name", default=None, des="模型名称")
|
||||
self.add_item("embed_model", default="zhipu-embedding-3", des="Embedding 模型", choices=list(EMBED_MODEL_INFO.keys()))
|
||||
self.add_item("reranker", default="bge-reranker-v2-m3", des="Re-Ranker 模型", choices=list(RERANKER_LIST.keys()))
|
||||
self.add_item("model_local_paths", default={}, des="本地模型路径")
|
||||
self.add_item("use_rewrite_query", default="off", des="重写查询", choices=["off", "on", "hyde"])
|
||||
### <<< 默认配置结束
|
||||
|
||||
self.filename = filename or os.path.join(self.save_dir, "config", "config.yaml")
|
||||
os.makedirs(os.path.dirname(self.filename), exist_ok=True)
|
||||
|
||||
self.load()
|
||||
self.handle_self()
|
||||
@ -89,7 +90,9 @@ class Config(SimpleConfig):
|
||||
def handle_self(self):
|
||||
self.model_names = MODEL_NAMES
|
||||
model_provider_info = self.model_names.get(self.model_provider, {})
|
||||
self.model_dir = os.environ.get("MODEL_DIR", "")
|
||||
|
||||
# 检查模型提供商是否存在
|
||||
if self.model_provider != "custom":
|
||||
if self.model_name not in model_provider_info["models"]:
|
||||
logger.warning(f"Model name {self.model_name} not in {self.model_provider}, using default model name")
|
||||
@ -103,6 +106,7 @@ class Config(SimpleConfig):
|
||||
logger.warning(f"Model name {self.model_name} not in custom models, using default model name")
|
||||
self.model_name = self.custom_models[0]["custom_id"]
|
||||
|
||||
# 检查模型提供商的环境变量
|
||||
conds = {}
|
||||
self.model_provider_status = {}
|
||||
for provider in self.model_names:
|
||||
@ -110,10 +114,14 @@ class Config(SimpleConfig):
|
||||
conds_bool = [bool(os.getenv(_k)) for _k in conds[provider]]
|
||||
self.model_provider_status[provider] = all(conds_bool)
|
||||
|
||||
# 检查web_search的环境变量
|
||||
if self.enable_web_search and not os.getenv("TAVILY_API_KEY"):
|
||||
logger.warning("TAVILY_API_KEY not set, web search will be disabled")
|
||||
self.enable_web_search = False
|
||||
|
||||
self.valuable_model_provider = [k for k, v in self.model_provider_status.items() if v]
|
||||
assert len(self.valuable_model_provider) > 0, f"No model provider available, please check your `.env` file. API_KEY_LIST: {conds}"
|
||||
|
||||
|
||||
|
||||
def load(self):
|
||||
"""根据传入的文件覆盖掉默认配置"""
|
||||
|
||||
@ -14,6 +14,10 @@ class Retriever:
|
||||
if self.config.enable_reranker:
|
||||
self.reranker = Reranker(config)
|
||||
|
||||
if self.config.enable_web_search:
|
||||
from src.utils.web_search import WebSearcher
|
||||
self.web_searcher = WebSearcher()
|
||||
|
||||
def retrieval(self, query, history, meta):
|
||||
|
||||
refs = {"query": query, "history": history, "meta": meta}
|
||||
@ -21,6 +25,7 @@ class Retriever:
|
||||
refs["entities"] = self.reco_entities(query, history, refs)
|
||||
refs["knowledge_base"] = self.query_knowledgebase(query, history, refs)
|
||||
refs["graph_base"] = self.query_graph(query, history, refs)
|
||||
refs["web_search"] = self.query_web(query, history, refs)
|
||||
|
||||
return refs
|
||||
|
||||
@ -44,6 +49,12 @@ class Retriever:
|
||||
)
|
||||
external_parts.extend(["图数据库信息:", db_text])
|
||||
|
||||
# 解析网络搜索的结果
|
||||
web_res = refs.get("web_search", {}).get("results", [])
|
||||
if web_res:
|
||||
web_text = "\n".join(f"{r['title']}: {r['snippet']}" for r in web_res)
|
||||
external_parts.extend(["网络搜索信息:", web_text])
|
||||
|
||||
# 构造查询
|
||||
from src.utils.prompts import knowbase_qa_template
|
||||
if external_parts and len(external_parts) > 0:
|
||||
@ -60,8 +71,6 @@ class Retriever:
|
||||
raise NotImplementedError
|
||||
|
||||
def query_graph(self, query, history, refs):
|
||||
# res = model.predict("qiansdgsa, dasdh ashdsakjdk ak ").content
|
||||
|
||||
results = []
|
||||
if refs["meta"].get("use_graph") and self.config.enable_knowledge_base:
|
||||
for entity in refs["entities"]:
|
||||
@ -70,6 +79,7 @@ class Retriever:
|
||||
results.extend(result)
|
||||
return {"results": self.format_query_results(results)}
|
||||
|
||||
|
||||
def query_knowledgebase(self, query, history, refs):
|
||||
"""查询知识库"""
|
||||
|
||||
@ -116,9 +126,27 @@ class Retriever:
|
||||
|
||||
return {"results": kb_res, "all_results": all_kb_res, "rw_query": rw_query}
|
||||
|
||||
def query_web(self, query, history, refs):
|
||||
"""查询网络"""
|
||||
|
||||
if not (refs["meta"].get("enable_web_search") and self.config.enable_web_search):
|
||||
return {"results": [], "message": "Web search is disabled"}
|
||||
|
||||
try:
|
||||
search_results = self.web_searcher.search(query)
|
||||
except Exception as e:
|
||||
logger.error(f"Web search error: {str(e)}")
|
||||
return {"results": [], "message": "Web search error"}
|
||||
|
||||
return {"results": search_results}
|
||||
|
||||
def rewrite_query(self, query, history, refs):
|
||||
"""重写查询"""
|
||||
rewrite_query_span = refs["meta"].get("rewriteQuery", "off")
|
||||
if refs["meta"].get("mode") == "search": # 如果是搜索模式,就使用 meta 的配置,否则就使用全局的配置
|
||||
rewrite_query_span = refs["meta"].get("use_rewrite_query", "off")
|
||||
else:
|
||||
rewrite_query_span = self.config.use_rewrite_query
|
||||
|
||||
if rewrite_query_span == "off":
|
||||
rewritten_query = query
|
||||
else:
|
||||
|
||||
@ -1,3 +1,4 @@
|
||||
import os
|
||||
from src.core import DataBaseManager
|
||||
from src.core.retriever import Retriever
|
||||
from src.models import select_model
|
||||
@ -17,6 +18,10 @@ class Startup:
|
||||
self.dbm = DataBaseManager(self.config)
|
||||
self.retriever = Retriever(self.config, self.dbm, self.model)
|
||||
|
||||
self.model_lite = select_model(self.config,
|
||||
model_provider=self.config.model_provider_lite or "zhipu",
|
||||
model_name=self.config.model_name_lite or "glm-4-flash")
|
||||
|
||||
def restart(self):
|
||||
logger.info("Restarting...")
|
||||
self.start()
|
||||
|
||||
@ -1,10 +1,10 @@
|
||||
from src.utils.logging_config import logger
|
||||
|
||||
|
||||
def select_model(config):
|
||||
def select_model(config, model_provider=None, model_name=None):
|
||||
|
||||
model_provider = config.model_provider
|
||||
model_name = config.model_name
|
||||
model_provider = model_provider or config.model_provider
|
||||
model_name = model_name or config.model_name
|
||||
|
||||
logger.info(f"Selecting model from {model_provider} with {model_name}")
|
||||
|
||||
|
||||
@ -50,8 +50,8 @@ class OpenModel(OpenAIBase):
|
||||
class DeepSeek(OpenAIBase):
|
||||
def __init__(self, model_name=None):
|
||||
model_name = model_name or "deepseek-chat"
|
||||
api_key = os.getenv("DEEPSEEK_API_KEY")
|
||||
base_url = "https://api.deepseek.com"
|
||||
api_key = os.getenv("DEEPSEEK_API_KEY", "your-default-api-key")
|
||||
base_url = os.getenv("DEEPSEEK_API_BASE", "https://api.deepseek.com/v1")
|
||||
super().__init__(api_key=api_key, base_url=base_url, model_name=model_name)
|
||||
|
||||
|
||||
|
||||
@ -1,5 +1,4 @@
|
||||
import os
|
||||
import uuid
|
||||
from FlagEmbedding import FlagModel, FlagReranker
|
||||
|
||||
from src.config import EMBED_MODEL_INFO, RERANKER_LIST
|
||||
@ -15,11 +14,7 @@ GLOBAL_EMBED_STATE = {}
|
||||
class EmbeddingModel(FlagModel):
|
||||
def __init__(self, model_info, config, **kwargs):
|
||||
self.info = model_info
|
||||
model_name_or_path = handle_local_model(
|
||||
paths=config.model_local_paths,
|
||||
model_name=model_info["name"],
|
||||
default_path=model_info.get("default_path", None))
|
||||
|
||||
model_name_or_path = config.model_local_paths.get(model_info["name"], model_info.get("default_path"))
|
||||
logger.info(f"Loading embedding model {model_info['name']} from {model_name_or_path}")
|
||||
|
||||
super().__init__(model_name_or_path,
|
||||
@ -34,11 +29,8 @@ class Reranker(FlagReranker):
|
||||
|
||||
assert config.reranker in RERANKER_LIST.keys(), f"Unsupported Reranker: {config.reranker}, only support {RERANKER_LIST.keys()}"
|
||||
|
||||
model_name_or_path = handle_local_model(
|
||||
paths=config.model_local_paths,
|
||||
model_name=config.reranker,
|
||||
default_path=RERANKER_LIST[config.reranker])
|
||||
|
||||
model_info = RERANKER_LIST[config.reranker]
|
||||
model_name_or_path = config.model_local_paths.get(model_info["name"], model_info.get("default_path"))
|
||||
logger.info(f"Loading Reranker model {config.reranker} from {model_name_or_path}")
|
||||
|
||||
super().__init__(model_name_or_path, use_fp16=True, **kwargs)
|
||||
@ -113,6 +105,4 @@ def get_embedding_model(config):
|
||||
|
||||
def handle_local_model(paths, model_name, default_path):
|
||||
model_path = paths.get(model_name, default_path)
|
||||
if os.getenv("MODEL_ROOT_DIR") and not os.path.isabs(model_path):
|
||||
model_path = os.path.join(os.getenv("MODEL_ROOT_DIR"), model_path)
|
||||
return model_path
|
||||
179
src/models/ollama_embedding.py
Normal file
179
src/models/ollama_embedding.py
Normal file
@ -0,0 +1,179 @@
|
||||
import os
|
||||
import requests
|
||||
import numpy as np
|
||||
from typing import List, Union, Dict
|
||||
from src.utils.logging_config import setup_logger
|
||||
|
||||
logger = setup_logger("OllamaEmbedding")
|
||||
|
||||
class OllamaEmbedding:
|
||||
"""
|
||||
使用 Ollama API 进行文本嵌入的类
|
||||
"""
|
||||
def __init__(self, model_info: Dict, config) -> None:
|
||||
"""
|
||||
初始化 Ollama Embedding 模型
|
||||
|
||||
Args:
|
||||
model_info: 模型信息字典
|
||||
config: 配置对象
|
||||
"""
|
||||
self.config = config
|
||||
self.model_info = model_info
|
||||
self.base_url = os.getenv("OLLAMA_BASE_URL", "http://localhost:11434")
|
||||
self.model_name = model_info.get("name", "nomic-embed-text")
|
||||
self.query_instruction_for_retrieval = "为这个句子生成表示以用于检索相关文章:"
|
||||
logger.info(f"Ollama Embedding model {self.model_name} initialized")
|
||||
|
||||
def _get_embedding(self, text: str) -> List[float]:
|
||||
"""
|
||||
获取单个文本的嵌入向量
|
||||
|
||||
Args:
|
||||
text: 输入文本
|
||||
|
||||
Returns:
|
||||
嵌入向量
|
||||
"""
|
||||
url = f"{self.base_url}/api/embeddings"
|
||||
try:
|
||||
response = requests.post(url, json={
|
||||
"model": self.model_name,
|
||||
"prompt": text
|
||||
})
|
||||
response.raise_for_status()
|
||||
return response.json()["embedding"]
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting embedding: {str(e)}")
|
||||
raise
|
||||
|
||||
def predict(self, messages: List[str]) -> List[List[float]]:
|
||||
"""
|
||||
批量获取文本嵌入向量
|
||||
|
||||
Args:
|
||||
messages: 文本列表
|
||||
|
||||
Returns:
|
||||
嵌入向量列表
|
||||
"""
|
||||
embeddings = []
|
||||
batch_size = 20
|
||||
|
||||
for i in range(0, len(messages), batch_size):
|
||||
batch = messages[i:i + batch_size]
|
||||
logger.info(f"Processing batch {i//batch_size + 1}, size: {len(batch)}")
|
||||
|
||||
batch_embeddings = []
|
||||
for text in batch:
|
||||
embedding = self._get_embedding(text)
|
||||
batch_embeddings.append(embedding)
|
||||
|
||||
embeddings.extend(batch_embeddings)
|
||||
|
||||
return embeddings
|
||||
|
||||
def encode(self, messages: Union[str, List[str]]) -> List[List[float]]:
|
||||
"""
|
||||
编码文本
|
||||
|
||||
Args:
|
||||
messages: 单个文本或文本列表
|
||||
|
||||
Returns:
|
||||
嵌入向量列表
|
||||
"""
|
||||
if isinstance(messages, str):
|
||||
messages = [messages]
|
||||
return self.predict(messages)
|
||||
|
||||
def encode_queries(self, queries: List[str]) -> List[List[float]]:
|
||||
"""
|
||||
编码查询文本
|
||||
|
||||
Args:
|
||||
queries: 查询文本列表
|
||||
|
||||
Returns:
|
||||
查询文本的嵌入向量列表
|
||||
"""
|
||||
return self.predict(queries)
|
||||
|
||||
|
||||
class OllamaReranker:
|
||||
"""
|
||||
使用 Ollama API 进行文本重排序的类
|
||||
"""
|
||||
def __init__(self, config) -> None:
|
||||
"""
|
||||
初始化 Ollama Reranker
|
||||
|
||||
Args:
|
||||
config: 配置对象
|
||||
"""
|
||||
self.config = config
|
||||
self.base_url = os.getenv("OLLAMA_BASE_URL", "http://localhost:11434")
|
||||
self.model_name = config.reranker
|
||||
logger.info(f"Ollama Reranker model {self.model_name} initialized")
|
||||
|
||||
def compute_score(self, query: str, passage: str) -> float:
|
||||
"""
|
||||
计算查询和文本段落之间的相关性分数
|
||||
|
||||
Args:
|
||||
query: 查询文本
|
||||
passage: 段落文本
|
||||
|
||||
Returns:
|
||||
相关性分数
|
||||
"""
|
||||
prompt = f"Query: {query}\nPassage: {passage}\nRate the relevance of the passage to the query on a scale of 0 to 1:"
|
||||
|
||||
try:
|
||||
response = requests.post(
|
||||
f"{self.base_url}/api/generate",
|
||||
json={
|
||||
"model": self.model_name,
|
||||
"prompt": prompt,
|
||||
"stream": False
|
||||
}
|
||||
)
|
||||
response.raise_for_status()
|
||||
|
||||
# 提取生成的数字作为分数
|
||||
result = response.json()["response"].strip()
|
||||
try:
|
||||
score = float(result)
|
||||
return min(max(score, 0), 1) # 确保分数在 0-1 之间
|
||||
except ValueError:
|
||||
logger.warning(f"Could not parse score from response: {result}")
|
||||
return 0.0
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error computing rerank score: {str(e)}")
|
||||
return 0.0
|
||||
|
||||
def rerank(self, query: str, passages: List[str], top_n: int = None) -> List[Dict]:
|
||||
"""
|
||||
重新排序文本段落
|
||||
|
||||
Args:
|
||||
query: 查询文本
|
||||
passages: 段落文本列表
|
||||
top_n: 返回前 n 个结果
|
||||
|
||||
Returns:
|
||||
排序后的结果列表,每个元素包含索引和分数
|
||||
"""
|
||||
scores = []
|
||||
for i, passage in enumerate(passages):
|
||||
score = self.compute_score(query, passage)
|
||||
scores.append({"index": i, "score": score})
|
||||
|
||||
# 按分数降序排序
|
||||
sorted_results = sorted(scores, key=lambda x: x["score"], reverse=True)
|
||||
|
||||
if top_n:
|
||||
sorted_results = sorted_results[:top_n]
|
||||
|
||||
return sorted_results
|
||||
@ -16,8 +16,6 @@ logger = setup_logger("OneKE")
|
||||
|
||||
dotenv.load_dotenv()
|
||||
|
||||
MODEL_NAME_OR_PATH = os.path.join(os.getenv('MODEL_ROOT_DIR', './'), 'OneKE')
|
||||
|
||||
instruction_mapper = {
|
||||
'NERzh': "你是专门进行实体抽取的专家。请从input中抽取出符合schema定义的实体,不存在的实体类型返回空列表。请按照JSON字符串的格式回答。",
|
||||
'REzh': "你是专门进行关系抽取的专家。请从input中抽取出符合schema定义的关系三元组。请按照JSON字符串的格式回答。",
|
||||
@ -42,7 +40,7 @@ class OneKE:
|
||||
def __init__(self, config=None):
|
||||
|
||||
self.config = config
|
||||
model_name_or_path = config.model_local_paths.get('oneke', "zjunlp/OneKE")
|
||||
model_name_or_path = config.model_local_paths.get('zjunlp/OneKE', "zjunlp/OneKE")
|
||||
logger.info(f"Loading KGC model OneKE from {model_name_or_path}")
|
||||
|
||||
model_config = AutoConfig.from_pretrained(model_name_or_path, trust_remote_code=True)
|
||||
|
||||
@ -27,9 +27,10 @@ def chat_post(
|
||||
|
||||
history_manager = HistoryManager(history)
|
||||
|
||||
def make_chunk(content, status, history):
|
||||
def make_chunk(content=None, status=None, history=None, reasoning_content=None):
|
||||
return json.dumps({
|
||||
"response": content,
|
||||
"reasoning_response": reasoning_content,
|
||||
"history": history,
|
||||
"model_name": startup.config.model_name,
|
||||
"status": status,
|
||||
@ -37,33 +38,44 @@ def chat_post(
|
||||
}, ensure_ascii=False).encode('utf-8') + b"\n"
|
||||
|
||||
def generate_response():
|
||||
modified_query = query
|
||||
|
||||
if meta.get("enable_retrieval"):
|
||||
chunk = make_chunk("", "searching", history=None)
|
||||
# 处理知识库检索
|
||||
if meta and meta.get("enable_retrieval"):
|
||||
chunk = make_chunk(status="searching")
|
||||
yield chunk
|
||||
|
||||
new_query, refs = startup.retriever(query, history_manager.messages, meta)
|
||||
modified_query, refs = startup.retriever(modified_query, history_manager.messages, meta)
|
||||
refs_pool[cur_res_id] = refs
|
||||
else:
|
||||
new_query = query
|
||||
|
||||
messages = history_manager.get_history_with_msg(new_query, max_rounds=meta.get('history_round'))
|
||||
history_manager.add_user(query)
|
||||
messages = history_manager.get_history_with_msg(modified_query, max_rounds=meta.get('history_round'))
|
||||
history_manager.add_user(query) # 注意这里使用原始查询
|
||||
logger.debug(f"Web history: {history_manager.messages}")
|
||||
|
||||
content = ""
|
||||
reasoning_content = ""
|
||||
for delta in startup.model.predict(messages, stream=True):
|
||||
if not delta.content:
|
||||
if not delta.content and hasattr(delta, 'reasoning_content'):
|
||||
reasoning_content += delta.reasoning_content
|
||||
chunk = make_chunk(reasoning_content=reasoning_content, status="reasoning")
|
||||
yield chunk
|
||||
continue
|
||||
|
||||
# 文心一言
|
||||
if hasattr(delta, 'is_full') and delta.is_full:
|
||||
content = delta.content
|
||||
else:
|
||||
content += delta.content
|
||||
content += delta.content or ""
|
||||
|
||||
chunk = make_chunk(content, "loading", history=history_manager.update_ai(content))
|
||||
chunk = make_chunk(content=content,
|
||||
reasoning_content=reasoning_content,
|
||||
status="loading",
|
||||
history=history_manager.update_ai(content))
|
||||
yield chunk
|
||||
|
||||
logger.debug(f"Final response: {content}")
|
||||
logger.debug(f"Final reasoning response: {reasoning_content}")
|
||||
|
||||
return StreamingResponse(generate_response(), media_type='application/json')
|
||||
|
||||
@chat.post("/call")
|
||||
@ -77,6 +89,17 @@ async def call(query: str = Body(...), meta: dict = Body(None)):
|
||||
|
||||
return {"response": response.content}
|
||||
|
||||
@chat.post("/call_lite")
|
||||
async def call(query: str = Body(...), meta: dict = Body(None)):
|
||||
async def predict_async(query):
|
||||
loop = asyncio.get_event_loop()
|
||||
return await loop.run_in_executor(executor, startup.model_lite.predict, query)
|
||||
|
||||
response = await predict_async(query)
|
||||
logger.debug({"query": query, "response": response.content})
|
||||
|
||||
return {"response": response.content}
|
||||
|
||||
@chat.get("/refs")
|
||||
def get_refs(cur_res_id: str):
|
||||
global refs_pool
|
||||
|
||||
0
src/static/config.dev.yaml
Normal file
0
src/static/config.dev.yaml
Normal file
0
src/static/config.yaml
Normal file
0
src/static/config.yaml
Normal file
@ -18,6 +18,7 @@ MODEL_NAMES:
|
||||
- DEEPSEEK_API_KEY
|
||||
models:
|
||||
- deepseek-chat
|
||||
- deepseek-reasoner
|
||||
zhipu:
|
||||
name: 智谱AI (Zhipu)
|
||||
url: https://open.bigmodel.cn/dev/api
|
||||
@ -74,13 +75,13 @@ MODEL_NAMES:
|
||||
- meta-llama/Meta-Llama-3.1-8B-Instruct
|
||||
- meta-llama/Meta-Llama-3.1-70B-Instruct
|
||||
- meta-llama/Meta-Llama-3.1-405B-Instruct
|
||||
- deepseek-ai/DeepSeek-R1
|
||||
|
||||
EMBED_MODEL_INFO:
|
||||
bge-large-zh-v1.5:
|
||||
name: bge-large-zh-v1.5
|
||||
default_path: BAAI/bge-large-zh-v1.5
|
||||
bge-m3:
|
||||
name: BAAI/bge-m3
|
||||
default_path: BAAI/bge-m3
|
||||
dimension: 1024
|
||||
query_instruction: "为这个句子生成表示以用于检索相关文章:"
|
||||
zhipu-embedding-2:
|
||||
name: zhipu-embedding-2
|
||||
default_path: embedding-2
|
||||
@ -91,4 +92,6 @@ EMBED_MODEL_INFO:
|
||||
dimension: 2048
|
||||
|
||||
RERANKER_LIST:
|
||||
bge-reranker-v2-m3: BAAI/bge-reranker-v2-m3
|
||||
bge-reranker-v2-m3:
|
||||
name: BAAI/bge-reranker-v2-m3
|
||||
default_path: BAAI/bge-reranker-v2-m3
|
||||
69
src/utils/web_search.py
Normal file
69
src/utils/web_search.py
Normal file
@ -0,0 +1,69 @@
|
||||
import os
|
||||
from typing import List, Dict
|
||||
from tavily import TavilyClient
|
||||
from src.utils.logging_config import setup_logger
|
||||
|
||||
logger = setup_logger("web-search")
|
||||
|
||||
class WebSearcher:
|
||||
def __init__(self):
|
||||
api_key = os.getenv("TAVILY_API_KEY")
|
||||
if not api_key:
|
||||
raise ValueError("TAVILY_API_KEY environment variable is not set")
|
||||
self.client = TavilyClient(api_key)
|
||||
logger.info("WebSearcher initialized with Tavily client")
|
||||
|
||||
def search(self, query: str, max_results: int = 1) -> List[Dict]:
|
||||
"""
|
||||
使用 Tavily 搜索相关内容
|
||||
|
||||
Args:
|
||||
query: 搜索查询
|
||||
max_results: 最大返回结果数
|
||||
|
||||
Returns:
|
||||
搜索结果列表
|
||||
"""
|
||||
try:
|
||||
search_results = self.client.search(
|
||||
query=query,
|
||||
search_depth="basic",
|
||||
max_results=max_results
|
||||
)
|
||||
|
||||
# 提取需要的信息
|
||||
formatted_results = []
|
||||
for result in search_results['results'][:max_results]:
|
||||
formatted_results.append({
|
||||
'title': result.get('title', ''),
|
||||
'content': result.get('content', ''),
|
||||
'url': result.get('url', ''),
|
||||
'score': result.get('score', 0)
|
||||
})
|
||||
|
||||
return formatted_results
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error during web search: {str(e)}")
|
||||
return []
|
||||
|
||||
def format_search_results(self, results: List[Dict]) -> str:
|
||||
"""
|
||||
将搜索结果格式化为文本
|
||||
|
||||
Args:
|
||||
results: 搜索结果列表
|
||||
|
||||
Returns:
|
||||
格式化后的文本
|
||||
"""
|
||||
if not results:
|
||||
return "没有找到相关的网络搜索结果。"
|
||||
|
||||
formatted_text = "以下是相关的网络搜索结果:\n\n"
|
||||
for i, result in enumerate(results, 1):
|
||||
formatted_text += f"{i}. {result['title']}\n"
|
||||
formatted_text += f" {result['content']}\n"
|
||||
formatted_text += f" 来源: {result['url']}\n\n"
|
||||
|
||||
return formatted_text
|
||||
121
test/test_concurrency.py
Normal file
121
test/test_concurrency.py
Normal file
@ -0,0 +1,121 @@
|
||||
import asyncio
|
||||
import aiohttp
|
||||
import time
|
||||
import json
|
||||
from typing import List
|
||||
|
||||
async def make_request(session: aiohttp.ClientSession, request_id: int) -> dict:
|
||||
"""发送单个请求到API"""
|
||||
url = "http://localhost:5000/chat/call"
|
||||
payload = {
|
||||
"query": "写一个冒泡排序",
|
||||
"meta": {}
|
||||
}
|
||||
|
||||
start_time = time.time()
|
||||
print(f"请求 {request_id} 开始时间: {time.strftime('%H:%M:%S', time.localtime(start_time))}")
|
||||
try:
|
||||
async with session.post(url, json=payload) as response:
|
||||
result = await response.json()
|
||||
end_time = time.time()
|
||||
duration = end_time - start_time
|
||||
print(f"请求 {request_id} 完成时间: {time.strftime('%H:%M:%S', time.localtime(end_time))} (耗时: {duration:.2f}秒)")
|
||||
return {
|
||||
"request_id": request_id,
|
||||
"status": response.status,
|
||||
"time": duration,
|
||||
"start_time": start_time,
|
||||
"end_time": end_time,
|
||||
"success": True
|
||||
}
|
||||
except Exception as e:
|
||||
end_time = time.time()
|
||||
duration = end_time - start_time
|
||||
print(f"请求 {request_id} 失败时间: {time.strftime('%H:%M:%S', time.localtime(end_time))} (耗时: {duration:.2f}秒)")
|
||||
return {
|
||||
"request_id": request_id,
|
||||
"status": None,
|
||||
"time": duration,
|
||||
"start_time": start_time,
|
||||
"end_time": end_time,
|
||||
"success": False,
|
||||
"error": str(e)
|
||||
}
|
||||
|
||||
async def run_concurrent_test(num_requests: int = 10) -> List[dict]:
|
||||
"""运行并发测试"""
|
||||
async with aiohttp.ClientSession() as session:
|
||||
tasks = [make_request(session, i) for i in range(num_requests)]
|
||||
return await asyncio.gather(*tasks)
|
||||
|
||||
def analyze_results(results: List[dict]) -> None:
|
||||
"""分析并打印测试结果"""
|
||||
total_requests = len(results)
|
||||
successful_requests = sum(1 for r in results if r["success"])
|
||||
failed_requests = total_requests - successful_requests
|
||||
|
||||
response_times = [r["time"] for r in results if r["success"]]
|
||||
if response_times:
|
||||
avg_time = sum(response_times) / len(response_times)
|
||||
max_time = max(response_times)
|
||||
min_time = min(response_times)
|
||||
else:
|
||||
avg_time = max_time = min_time = 0
|
||||
|
||||
print(f"\n=== 并发测试结果 ===")
|
||||
print(f"总请求数: {total_requests}")
|
||||
print(f"成功请求: {successful_requests}")
|
||||
print(f"失败请求: {failed_requests}")
|
||||
print(f"平均响应时间: {avg_time:.2f} 秒")
|
||||
print(f"最长响应时间: {max_time:.2f} 秒")
|
||||
print(f"最短响应时间: {min_time:.2f} 秒")
|
||||
|
||||
if failed_requests > 0:
|
||||
print("\n失败的请求:")
|
||||
for result in results:
|
||||
if not result["success"]:
|
||||
print(f"请求 ID {result['request_id']}: {result.get('error', '未知错误')}")
|
||||
|
||||
# 添加请求时间线分析
|
||||
print("\n=== 请求时间线分析 ===")
|
||||
sorted_results = sorted(results, key=lambda x: x["start_time"])
|
||||
test_start_time = sorted_results[0]["start_time"]
|
||||
|
||||
print("\n时间线详情:")
|
||||
print("请求ID 开始时间 结束时间 耗时(秒) 重叠请求数")
|
||||
active_requests = []
|
||||
|
||||
for result in sorted_results:
|
||||
# 计算当前时间点的活跃请求数
|
||||
start_time = result["start_time"]
|
||||
end_time = result["end_time"]
|
||||
|
||||
# 清理已完成的请求
|
||||
active_requests = [t for t in active_requests if t > start_time]
|
||||
active_requests.append(end_time)
|
||||
|
||||
print(f"{result['request_id']:^7} {time.strftime('%H:%M:%S', time.localtime(start_time))} "
|
||||
f"{time.strftime('%H:%M:%S', time.localtime(end_time))} "
|
||||
f"{result['time']:^8.2f} {len(active_requests):^6}")
|
||||
|
||||
# 计算最大并发数
|
||||
max_concurrent = 0
|
||||
timeline = []
|
||||
for r in results:
|
||||
timeline.append((r["start_time"], 1))
|
||||
timeline.append((r["end_time"], -1))
|
||||
|
||||
timeline.sort(key=lambda x: x[0])
|
||||
current_concurrent = 0
|
||||
for _, change in timeline:
|
||||
current_concurrent += change
|
||||
max_concurrent = max(max_concurrent, current_concurrent)
|
||||
|
||||
print(f"\n最大并发请求数: {max_concurrent}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
NUM_REQUESTS = 100 # 设置并发请求数
|
||||
|
||||
print(f"开始运行 {NUM_REQUESTS} 个并发请求的测试...")
|
||||
results = asyncio.run(run_concurrent_test(NUM_REQUESTS))
|
||||
analyze_results(results)
|
||||
@ -81,9 +81,12 @@
|
||||
<div class="flex-center" @click="meta.use_web = !meta.use_web" v-if="configStore.config.enable_search_engine && meta.enable_retrieval">
|
||||
搜索引擎(Bing) <div @click.stop><a-switch v-model:checked="meta.use_web" /></div>
|
||||
</div>
|
||||
<div class="flex-center" v-if="configStore.config.enable_knowledge_base && meta.enable_retrieval">
|
||||
重写查询 <a-segmented v-model:value="meta.rewriteQuery" :options="['off', 'on', 'hyde']"/>
|
||||
<div class="flex-center" @click="meta.enable_web_search = !meta.enable_web_search" v-if="configStore.config.enable_search_engine && meta.enable_retrieval">
|
||||
网页搜索 <div @click.stop><a-switch v-model:checked="meta.enable_web_search" /></div>
|
||||
</div>
|
||||
<!-- <div class="flex-center" v-if="configStore.config.enable_knowledge_base && meta.enable_retrieval">
|
||||
重写查询 <a-segmented v-model:value="meta.use_rewrite_query" :options="['off', 'on', 'hyde']"/>
|
||||
</div> -->
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@ -114,6 +117,7 @@
|
||||
<div></div>
|
||||
</div>
|
||||
<div v-else-if="message.status == 'searching' && isStreaming" class="searching-msg"><i>正在检索……</i></div>
|
||||
<div v-else-if="message.status == 'reasoning' && isStreaming" class="searching-msg"><i>正在思考…… {{ message.reasoning }}</i></div>
|
||||
<div
|
||||
v-else-if="message.text.length == 0 || message.status == 'error' || (message.status != 'finished' && !isStreaming)"
|
||||
class="err-msg"
|
||||
@ -191,11 +195,7 @@ const panel = ref(null)
|
||||
const modelCard = ref(null)
|
||||
const examples = ref([
|
||||
'写一个冒泡排序',
|
||||
'肉碱的分子量是多少?直接回答',
|
||||
'总结大蒜的功效是什么?',
|
||||
'今天天气怎么样?',
|
||||
'吃饭吃出苍蝇可以索赔吗?',
|
||||
'帮我写一个请假条',
|
||||
'贾宝玉今年多少岁?',
|
||||
])
|
||||
|
||||
@ -210,11 +210,12 @@ const meta = reactive(JSON.parse(localStorage.getItem('meta')) || {
|
||||
enable_retrieval: false,
|
||||
use_graph: false,
|
||||
use_web: false,
|
||||
enable_web_search: false,
|
||||
graph_name: "neo4j",
|
||||
rewriteQuery: "off",
|
||||
// use_rewrite_query: "off",
|
||||
selectedKB: null,
|
||||
stream: true,
|
||||
summary_title: true,
|
||||
summary_title: false,
|
||||
history_round: 5,
|
||||
})
|
||||
|
||||
@ -313,11 +314,12 @@ const appendUserMessage = (message) => {
|
||||
scrollToBottom()
|
||||
}
|
||||
|
||||
const appendAiMessage = (message, refs=null) => {
|
||||
const appendAiMessage = (text, refs=null) => {
|
||||
conv.value.messages.push({
|
||||
id: generateRandomHash(16),
|
||||
role: 'received',
|
||||
text: message,
|
||||
text: text,
|
||||
reasoning: '',
|
||||
refs,
|
||||
status: "init",
|
||||
meta: {},
|
||||
@ -329,33 +331,42 @@ const updateMessage = (info) => {
|
||||
const message = conv.value.messages.find((message) => message.id === info.id);
|
||||
|
||||
if (message) {
|
||||
// 只有在 text 不为空时更新
|
||||
if (info.text !== null && info.text !== undefined && info.text !== '') {
|
||||
message.text = info.text;
|
||||
}
|
||||
try {
|
||||
// 只有在 text 不为空时更新
|
||||
if (info.text !== null && info.text !== undefined && info.text !== '') {
|
||||
message.text = info.text;
|
||||
}
|
||||
|
||||
// 只有在 refs 不为空时更新
|
||||
if (info.refs !== null && info.refs !== undefined) {
|
||||
message.refs = info.refs;
|
||||
}
|
||||
if (info.reasoning !== null && info.reasoning !== undefined && info.reasoning !== '') {
|
||||
message.reasoning = info.reasoning;
|
||||
}
|
||||
|
||||
if (info.model_name !== null && info.model_name !== undefined && info.model_name !== '') {
|
||||
message.model_name = info.model_name;
|
||||
}
|
||||
// 只有在 refs 不为空时更新
|
||||
if (info.refs !== null && info.refs !== undefined) {
|
||||
message.refs = info.refs;
|
||||
}
|
||||
|
||||
if (info.model_name !== null && info.model_name !== undefined && info.model_name !== '') {
|
||||
message.model_name = info.model_name;
|
||||
}
|
||||
|
||||
// 只有在 status 不为空时更新
|
||||
if (info.status !== null && info.status !== undefined && info.status !== '') {
|
||||
message.status = info.status;
|
||||
}
|
||||
if (info.status !== null && info.status !== undefined && info.status !== '') {
|
||||
message.status = info.status;
|
||||
}
|
||||
|
||||
if (info.meta !== null && info.meta !== undefined) {
|
||||
message.meta = info.meta;
|
||||
if (info.meta !== null && info.meta !== undefined) {
|
||||
message.meta = info.meta;
|
||||
}
|
||||
scrollToBottom();
|
||||
} catch (error) {
|
||||
console.error('Error updating message:', error);
|
||||
message.status = 'error';
|
||||
message.text = '消息更新失败';
|
||||
}
|
||||
} else {
|
||||
console.error('Message not found');
|
||||
console.error('Message not found:', info.id);
|
||||
}
|
||||
|
||||
scrollToBottom();
|
||||
};
|
||||
|
||||
|
||||
@ -378,7 +389,7 @@ const groupRefs = (id) => {
|
||||
|
||||
const simpleCall = (message) => {
|
||||
return new Promise((resolve, reject) => {
|
||||
fetch('/api/chat/call', {
|
||||
fetch('/api/chat/call_lite', {
|
||||
method: 'POST',
|
||||
body: JSON.stringify({ query: message, }),
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
@ -457,10 +468,12 @@ const fetchChatResponse = (user_input, cur_res_id) => {
|
||||
updateMessage({
|
||||
id: cur_res_id,
|
||||
text: data.response,
|
||||
reasoning: data.reasoning_response,
|
||||
model_name: data.model_name,
|
||||
status: data.status,
|
||||
meta: data.meta,
|
||||
});
|
||||
// console.log(data)
|
||||
// console.log("Last message", conv.value.messages[conv.value.messages.length - 1].text)
|
||||
// console.log("Last message", conv.value.messages[conv.value.messages.length - 1].status)
|
||||
|
||||
@ -648,6 +661,17 @@ watch(
|
||||
&:hover {
|
||||
background-color: var(--main-light-3);
|
||||
}
|
||||
|
||||
.anticon {
|
||||
margin-right: 8px;
|
||||
font-size: 16px;
|
||||
}
|
||||
|
||||
.ant-switch {
|
||||
&.ant-switch-checked {
|
||||
background-color: var(--main-500);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -977,7 +1001,15 @@ watch(
|
||||
}
|
||||
}
|
||||
|
||||
.controls {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
|
||||
.search-switch {
|
||||
margin-right: 8px;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
|
||||
<style lang="less">
|
||||
|
||||
@ -2,8 +2,8 @@
|
||||
<div class="refs" v-if="showRefs">
|
||||
<div class="tags">
|
||||
<span class="item btn" @click="copyText(msg.text)"><CopyOutlined /></span>
|
||||
<span class="item btn" @click="likeThisResponse(msg)"><LikeOutlined /></span>
|
||||
<span class="item btn" @click="dislikeThisResponse(msg)"><DislikeOutlined /></span>
|
||||
<!-- <span class="item btn" @click="likeThisResponse(msg)"><LikeOutlined /></span> -->
|
||||
<!-- <span class="item btn" @click="dislikeThisResponse(msg)"><DislikeOutlined /></span> -->
|
||||
<span class="item"><GlobalOutlined /> {{ msg.model_name }}</span>
|
||||
<span
|
||||
class="item btn"
|
||||
|
||||
212
web/src/components/TableConfigComponent.vue
Normal file
212
web/src/components/TableConfigComponent.vue
Normal file
@ -0,0 +1,212 @@
|
||||
<template>
|
||||
<a-card class="config-card" style="max-width: 960px">
|
||||
<a-form layout="vertical">
|
||||
<div
|
||||
v-for="(item, index) in configList"
|
||||
:key="index"
|
||||
class="config-item"
|
||||
>
|
||||
<a-row :gutter="[16, 8]" align="middle">
|
||||
<a-col :span="8">
|
||||
<a-input
|
||||
v-model:value="item.key"
|
||||
placeholder="模型名称"
|
||||
readonly
|
||||
class="key-input"
|
||||
/>
|
||||
</a-col>
|
||||
<a-col :span="14">
|
||||
<a-input
|
||||
v-model:value="item.value"
|
||||
@change="updateValue(index)"
|
||||
placeholder="模型本地路径"
|
||||
class="value-input"
|
||||
/>
|
||||
</a-col>
|
||||
<a-col :span="2" class="delete-btn-col">
|
||||
<a-button
|
||||
type="link"
|
||||
danger
|
||||
class="delete-btn"
|
||||
@click="deleteConfig(index)"
|
||||
>
|
||||
<DeleteOutlined />
|
||||
</a-button>
|
||||
</a-col>
|
||||
</a-row>
|
||||
</div>
|
||||
|
||||
<a-button block @click="addConfig" class="add-btn" :disabled="isAdding">
|
||||
<PlusOutlined /> 添加路径映射
|
||||
</a-button>
|
||||
</a-form>
|
||||
|
||||
<a-modal
|
||||
title="添加路径映射"
|
||||
v-model:open="addConfigModalVisible"
|
||||
@ok="confirmAddConfig"
|
||||
class="config-modal"
|
||||
>
|
||||
<a-form layout="vertical">
|
||||
<a-form-item label="模型名称(与Huggingface名称一致,比如 BAAI/bge-large-zh-v1.5" required>
|
||||
<a-input
|
||||
v-model:value="newConfig.key"
|
||||
placeholder="请输入模型名称"
|
||||
class="modal-input"
|
||||
/>
|
||||
</a-form-item>
|
||||
<a-form-item label="模型本地路径(绝对路径,比如 /hdd/models/BAAI/bge-large-zh-v1.5)" required>
|
||||
<a-input
|
||||
v-model:value="newConfig.value"
|
||||
placeholder="请输入模型本地路径"
|
||||
class="modal-input"
|
||||
/>
|
||||
</a-form-item>
|
||||
</a-form>
|
||||
</a-modal>
|
||||
</a-card>
|
||||
</template>
|
||||
|
||||
<script setup>
|
||||
import { ref, reactive, computed, watch } from 'vue';
|
||||
import { message } from 'ant-design-vue';
|
||||
import { DeleteOutlined, PlusOutlined } from '@ant-design/icons-vue';
|
||||
|
||||
const props = defineProps({
|
||||
config: {
|
||||
type: Object,
|
||||
default: () => ({})
|
||||
}
|
||||
});
|
||||
|
||||
const emit = defineEmits(['update:config']);
|
||||
|
||||
// 配置列表
|
||||
const configList = reactive([]);
|
||||
|
||||
// 初始化配置列表
|
||||
props.config && Object.entries(props.config).forEach(([key, value]) => {
|
||||
configList.push({ key, value });
|
||||
});
|
||||
|
||||
// 控制模态框显示
|
||||
const addConfigModalVisible = ref(false);
|
||||
const isAdding = ref(false); // 添加新的ref变量
|
||||
|
||||
// 新增配置项数据
|
||||
const newConfig = ref({ key: '', value: '' });
|
||||
|
||||
// 添加配置
|
||||
const addConfig = () => {
|
||||
isAdding.value = true; // 设置添加状态
|
||||
addConfigModalVisible.value = true;
|
||||
};
|
||||
|
||||
// 确认添加配置
|
||||
const confirmAddConfig = () => {
|
||||
if (newConfig.value.key === '' || newConfig.value.value === '') {
|
||||
message.warning('键或值不能为空');
|
||||
return;
|
||||
}
|
||||
if (configList.some(item => item.key === newConfig.value.key)) {
|
||||
message.warning('键已存在');
|
||||
return;
|
||||
}
|
||||
configList.push({ key: newConfig.value.key, value: newConfig.value.value });
|
||||
addConfigModalVisible.value = false;
|
||||
newConfig.value = { key: '', value: '' };
|
||||
isAdding.value = false; // 重置添加状态
|
||||
};
|
||||
|
||||
// 删除配置
|
||||
const deleteConfig = (index) => {
|
||||
configList.splice(index, 1);
|
||||
};
|
||||
|
||||
// 更新值
|
||||
const updateValue = (index) => {
|
||||
// 值的更新实时反映在 configList 中,无需额外处理
|
||||
};
|
||||
|
||||
// 将配置列表转换为对象
|
||||
const configObject = computed(() => {
|
||||
return configList.reduce((acc, item) => {
|
||||
acc[item.key] = item.value;
|
||||
return acc;
|
||||
}, {});
|
||||
});
|
||||
|
||||
// 监听配置变化并传递回父组件
|
||||
watch(configObject, (newValue) => {
|
||||
emit('update:config', newValue);
|
||||
}, { deep: true });
|
||||
|
||||
// 监听模态框关闭
|
||||
watch(addConfigModalVisible, (newValue) => {
|
||||
if (!newValue) {
|
||||
isAdding.value = false; // 当模态框关闭时重置添加状态
|
||||
}
|
||||
});
|
||||
</script>
|
||||
|
||||
<style scoped>
|
||||
.config-card {
|
||||
background-color: var(--gray-10);
|
||||
border-radius: 8px;
|
||||
border: 1px solid var(--gray-300);
|
||||
}
|
||||
|
||||
.config-item {
|
||||
border-bottom: 1px solid #f0f0f0;
|
||||
padding: 12px 0;
|
||||
transition: background-color 0.3s ease;
|
||||
}
|
||||
|
||||
.config-item:hover {
|
||||
background-color: #fafafa;
|
||||
}
|
||||
|
||||
.config-item:last-child {
|
||||
border-bottom: none;
|
||||
}
|
||||
|
||||
.key-input {
|
||||
background-color: #f8f8f8;
|
||||
border-color: #e8e8e8;
|
||||
}
|
||||
|
||||
.delete-btn-col {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.delete-btn {
|
||||
opacity: 0.6;
|
||||
transition: opacity 0.3s ease;
|
||||
}
|
||||
|
||||
.delete-btn:hover {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.add-btn {
|
||||
margin-top: 16px;
|
||||
height: 40px;
|
||||
transition: all 0.3s ease;
|
||||
width: auto;
|
||||
}
|
||||
|
||||
.modal-input {
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
:deep(.ant-modal-content) {
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
:deep(.ant-card-body) {
|
||||
padding: 16px;
|
||||
}
|
||||
</style>
|
||||
|
||||
@ -11,7 +11,7 @@ const router = createRouter({
|
||||
component: BlankLayout,
|
||||
children: [ {
|
||||
path: '',
|
||||
name: 'home',
|
||||
name: 'Home',
|
||||
component: () => import('../views/HomeView.vue'),
|
||||
meta: { keepAlive: true }
|
||||
}
|
||||
|
||||
@ -132,7 +132,7 @@
|
||||
<div class="params-group">
|
||||
<div class="params-item col">
|
||||
<p>重写查询<small>(修改后需重新检索)</small>:</p>
|
||||
<a-segmented v-model:value="meta.rewriteQuery" :options="rewriteQueryOptions">
|
||||
<a-segmented v-model:value="meta.use_rewrite_query" :options="use_rewrite_queryOptions">
|
||||
<template #label="{ payload }">
|
||||
<div>
|
||||
<p style="margin: 4px 0">{{ payload.subTitle }}</p>
|
||||
@ -251,13 +251,13 @@ const meta = reactive({
|
||||
mode: 'search',
|
||||
maxQueryCount: 30,
|
||||
filter: true,
|
||||
rewriteQuery: 'off',
|
||||
use_rewrite_query: 'off',
|
||||
rerankThreshold: 0.1,
|
||||
distanceThreshold: 0.3,
|
||||
topK: 10,
|
||||
});
|
||||
|
||||
const rewriteQueryOptions = ref([
|
||||
const use_rewrite_queryOptions = ref([
|
||||
{ value: 'off', payload: { title: 'off', subTitle: '不启用' } },
|
||||
{ value: 'on', payload: { title: 'on', subTitle: '启用重写' } },
|
||||
{ value: 'hyde', payload: { title: 'hyde', subTitle: '伪文档生成' } },
|
||||
@ -546,10 +546,7 @@ watch(() => meta, () => {
|
||||
|
||||
// 添加示例查询
|
||||
const queryExamples = ref([
|
||||
'食品添加剂的安全性如何?',
|
||||
'如何识别和预防食物中毒?',
|
||||
'转基因食品对人体健康有什么影响?',
|
||||
'如何正确储存和处理生鲜食品?'
|
||||
'贾宝玉的丫鬟有哪些?',
|
||||
]);
|
||||
|
||||
// 使用示例查询的方法
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
<div class="">
|
||||
<HeaderComponent title="设置" class="setting-header">
|
||||
<template #description>
|
||||
<p>配置文件也可以在 <code>saves/config/config.yaml</code> 中修改</p>
|
||||
<p>配置文件也可以在 <code>src/static/config.yaml</code> 中修改</p>
|
||||
</template>
|
||||
<template #actions>
|
||||
<a-button type="primary" v-if="isNeedRestart" @click="sendRestart">
|
||||
@ -79,6 +79,21 @@
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<h3>检索配置</h3>
|
||||
<div class="section">
|
||||
<div class="card">
|
||||
<span class="label">{{ items?.use_rewrite_query.des }}</span>
|
||||
<a-select style="width: 200px"
|
||||
:value="configStore.config?.use_rewrite_query"
|
||||
@change="handleChange('use_rewrite_query', $event)"
|
||||
>
|
||||
<a-select-option
|
||||
v-for="(name, idx) in items?.use_rewrite_query.choices" :key="idx"
|
||||
:value="name">{{ name }}
|
||||
</a-select-option>
|
||||
</a-select>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="setting" v-if="state.section === 'model'">
|
||||
<h3>模型配置</h3>
|
||||
@ -161,16 +176,21 @@
|
||||
</div>
|
||||
<div class="model-provider-card" v-for="(item, key) in notModelKeys" :key="key">
|
||||
<div class="card-header">
|
||||
<h3>{{ modelNames[item].name }}</h3>
|
||||
<a :href="modelNames[item].url" target="_blank">详情</a>
|
||||
<h3 style="font-weight: 400">{{ modelNames[item].name }}</h3>
|
||||
<a :href="modelNames[item].url" target="_blank"><InfoCircleOutlined /></a>
|
||||
<div class="missing-keys">
|
||||
需配置 <span v-for="(key, idx) in modelNames[item].env" :key="idx">{{ key }}</span>
|
||||
<small>需配置</small> <span v-for="(key, idx) in modelNames[item].env" :key="idx">{{ key }}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="setting" v-if="state.section ==='path'">
|
||||
<h3>暂无配置</h3>
|
||||
<h3>本地模型配置</h3>
|
||||
<p>如果是 Docker 启动,务必确保在环境变量中设置了 MODEL_DIR,或者设置了 volumes 映射</p>
|
||||
<TableConfigComponent
|
||||
:config="configStore.config?.model_local_paths"
|
||||
@update:config="handleModelLocalPathsUpdate"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@ -191,6 +211,7 @@ import {
|
||||
InfoCircleOutlined,
|
||||
} from '@ant-design/icons-vue';
|
||||
import HeaderComponent from '@/components/HeaderComponent.vue';
|
||||
import TableConfigComponent from '@/components/TableConfigComponent.vue';
|
||||
import { notification, Button } from 'ant-design-vue';
|
||||
|
||||
const configStore = useConfigStore()
|
||||
@ -231,6 +252,10 @@ const generateRandomHash = (length) => {
|
||||
return hash;
|
||||
}
|
||||
|
||||
const handleModelLocalPathsUpdate = (config) => {
|
||||
handleChange('model_local_paths', config)
|
||||
}
|
||||
|
||||
const handleChange = (key, e) => {
|
||||
if (key == 'enable_knowledge_graph' && e && !configStore.config.enable_knowledge_base) {
|
||||
message.error('启动知识图谱必须请先启用知识库功能')
|
||||
@ -249,7 +274,8 @@ const handleChange = (key, e) => {
|
||||
|| key == 'model_provider'
|
||||
|| key == 'model_name'
|
||||
|| key == 'embed_model'
|
||||
|| key == 'reranker') {
|
||||
|| key == 'reranker'
|
||||
|| key == 'model_local_paths') {
|
||||
if (!isNeedRestart.value) {
|
||||
isNeedRestart.value = true
|
||||
notification.info({
|
||||
|
||||
Loading…
Reference in New Issue
Block a user