Commit Graph

591 Commits

Author SHA1 Message Date
Wenjie Zhang
87807953b5 1. 添加 chunk 信息
2. 前端的chunk展示优化,默认使用 start_char_idx,如果没有则使用 id
3. 修改默认overlap为100token
2025-03-12 21:15:04 +08:00
Wenjie Zhang
568d859826
Merge pull request #65 from xerrors/61-bugtext-type-pdf
修复部分文档被识别为Text-lilke PDF 而导致没有触发OCR识别的问题
2025-03-11 18:12:18 +08:00
Wenjie Zhang
914e666436 修复OCR识别问题 2025-03-11 18:10:31 +08:00
Wenjie Zhang
0af178bf91 更新模型本地配置 2025-03-11 16:26:55 +08:00
Wenjie Zhang
d1c4b0d1ff 优化日志细节 2025-03-11 14:18:48 +08:00
Wenjie Zhang
a22f4d908c 修复模型切换bug 2025-03-10 16:33:40 +08:00
Wenjie Zhang
e3fd24f383 feat: 添加 indexing nodes 2025-03-08 22:49:13 +08:00
Wenjie Zhang
8132ac2e9b 更新 ocr 模型 2025-03-08 20:26:00 +08:00
Wenjie Zhang
6fdd134e26 添加 QWQ 在 DashScope上的支持,添加provider 的ICON。删除对于 百度千帆模型的支持。 2025-03-07 12:38:39 +08:00
Wenjie Zhang
15f52bfc8f 修复 embedding 和 reranker 无法本地调用 gpu 的问题,需要安装 nvidia-container-toolkit 2025-03-07 01:05:50 +08:00
Wenjie Zhang
952841c68d 优化部分并行性能 2025-03-06 23:55:20 +08:00
Wenjie Zhang
fb15e258db 支持 ollma embedding 2025-03-04 13:49:00 +08:00
Wenjie Zhang
f52d342afb update logger config 2025-03-03 22:06:28 +08:00
Wenjie Zhang
e998102eeb 修复本地embedding模型的运行bug,添加了 batch_encode 的方法 2025-03-03 21:57:27 +08:00
Wenjie Zhang
17163a3b86 优化部分UI界面 2025-03-03 19:46:36 +08:00
Wenjie Zhang
e6f76aee9d 修复ollama 2025-02-28 14:39:03 +08:00
Wenjie Zhang
577b78821f 修复子图检索的bug 2025-02-28 02:42:21 +08:00
Wenjie Zhang
bb88a696d8 修复embedding的bug 2025-02-28 02:41:45 +08:00
Wenjie Zhang
0023107c48 图的信息的持久存储 2025-02-28 02:40:34 +08:00
Wenjie Zhang
ce783a7e9b 图的显示和上传优化 2025-02-28 02:39:47 +08:00
Wenjie Zhang
99c7ac659b 暂存 2025-02-28 00:38:59 +08:00
Wenjie Zhang
582e367f52 无关紧要的更新 2025-02-28 00:27:58 +08:00
Wenjie Zhang
c84fa50ef2 更新一些部署相关的更改 2025-02-28 00:26:57 +08:00
Wenjie Zhang
e04add5bc2 update logger 2025-02-27 19:35:25 +08:00
Wenjie Zhang
68a3ab621e 添加模型供应商:Together AI 2025-02-27 13:26:22 +08:00
Wenjie Zhang
a2e26e34c8 更新模型列表 2025-02-27 00:10:51 +08:00
Wenjie Zhang
0e3d384d4d 再次优化配置文件路径 2025-02-26 23:58:26 +08:00
Wenjie Zhang
c20ac54515 stop tracking config.dev.yaml 2025-02-25 21:38:00 +08:00
Wenjie Zhang
5f669f2646 update details 2025-02-25 21:36:00 +08:00
Wenjie Zhang
af827d5437 完善 debug 信息 2025-02-25 21:26:55 +08:00
Wenjie Zhang
becbe5864d 添加更多模型支持 2025-02-25 21:26:37 +08:00
Wenjie Zhang
a155fd97a3 优化响应逻辑 2025-02-25 14:45:49 +08:00
Wenjie Zhang
4419de18f7 优化输入表现和网页搜索结果 2025-02-24 19:43:40 +08:00
Wenjie Zhang
04d7ddb741 新增网页检索,需要配置TAVILY_API_KEY 2025-02-24 14:58:35 +08:00
Wenjie Zhang
f1ba6cd787 修改环境要求,确保在 ARM 架构的设备上也能正常使用 Docker 启动 2025-02-24 11:27:08 +08:00
Wenjie Zhang
19a8e4714e 细节优化 2025-02-23 16:39:52 +08:00
Wenjie Zhang
b2dbc17fec 修改并适配硅基流动模型 2025-02-23 16:38:56 +08:00
Wenjie Zhang
4badd193b9 Merge branch 'main' of https://github.com/xerrors/Yuxi-Know 2025-02-20 02:04:56 +08:00
Wenjie Zhang
d9e39a7a7d Merge branch 'main' of https://github.com/xerrors/Yuxi-Know 2025-02-20 01:26:12 +08:00
Wenjie Zhang
f1da492444 update deploy details 2025-02-19 19:59:50 +08:00
littlewwwhite
0b62cf686d bug: -- 2025-02-15 23:25:51 +08:00
littlewwwhite
31bbaa307d feat: 添加了基于 tavil的web 搜索 2025-02-15 19:31:33 +08:00
littlewwwhite
5453452452 feat: 添加了基于 tavil的web 搜索 2025-02-15 19:29:32 +08:00
Wenjie Zhang
66e18298ce 添加BGE-M3 2024-11-23 22:35:30 +08:00
Wenjie Zhang
bd679f47d6 本地模型参数修改(未测试) 2024-11-23 22:25:46 +08:00
Wenjie Zhang
6854a7812f 修复 rewrite_query 的问题 2024-11-23 20:53:51 +08:00
Wenjie Zhang
5cad89e3d4 fix embedding bug 2024-11-22 00:13:01 +08:00
Wenjie Zhang
c51c9b152d fix 2024-11-22 00:04:30 +08:00
Wenjie Zhang
6433046c0b fix model info bugs 2024-11-22 00:03:25 +08:00
Wenjie Zhang
8c26722aa9 fix graph init bugs 2024-11-21 23:50:52 +08:00
Wenjie Zhang
c6a925a4c5 fix models.yaml empty error 2024-11-21 15:07:29 +08:00
Wenjie Zhang
40f5b3dc1f 更新了一些细节 2024-11-14 20:07:49 +08:00
Wenjie Zhang
7e23545ea6
Create .env.template 2024-11-14 12:21:00 +08:00
Wenjie Zhang
aeabe769d6 优化部分视觉表现 2024-10-24 20:08:23 +08:00
Wenjie Zhang
7b021e1efc 修复自定义模型名称修改问题 2024-10-24 17:32:24 +08:00
Wenjie Zhang
f0984191ae 修复config 不同步的问题 2024-10-24 13:09:59 +08:00
Wenjie Zhang
1115bf18b2 update model details 2024-10-24 13:09:03 +08:00
Wenjie Zhang
f3ddd1d211 修复refs 获取存在问题 2024-10-23 16:24:29 +08:00
Wenjie Zhang
c533f86b3f 支持多线程 2024-10-23 16:24:09 +08:00
Wenjie Zhang
5c1dc66134 update readme 2024-10-17 16:20:48 +08:00
Wenjie Zhang
bf8580d649 优化响应逻辑以及中间状态显示 2024-10-15 00:38:24 +08:00
Wenjie Zhang
d18a0a1039 修复并发问题,修复config不更新的问题 2024-10-14 22:10:56 +08:00
Wenjie Zhang
0ff38ffe8c update 2024-10-14 16:51:20 +08:00
Wenjie Zhang
39ff8a07a9 sync 2024-10-08 22:16:17 +08:00
Wenjie Zhang
02686db89f temp 2024-10-02 20:11:28 +08:00
Wenjie Zhang
9744b738ba add pad2txt tools 2024-09-29 16:13:35 +08:00
Wenjie Zhang
e968a3d9f8 fix pdf parser bug 2024-09-29 16:13:06 +08:00
Wenjie Zhang
ee84c4f391
Update README.md 2024-09-28 16:20:32 +08:00
Wenjie Zhang
5dbb1f4d76 update setting page 2024-09-28 00:41:18 +08:00
Wenjie Zhang
ee950e40c9 update zhipuai_api_key and siliconflow 2024-09-28 00:39:31 +08:00
Wenjie Zhang
542562dd5b fix chunking 2024-09-26 22:45:02 +08:00
Wenjie Zhang
6f81b061d2 update tools view and indexing 2024-09-25 13:46:23 +08:00
Wenjie Zhang
b6abb2c8e7 graph support 2024-09-18 21:03:12 +08:00
Wenjie Zhang
0780638152 rename 2024-09-14 02:45:35 +08:00
Wenjie Zhang
5c03d3d7cc fix reranker bug 2024-09-14 02:45:13 +08:00
Wenjie Zhang
5925e359e9 fix: 图查询出错的问题 2024-09-14 02:44:53 +08:00
Wenjie Zhang
0f249759a7 fix reranker bug 2024-09-13 15:25:01 +08:00
Wenjie Zhang
13c6037bdb optimize: log style 2024-09-13 15:24:36 +08:00
Wenjie Zhang
effa086c36 fix log bugs 2024-09-11 01:08:44 +08:00
Wenjie Zhang
a3a7ab034b trans cv2 to PIL 2024-09-11 01:08:37 +08:00
Wenjie Zhang
152bfbf49f fix graph bugs 2024-09-11 01:08:13 +08:00
Wenjie Zhang
7ce9b5bd8a fix local model bugs 2024-09-11 01:07:19 +08:00
Wenjie Zhang
302e59c2c9 update 2024-09-09 17:07:03 +08:00
Wenjie Zhang
b7e65fe108 fix graph bugs 2024-09-06 21:30:38 +08:00
Wenjie Zhang
0d455bcc3d update 2024-09-06 12:54:17 +08:00
xerrors
4f4928e536 syncupdate packages 2024-09-03 16:37:59 +08:00
xerrors
d89b7fdeec syc 2024-08-30 11:54:21 +08:00
Wenjie Zhang
0b2dd0179b sync 2024-08-25 20:29:24 +08:00
Wenjie Zhang
def68e8a3e sync 2024-08-25 12:34:35 +08:00
Wenjie Zhang
4083fa6938 sync 2024-08-25 10:33:48 +08:00
Xerrors
4550635dc1 在检索测试里面支持 HyDE 2024-08-08 19:51:13 +08:00
Xerrors
14d965065c Merge branch 'dev' of https://github.com/xerrors/ProjectAthena into dev 2024-08-08 17:13:32 +08:00
Xerrors
32740f9979 reformat 2024-08-08 17:13:01 +08:00
Wenjie Zhang
4a429d666a 最小化依赖 2024-08-08 17:10:41 +08:00
Wenjie Zhang
317974c8ac 添加历史对话轮数的支持,默认 5 轮对话 2024-08-07 17:42:26 +08:00
Wenjie Zhang
312e3af23e Merge branch 'dev' of https://github.com/xerrors/ProjectAthena into dev 2024-08-07 17:25:16 +08:00
Wenjie Zhang
adf130740b 最小化启动依赖 2024-08-07 17:24:46 +08:00
Wenjie Zhang
8d3280997e
Update README.md 2024-08-07 16:14:23 +08:00
QiyiJiang
d5121b446a 图检索中的query_by_vector小bug 2024-07-31 14:50:46 +00:00
Wenjie Zhang
cb00a8f4a9 update kb config 2024-07-31 20:22:05 +08:00
Wenjie Zhang
1906c97782 database query test 2024-07-29 01:00:02 +08:00
Wenjie Zhang
6971e7e1b7 修改后端入口为 src, 运行代码不需要使用 cd src, 添加前端查看检索结果 2024-07-28 16:16:52 +08:00
Wenjie Zhang
0d67db6629 Merge branch 'dev' of https://github.com/xerrors/ProjectAthena into dev 2024-07-25 20:30:40 +08:00
Wenjie Zhang
84f6761a67 升级了设置页面/聊天布局/数据库展示等,新增了参考资料查询 2024-07-25 20:30:28 +08:00
QiyiJiang
61ba90d301 query_by_vector添加检索数量和阈值两个参数 2024-07-24 12:23:31 +00:00
QiyiJiang
28e2b4bd0d 通过jsonl文件批量添加三元组到知识图谱 2024-07-24 11:35:33 +00:00
Wenjie Zhang
4e1be33ebb update graph page 2024-07-24 18:32:14 +08:00
Wenjie Zhang
3ab20611a2 modified config embedding model and startup 2024-07-22 00:00:54 +08:00
Wenjie Zhang
590dfd1836 Merge branch 'dev' of https://github.com/xerrors/ProjectAthena into dev 2024-07-21 18:17:25 +08:00
Wenjie Zhang
429c5e3b14 update some details 2024-07-21 18:15:28 +08:00
QiyiJiang
956a8d1b02 知识图谱检索问答 2024-07-20 12:30:32 +00:00
Wenjie Zhang
ddd4ce99c3 vllm supported 2024-07-20 15:27:33 +08:00
Wenjie Zhang
789ab07f51 some little details and change default qianfan model from erine_lit_8k to erine_speed 2024-07-18 02:47:41 +08:00
Wenjie Zhang
9e070a1318 update meta infos 2024-07-18 02:46:58 +08:00
Wenjie Zhang
d7ab60e564 format graph result 2024-07-17 20:53:55 +08:00
QiyiJiang
292803f989 Merge branch 'dev' of https://github.com/xerrors/ProjectAthena into dev 2024-07-17 11:50:33 +00:00
QiyiJiang
f9cabc54da change output 2024-07-17 11:34:05 +00:00
Wenjie Zhang
b0d0a77af3 fix bug 2024-07-17 19:10:41 +08:00
Wenjie Zhang
92a8ec3996 Merge branch 'dev' of https://github.com/xerrors/ProjectAthena into dev 2024-07-17 18:55:04 +08:00
Wenjie Zhang
86e8910eee update 2024-07-17 18:52:20 +08:00
QiyiJiang
62d1a86950 graph_retriever 2024-07-17 10:50:01 +00:00
Wenjie Zhang
6c5c099132 update details 2024-07-16 23:12:35 +08:00
Wenjie Zhang
d0614400ce add database api and front-end pages 2024-07-16 18:14:27 +08:00
QiyiJiang
158bc08bd6 指定跳数检索 2024-07-16 04:27:46 +00:00
QiyiJiang
df7a511001 graphbase_init 2024-07-15 10:58:34 +00:00
Wenjie Zhang
523cc63b51 add database server api partial support 2024-07-14 23:59:52 +08:00
Wenjie Zhang
e28f38627a Merge branch 'main' of https://github.com/xerrors/ProjectAthena into dev 2024-07-14 21:44:59 +08:00
Wenjie Zhang
5fbadef1d4 update 2024-07-14 21:44:27 +08:00
QiyiJiang
30be5efd3a jsonl 格式 2024-07-14 13:29:58 +00:00
QiyiJiang
812ad2cbd8 kg 修改 2024-07-14 13:04:00 +00:00
Wenjie Zhang
39e7e76e58 oneke update [未验证] 2024-07-14 18:31:23 +08:00
QiyiJiang
f7132e321f ocr and kg 2024-07-14 08:56:11 +00:00
QiyiJiang
401009bc9c makedir 2024-07-14 08:52:30 +00:00
Wenjie Zhang
d947d82602 UI重构以及历史问答bug修复 2024-07-14 16:42:38 +08:00
Wenjie Zhang
9cec978383 add plugin oneke 2024-07-10 12:57:04 +08:00
Wenjie Zhang
39bd774159 update retriever 2024-07-10 12:56:53 +08:00
Wenjie Zhang
a9124a1881 fix typo 2024-07-09 16:38:05 +08:00
Wenjie Zhang
b9d6d13d72 support for knowledge base RAG and fix UI bug 2024-07-09 05:04:20 +08:00
Wenjie Zhang
fdf016254d add wenxin support and fix double-predict bugs 2024-07-07 17:21:07 +08:00
Wenjie Zhang
00a0f87453 add requirements 2024-07-07 02:03:30 +08:00
Wenjie Zhang
9d6f9e35e7 first commit: chat support deepseek, zhipu 2024-07-07 01:58:23 +08:00