fix: 第18轮搜索审计修复 — journal_iso补全/PA ui检查/text变量重命名等6项
- Bug-R18-1 (MEDIUM): MH:NOEXP在括号组内被忽略 → _parse_primary传播_noexp - Bug-R18-2 (MEDIUM): full_date:year同一年份不交换 → swap条件>=修复 - Bug-R18-3 (HIGH): 非DOI"/"路径缺失journal_iso → 补充ILIKE - Bug-R18-4 (MEDIUM): Chinese路径缺失journal_iso/pmid/doi → 补全ILIKE - Bug-R18-5 (MEDIUM): PA仅检查name不检查ui → 增加or_(contains(ui)) - Bug-R18-6 (MEDIUM): text变量遮蔽SQLAlchemy text() → 重命名为_term_text
This commit is contained in:
@@ -709,6 +709,10 @@ class PubmedQueryParser:
|
||||
_field = _normalize_field_label(_raw_field)
|
||||
for t in terms:
|
||||
t.field = _field
|
||||
# P17: (lung OR breast)[MH:NOEXP] — 将 _noexp 传播到组内词
|
||||
if _raw_field.startswith("MH:") and "NOEXP" in _raw_field.upper():
|
||||
for t in terms:
|
||||
t._noexp = True
|
||||
# P15-PRIMARY: 如果 _parse_or_expr 已为 AND 集群(如 A OR B AND C → [B, C] sub-group)
|
||||
# 或嵌套括号创建了子组,则这些子组已正确处理结构。
|
||||
# 只对尚未分组的词创建外层组,避免 Term 被放入两个组导致 _pubmed_conditions 重复处理。
|
||||
@@ -803,13 +807,13 @@ class PubmedQueryParser:
|
||||
start_val, end_val = end_val, start_val
|
||||
elif _start_is_digit and not _end_is_digit:
|
||||
try:
|
||||
if int(start_val) > int(end_val[:4]):
|
||||
if int(start_val) >= int(end_val[:4]):
|
||||
start_val, end_val = end_val, start_val
|
||||
except (ValueError, TypeError):
|
||||
pass
|
||||
elif not _start_is_digit and _end_is_digit:
|
||||
try:
|
||||
if int(start_val[:4]) > int(end_val):
|
||||
if int(start_val[:4]) >= int(end_val):
|
||||
start_val, end_val = end_val, start_val
|
||||
except (ValueError, TypeError):
|
||||
pass
|
||||
|
||||
@@ -1008,11 +1008,20 @@ class AdvancedSearchEngine:
|
||||
neg = [t for t in pp.pharmaco_terms if t.is_not]
|
||||
if pos:
|
||||
term_conditions.append(or_(*[
|
||||
GlobalLiterature.pharmacological_actions.cast(JSONB).contains([{"name": t.text}])
|
||||
or_(
|
||||
GlobalLiterature.pharmacological_actions.cast(JSONB).contains([{"name": t.text}]),
|
||||
GlobalLiterature.pharmacological_actions.cast(JSONB).contains([{"ui": t.text}]),
|
||||
)
|
||||
for t in pos
|
||||
]))
|
||||
if neg:
|
||||
neg_conds = [GlobalLiterature.pharmacological_actions.cast(JSONB).contains([{"name": t.text}]) for t in neg]
|
||||
neg_conds = [
|
||||
or_(
|
||||
GlobalLiterature.pharmacological_actions.cast(JSONB).contains([{"name": t.text}]),
|
||||
GlobalLiterature.pharmacological_actions.cast(JSONB).contains([{"ui": t.text}]),
|
||||
)
|
||||
for t in neg
|
||||
]
|
||||
term_conditions.append(not_(or_(*neg_conds)))
|
||||
|
||||
# P4: [OT] → keywords JSONB contains(不再映射到 all)
|
||||
@@ -1366,7 +1375,10 @@ class AdvancedSearchEngine:
|
||||
if field == "SI":
|
||||
return GlobalLiterature.databank_list.cast(JSONB).contains([{"accession_numbers": [term.text]}])
|
||||
if field == "PA":
|
||||
return GlobalLiterature.pharmacological_actions.cast(JSONB).contains([{"name": term.text}])
|
||||
return or_(
|
||||
GlobalLiterature.pharmacological_actions.cast(JSONB).contains([{"name": term.text}]),
|
||||
GlobalLiterature.pharmacological_actions.cast(JSONB).contains([{"ui": term.text}]),
|
||||
)
|
||||
if field == "ED":
|
||||
return GlobalLiterature.authors.cast(JSONB).contains([{"type": "editor", "family": term.text}])
|
||||
if field == "IR":
|
||||
@@ -1414,9 +1426,9 @@ class AdvancedSearchEngine:
|
||||
# P14: 日期字段在括号组内
|
||||
if field in ("DP", "EDAT", "CRDT", "MHDA", "LR", "DCOM", "DEP"):
|
||||
from app.services.pubmed_query_parser import _validate_date_str as _vds
|
||||
text = term.text
|
||||
if text.isdigit() and len(text) == 4:
|
||||
year = int(text)
|
||||
_term_text = term.text
|
||||
if _term_text.isdigit() and len(_term_text) == 4:
|
||||
year = int(_term_text)
|
||||
if field == "DP":
|
||||
return and_(GlobalLiterature.pub_year >= year, GlobalLiterature.pub_year <= year)
|
||||
col_map = {
|
||||
@@ -1542,6 +1554,7 @@ class AdvancedSearchEngine:
|
||||
GlobalLiterature.abstract.ilike(like_val),
|
||||
GlobalLiterature.author_names_text.ilike(like_val),
|
||||
GlobalLiterature.journal.ilike(like_val),
|
||||
GlobalLiterature.journal_iso.ilike(like_val),
|
||||
text("EXISTS (SELECT 1 FROM jsonb_array_elements(global_literature.authors) AS _e "
|
||||
"WHERE _e->>'affiliation' ILIKE :aff_pat)").bindparams(aff_pat=pat),
|
||||
)
|
||||
@@ -1552,6 +1565,9 @@ class AdvancedSearchEngine:
|
||||
GlobalLiterature.abstract.ilike(like_val),
|
||||
GlobalLiterature.author_names_text.ilike(like_val),
|
||||
GlobalLiterature.journal.ilike(like_val),
|
||||
GlobalLiterature.journal_iso.ilike(like_val),
|
||||
cast(GlobalLiterature.pmid, String).ilike(like_val),
|
||||
GlobalLiterature.doi.ilike(like_val),
|
||||
text("EXISTS (SELECT 1 FROM jsonb_array_elements(global_literature.authors) AS _e "
|
||||
"WHERE _e->>'affiliation' ILIKE :aff_pat)").bindparams(aff_pat=pat),
|
||||
)
|
||||
|
||||
+61
-2
@@ -2,7 +2,7 @@
|
||||
|
||||
> 本文档按修复轮次详细记录所有搜索功能合规性修复的背景、根因分析和修改内容。
|
||||
>
|
||||
> **累计**:14 轮,200 项修复,50+ 字段标签注册,1007 项测试覆盖,7 项已知限制
|
||||
> **累计**:18 轮,206 项修复,50+ 字段标签注册,1007 项测试覆盖,7 项已知限制
|
||||
> **时间跨度**:2026-07-24 ~ 2026-07-29
|
||||
> **核心文件**:`pubmed_query_parser.py`(~850 行)→ `search_engine.py`(~1360 行)
|
||||
|
||||
@@ -1253,10 +1253,69 @@
|
||||
|
||||
---
|
||||
|
||||
## 第十八轮:第 18 轮审计修复(6 项)
|
||||
|
||||
**日期**:2026-07-29
|
||||
**提交**:`5fa2fbe`(与第 17 轮同一提交)
|
||||
**数量**:6 项(1 HIGH + 5 MEDIUM)
|
||||
**触发**:用户第 13 次要求全面检查(Round 18,3 并行 agent:SQL 生成、Pub 高级搜索前端、解析器/搜索 parity)
|
||||
**测试**:1007 全部通过 + 前端 build 通过
|
||||
|
||||
### Bug-R18-1 (MEDIUM): MH:NOEXP 在括号组内被忽略
|
||||
|
||||
- **文件**:`pubmed_query_parser.py:710-712`
|
||||
- **根因**:`(lung OR breast)[MH:NOEXP]` 的 `[MH:NOEXP]` 末尾字段标签被 `_normalize_field_label` 返回 `"MH"`,但 `_noexp=True` 未传播到括号组内的各个 term。组内每个 term 照常走 MeSH 展开路径(`mesh_headings JSONB contains`),无视 NOEXP 要求
|
||||
- **修复**:在 `_parse_primary` 中,`_raw_field.startswith("MH:") and "NOEXP" in _raw_field.upper()` 时,将该 field 对应 group 内所有 term 标记 `_noexp=True`
|
||||
- **验证**:`(lung OR breast)[MH:NOEXP]` 正确使用 `mesh_headings JSONB contains` 而非 `explode`
|
||||
|
||||
### Bug-R18-2 (MEDIUM): 日期范围 full_date:year 同一年份不交换
|
||||
|
||||
- **文件**:`pubmed_query_parser.py:806,812`
|
||||
- **根因**:`2024-12-31:2024[DP]` 的 swap 条件 `int(start_val[:4]) > int(end_val)` → `2024 > 2024` 为 False,不触发交换。`2024-12-31`(完整日期)作为 start,`2024`(纯年份)作为 end,`_parse_range` 解释器要求 start ≤ end 才能正确生成范围条件
|
||||
- **修复**:将两个 swap 条件从 `>` 改为 `>=`
|
||||
- **验证**:`2024-12-31:2024[DP]` 正确交换为 `2024:2024-12-31`
|
||||
|
||||
### Bug-R18-3 (HIGH): 非 DOI "/" 路径缺失 journal_iso ILIKE
|
||||
|
||||
- **文件**:`search_engine.py:1538-1547`
|
||||
- **根因**:`_field_condition("all")` 的 "/" 分支(非 DOI、非 Chinese、非通配符)包含 `journal ILIKE` 但缺少 `journal_iso ILIKE`。其他所有 ILIKE 分支(wildcard、text、Chinese)都同时包含 `journal` 和 `journal_iso`。只有此分支遗漏了 `journal_iso`
|
||||
- **影响**:PubMed 等数据库中大量缩写刊名通过 `journal_iso` 存储,纯文本搜索不含斜杠的词时,缩写刊名匹配性能低于应有水平
|
||||
- **修复**:在 `or_(...)` 中加入 `GlobalLiterature.journal_iso.ilike(like_val)`
|
||||
|
||||
### Bug-R18-4 (MEDIUM): Chinese 路径缺失 journal_iso/pmid/doi ILIKE
|
||||
|
||||
- **文件**:`search_engine.py:1549-1557`
|
||||
- **根因**:Chinese 字符路径的 `or_(...)` 仅包含 `title/abstract/author_names_text/journal ILIKE` + `affiliation EXISTS`,比通配符分支少了 `journal_iso`、`pmid` 和 `doi` 字段
|
||||
- **修复**:补全 `journal_iso.ilike(like_val)`、`cast(GlobalLiterature.pmid, String).ilike(like_val)`、`GlobalLiterature.doi.ilike(like_val)`
|
||||
|
||||
### Bug-R18-5 (MEDIUM): PA 仅检查 name 不检查 ui
|
||||
|
||||
- **文件**:`search_engine.py:1011,1015,1369`
|
||||
- **根因**:`pharmacological_actions JSONB contains` 只检查 `{"name": t.text}`。但 PubMed PA 字段允许按 UI(唯一标识符)搜索,如 `d015056[PA]`(UI = D015056,对应名称 = "Antineoplastic Agents")。`pharmacological_actions` JSONB 中同时存储 `name` 和 `ui`,但代码只匹配 `name`
|
||||
- **影响**:按 UI 搜索 PA 时返回零结果
|
||||
- **修复**:批量路径(`_pubmed_conditions` 的 PA 块)和单路径(`_single_term_condition`)均改为 `or_(contains({"name": ...}), contains({"ui": ...}))`
|
||||
|
||||
### Bug-R18-6 (MEDIUM): text 变量遮蔽 SQLAlchemy text()
|
||||
|
||||
- **文件**:`search_engine.py:1417`
|
||||
- **根因**:`_single_term_condition` 中日期字段分支内 `text = term.text` 将 `text` 作为局部变量。虽然此后该分支未再调用 `text()`,但 `text` 命名会遮蔽 Python 内置/导入的 `text()`。若未来在该分支后增加 SQL text() 调用,将产生不易追踪的错误
|
||||
- **修复**:重命名为 `_term_text`
|
||||
|
||||
### 审计结果汇总
|
||||
|
||||
| 审计维度 | 结果 |
|
||||
|---------|------|
|
||||
| SQL 生成全路径 | ✅ "/" 分支 journal_iso 、Chinese 分支补全、PA ui 检查均已修复 |
|
||||
| Pub 高级搜索前端 | ✅ 无代码级 bug(auth 限制属设计决策、#N 解析由前端覆盖、公共路由约束合理) |
|
||||
| 解析器/搜索 parity | ✅ MH:NOEXP 组内传播、日期 swap 边界均已修复 |
|
||||
| text() 变量名 | ✅ 已重命名为 `_term_text` |
|
||||
|
||||
---
|
||||
|
||||
## 第十七轮:第 17 轮审计修复(3 项)
|
||||
|
||||
**日期**:2026-07-29
|
||||
**提交**:`(待推送)`
|
||||
**提交**:`5fa2fbe`
|
||||
**数量**:3 项(1 HIGH + 2 MEDIUM)
|
||||
**触发**:用户第 12 次要求全面检查(Round 17,3 并行 agent:`_is_flat_text` 降级路径、facet 一致性、前端参数映射)
|
||||
**审计**:facet 一致性和缓存键验证通过,未发现问题
|
||||
|
||||
Reference in New Issue
Block a user