fix: 批次1爆盘根因 — alembic 独立事务 + 5日期字段合并单条ALTER
CI / backend (push) Waiting to run
CI / frontend (push) Waiting to run

生产 123 万行表(9.8GB)跑批次1时磁盘满回滚(2026-08-10)。
根因二:① alembic 无 transaction_per_migration,批次1的 01+02+03 全在一个
事务,JSON→JSONB 重写空间不释放、与后续重写叠加;② 1421ea169bb6 循环逐个
ALTER 5 个日期字段 = 5 次整表重写,同一事务累积。
修复:alembic.ini 开启 transaction_per_migration(每迁移独立提交,前序空间
释放);1421ea169bb6 合并 5 列为单条 ALTER(一次重写)。
本地 DB 已在 head 不受影响,heads 仍单一头 55105f0bb1d7。
This commit is contained in:
34047007@qq.com
2026-08-10 06:06:41 +08:00
parent 3c85ded216
commit 41b83e3a64
2 changed files with 19 additions and 13 deletions
+6
View File
@@ -1,6 +1,12 @@
[alembic]
script_location = alembic
sqlalchemy.url = postgresql://scilit:scilit_dev@localhost:5432/scilit
# Commit each migration in its own transaction (default: whole upgrade in one).
# Rewrite-heavy migrations (JSON->JSONB / ALTER COLUMN TYPE DATE) on the
# 1.23M-row prod table peak at huge disk usage; one shared txn accumulates
# them -> disk-full (2026-08-10 batch 1 incident root cause).
# Independent commits release space early.
transaction_per_migration = true
[loggers]
keys = root,sqlalchemy,alembic
@@ -21,19 +21,19 @@ DATE_FIELDS = ["pubmed_revised", "date_completed", "meshed_date", "create_date",
def upgrade() -> None:
for col in DATE_FIELDS:
op.alter_column("global_literature", col,
existing_type=postgresql.TIMESTAMP(timezone=True),
type_=sa.Date(),
existing_nullable=True,
postgresql_using=f"{col}::date",
# 合并为单条 ALTER:5 列一次全表重写。
# 原实现循环逐个 ALTER = 5 次整表重写,同一事务内空间峰值叠加,
# 生产 123 万行表(~9.8GB)会爆盘(2026-08-10 批次 1 事故根因)。
_cols = ", ".join(
f'ALTER COLUMN "{col}" TYPE DATE USING "{col}"::date'
for col in DATE_FIELDS
)
op.execute(f"ALTER TABLE global_literature {_cols}")
def downgrade() -> None:
for col in reversed(DATE_FIELDS):
op.alter_column("global_literature", col,
existing_type=sa.Date(),
type_=postgresql.TIMESTAMP(timezone=True),
existing_nullable=True,
_cols = ", ".join(
f'ALTER COLUMN "{col}" TYPE TIMESTAMP WITH TIME ZONE'
for col in reversed(DATE_FIELDS)
)
op.execute(f"ALTER TABLE global_literature {_cols}")