fix: 批次1爆盘根因 — alembic 独立事务 + 5日期字段合并单条ALTER
CI / backend (push) Waiting to run
CI / frontend (push) Waiting to run

生产 123 万行表(9.8GB)跑批次1时磁盘满回滚(2026-08-10)。
根因二:① alembic 无 transaction_per_migration,批次1的 01+02+03 全在一个
事务,JSON→JSONB 重写空间不释放、与后续重写叠加;② 1421ea169bb6 循环逐个
ALTER 5 个日期字段 = 5 次整表重写,同一事务累积。
修复:alembic.ini 开启 transaction_per_migration(每迁移独立提交,前序空间
释放);1421ea169bb6 合并 5 列为单条 ALTER(一次重写)。
本地 DB 已在 head 不受影响,heads 仍单一头 55105f0bb1d7。
This commit is contained in:
34047007@qq.com
2026-08-10 06:06:41 +08:00
parent 3c85ded216
commit 41b83e3a64
2 changed files with 19 additions and 13 deletions
+6
View File
@@ -1,6 +1,12 @@
[alembic] [alembic]
script_location = alembic script_location = alembic
sqlalchemy.url = postgresql://scilit:scilit_dev@localhost:5432/scilit sqlalchemy.url = postgresql://scilit:scilit_dev@localhost:5432/scilit
# Commit each migration in its own transaction (default: whole upgrade in one).
# Rewrite-heavy migrations (JSON->JSONB / ALTER COLUMN TYPE DATE) on the
# 1.23M-row prod table peak at huge disk usage; one shared txn accumulates
# them -> disk-full (2026-08-10 batch 1 incident root cause).
# Independent commits release space early.
transaction_per_migration = true
[loggers] [loggers]
keys = root,sqlalchemy,alembic keys = root,sqlalchemy,alembic
@@ -21,19 +21,19 @@ DATE_FIELDS = ["pubmed_revised", "date_completed", "meshed_date", "create_date",
def upgrade() -> None: def upgrade() -> None:
for col in DATE_FIELDS: # 合并为单条 ALTER:5 列一次全表重写。
op.alter_column("global_literature", col, # 原实现循环逐个 ALTER = 5 次整表重写,同一事务内空间峰值叠加,
existing_type=postgresql.TIMESTAMP(timezone=True), # 生产 123 万行表(~9.8GB)会爆盘(2026-08-10 批次 1 事故根因)。
type_=sa.Date(), _cols = ", ".join(
existing_nullable=True, f'ALTER COLUMN "{col}" TYPE DATE USING "{col}"::date'
postgresql_using=f"{col}::date", for col in DATE_FIELDS
) )
op.execute(f"ALTER TABLE global_literature {_cols}")
def downgrade() -> None: def downgrade() -> None:
for col in reversed(DATE_FIELDS): _cols = ", ".join(
op.alter_column("global_literature", col, f'ALTER COLUMN "{col}" TYPE TIMESTAMP WITH TIME ZONE'
existing_type=sa.Date(), for col in reversed(DATE_FIELDS)
type_=postgresql.TIMESTAMP(timezone=True), )
existing_nullable=True, op.execute(f"ALTER TABLE global_literature {_cols}")
)