feat: automate business upgrades and sync reviewed frontend
This commit is contained in:
+20
-13
@@ -15,23 +15,30 @@
|
||||
- 后端仅绑定 `127.0.0.1:9602`,前端仅绑定 `127.0.0.1:9601`。
|
||||
- 宝塔 `sw2.fuhua91.com` 代理至 9601,前端容器将 `/api/` 转发至后端并去除前缀,支持 WebSocket、SSE 和文件上传。
|
||||
|
||||
数据库沿用已迁移并验证的 `sw10`,设置 `admin.system.disable_auto_migrate: true`,由部署前的独立迁移命令更新结构。平台别名为 `lx/hw/yq/oa`,其中三个 SDK 别名连接 `game_lx/game_hw/game_yq` 测试库,Redis 沿用当前关闭状态。生产配置使用 `admin.app.env: production`,关闭开发 Swagger。七条常规任务启用(四条商务/区服日汇总、预警检测、两条有效端刷新),两个历史回填停用。**使用同一 sw10 时只运行一个调度实例,切换到服务器前停止本机的后端服务。**
|
||||
测试数据库为 `sw10`。程序和部署入口会自动识别旧商务库并完成整套升级;已有 `disable_auto_migrate` 配置不会跳过商务升级。平台别名为 `lx/hw/yq/oa`,其中三个 SDK 别名连接 `game_lx/game_hw/game_yq` 测试库,Redis 沿用当前关闭状态。生产配置使用 `admin.app.env: production`,关闭开发 Swagger。七条常规任务启用(四条商务/区服日汇总、预警检测、两条有效端刷新),两个历史回填停用。**使用同一 sw10 时只运行一个调度实例,切换到服务器前停止本机的后端服务。**
|
||||
|
||||
## 数据库迁移
|
||||
## 全自动数据库升级
|
||||
|
||||
新副本必须先完整复制旧商务数据库,再执行 [旧框架转换 SQL](../migrations/legacy-framework.sql),最后使用本次发布的后端执行全部增量迁移:
|
||||
将主数据库配置为准备接管的旧商务库,然后正常部署或启动程序即可。程序不创建空库替代旧数据,不需要手动运行 SQL、建立索引、回填角色映射或执行旧表清理。部署脚本中的 `-migrate` 与直接启动后端使用同一条自动流程。
|
||||
|
||||
执行顺序:识别旧库并取得迁移锁 → 备份受影响的数据 → 转换旧框架 → 执行全部增量迁移和索引 → 校验结构、数据记录和商务角色 → 备份待清理表 → 删除明确名单中的旧表 → 再次校验 → 开放业务接口及调度任务。首次接管前,旧程序应处于正常发布停写状态;检测到迁移期间账号发生变化时会停止清理。
|
||||
|
||||
阶段进度、失败原因、备份摘要和完成时间记录在 `sys_business_upgrade_state`,迁移版本仍记录在 `sys_schema_migrations`。同库的多个新实例会等待同一把锁。建索引不设置总超时,日志每 10 秒报告当前阶段;在全部步骤完成前,业务服务不会就绪。
|
||||
|
||||
备份自动写入配置文件旁的 `migration-backups/`,在当前 Docker 部署中对应 `/www/wwwroot/sw2.fuhua91.com/config/migration-backups/`。目录权限为 700、文件为 600,不对外提供下载。备份包含受转换影响的系统数据、待删除旧表及其结构;只新增索引的原始业务大表保留在原库,其旧结构以注释写入备份。渠道主键扩容涉及的 `tab_promote` 同时备份数据。备份完整写入并校验 SHA-256 后才允许清理,历史备份不会随容器替换删除。
|
||||
|
||||
旧角色映射会一次性回填到 `sys_authorities.business_level`,之后创建商务账号只按稳定的等级标识查角色。角色可改名,复制角色不复制该标识,内置商务角色禁止删除。旧用户 ID、角色 ID、权限关联及主键自增高水位保持迁移约定,不因移除映射表再次编号。
|
||||
|
||||
当前自动清理名单为 79 张:19 张旧框架归档、26 张迁移前备份、23 张旧 SnailJob 表、4 张旧代码生成/演示表、2 张旧预警去重备份,以及 5 张已归档的 `sw_kra_*` 迁移辅助表。正常业务表和两个必要的迁移状态表保留;原 156 表的已迁移库升级后为 78 表。初次迁移时沿用旧任务启停状态,已运行的新商务库则保持现有任务设置。
|
||||
|
||||
发生中断后,重新启动或重新部署会根据持久化状态继续:尚未提交数据的旧框架转换可重新创建空目标结构,已提交的数据不会再次映射 ID,已有索引会校验后跳过,部分完成的清理使用原来的完整备份继续。验证失败、备份损坏或发现外键/视图等依赖时保留原表并报告原因。数据库已改变时,部署脚本只在旧版本通过结构检查后才允许回退到旧镜像。
|
||||
|
||||
[框架转换 SQL](../migrations/legacy-framework.sql) 与 [清理 SQL](../migrations/cleanup-legacy.sql) 是程序内嵌的执行内容,供审阅和应急排查,正常部署无需手工调用。可选的只读诊断命令:
|
||||
|
||||
```bash
|
||||
/app/kratos-admin -conf /data/conf/config.yaml -migrate
|
||||
/app/kratos-admin -conf /data/conf/config.yaml -check-schema
|
||||
```
|
||||
|
||||
框架转换 SQL 的 `USE` 默认指向 sw10,其他目标库需修改该行。`-migrate` 使用与服务相同的模块迁移目录,包含新表、字段、索引、权限及任务定义,完成检查后退出;不会启动 HTTP、定时任务或 SDK 连接。`-check-schema` 只读检查版本、必需表和字段、主键与索引。旧业务表缺失时应重新完成复制,不能用空表替代历史数据。
|
||||
|
||||
转换 SQL 和增量迁移必须全部执行。服务关闭自动迁移后,会在连接失败、版本未完成或结构缺失时明确报错,不进入初始化向导。新增迁移需同时更新模块结构检查与相邻回归测试。部署脚本在停止旧调度实例后自动运行 `-migrate`,全部检查通过才启动新服务。
|
||||
|
||||
旧库 `tab_promote.id` 的 INT 自增容量由增量迁移扩大为同符号 BIGINT,保留现存 ID、自增高水位、关联数据与注释。渠道、玩家、流水等可能变大的旧业务 ID 在 JSON 和 Excel 中使用文本,请求兼容十进制字符串;不降低自增值或重新编号。
|
||||
|
||||
## 构建与发布
|
||||
|
||||
Windows 本机安装 Go、Node、pnpm、tar,先提交源代码,再执行:
|
||||
@@ -42,7 +49,7 @@ Windows 本机安装 Go、Node、pnpm、tar,先提交源代码,再执行:
|
||||
|
||||
输出位于 `.cache/deploy/sw2-<commit>.tar.gz` 及对应 SHA-256 文件。后端交叉编译为 Linux amd64,前端执行生产构建;包内只含二进制、前端产物、Dockerfile 和部署配置,真实运行配置及环境变量不进入镜像或 Git。
|
||||
|
||||
通过 SSH 将包上传到 226,校验 SHA-256 后解压到 `releases/<commit>/`。首次部署另行准备上述私有配置文件,保持 `admin.router_prefix` 为空,并确认主库为 sw10。SDK 环境变量名称为 `SW_SDK_AUTH_CODE`、`SW_SDK_MIGRATE_AUTH_CODE`、`SW_SDK_TABLE_PREFIX`,值从已有私有运行配置取得。
|
||||
通过 SSH 将包上传到 226,校验 SHA-256 后解压到 `releases/<commit>/`。首次部署准备上述私有配置文件,保持 `admin.router_prefix` 为空,主库填写待接管的旧商务数据库。SDK 环境变量名称为 `SW_SDK_AUTH_CODE`、`SW_SDK_MIGRATE_AUTH_CODE`、`SW_SDK_TABLE_PREFIX`,值从已有私有运行配置取得。
|
||||
|
||||
```bash
|
||||
bash /www/wwwroot/sw2.fuhua91.com/releases/<commit>/deploy.sh <commit> --build-only
|
||||
@@ -50,9 +57,9 @@ bash /www/wwwroot/sw2.fuhua91.com/releases/<commit>/deploy.sh <commit> --build-o
|
||||
bash /www/wwwroot/sw2.fuhua91.com/releases/<commit>/deploy.sh <commit>
|
||||
```
|
||||
|
||||
脚本先构建和检查镜像,再停止当前受管容器,保留一份停止的 `-previous` 容器。迁移或健康检查失败时恢复前一组容器。配置、上传文件和数据库不随镜像替换;索引在独立迁移容器中按版本执行。
|
||||
脚本先构建和检查镜像,再停止当前受管容器,保留一份停止的 `-previous` 容器。迁移或健康检查失败时,先检查旧版本是否兼容当前数据库,再决定是否恢复前一组容器。配置、备份和上传文件不随镜像替换;完整数据库升级在独立迁移容器中自动执行。
|
||||
|
||||
迁移不设置总等待时限,日志记录每一步的名称、结果和耗时;耗时索引完成前不会启动业务服务。业务容器退出或重启时立即回退。切换数据库前需备份运行配置;镜像回退不会自动还原数据库连接配置或撤销已执行的数据库迁移,回退版本需兼容当前结构和迁移版本。
|
||||
迁移不设置总等待时限,日志记录每一步的名称、结果和耗时;耗时索引完成前不会启动业务服务。业务容器退出或重启时进入兼容性检查后再处理回退。切换数据库前需备份运行配置;镜像回退不会自动还原数据库连接配置或撤销已执行的数据库迁移,回退版本需兼容当前结构和迁移版本。
|
||||
|
||||
首次接入域名时,备份宝塔原站点配置,再安装 `bt-nginx-sw2.fuhua91.com.conf` 至 `/www/server/panel/vhost/nginx/`,执行 `/www/server/nginx/sbin/nginx -t` 通过后重载。模板保留证书验证目录,当前监听 HTTP;后续由宝塔为 sw2 单独配置匹配域名的证书。
|
||||
|
||||
|
||||
@@ -47,7 +47,11 @@ try {
|
||||
foreach ($name in @('deploy.sh', 'wait-backend.sh', 'bt-nginx-sw2.fuhua91.com.conf')) {
|
||||
Copy-Item -LiteralPath (Join-Path $PSScriptRoot $name) -Destination (Join-Path $stage $name)
|
||||
}
|
||||
Copy-Item -LiteralPath (Join-Path $PSScriptRoot '..\migrations') -Destination (Join-Path $stage 'migrations') -Recurse
|
||||
$migrationOutput = Join-Path $stage 'migrations'
|
||||
New-Item -ItemType Directory -Force -Path $migrationOutput | Out-Null
|
||||
Get-ChildItem -LiteralPath (Join-Path $PSScriptRoot '..\migrations') -Filter '*.sql' -File | ForEach-Object {
|
||||
Copy-Item -LiteralPath $_.FullName -Destination $migrationOutput
|
||||
}
|
||||
& tar -czf $archive -C $stage .
|
||||
if ($LASTEXITCODE -ne 0) { throw 'Release archive failed' }
|
||||
$hash = (Get-FileHash -LiteralPath $archive -Algorithm SHA256).Hash.ToLowerInvariant()
|
||||
|
||||
+13
-2
@@ -73,6 +73,17 @@ rollback() {
|
||||
result=$?
|
||||
trap - ERR
|
||||
set +eu
|
||||
# A successful schema/data upgrade cannot be undone by switching images.
|
||||
# Probe the old binary before letting it touch the upgraded database.
|
||||
if exists sw2-backend-previous; then
|
||||
previous_image=$(docker inspect --format '{{.Config.Image}}' sw2-backend-previous)
|
||||
if ! docker run --rm --network "$NETWORK" --env-file "$BASE_DIR/runtime.env" \
|
||||
-v "$BASE_DIR/config:/data/conf:ro" -v "$BASE_DIR/log:/app/logs" \
|
||||
"$previous_image" /app/kratos-admin -conf /data/conf/config.yaml -check-schema; then
|
||||
echo 'Deployment failed after a database change. Upgrade state/backups and current containers are retained; the incompatible previous backend stays stopped. Redeploying resumes the database upgrade.' >&2
|
||||
exit "$result"
|
||||
fi
|
||||
fi
|
||||
for name in "${created[@]}"; do docker rm -f "$name" >/dev/null 2>&1; done
|
||||
for name in "${moved[@]}"; do
|
||||
if exists "$name-previous"; then docker rename "$name-previous" "$name"; fi
|
||||
@@ -91,10 +102,10 @@ for name in "${NAMES[@]}"; do
|
||||
fi
|
||||
done
|
||||
|
||||
echo 'Applying and verifying database migrations before starting the service.'
|
||||
echo 'Automatically converting, migrating, verifying and cleaning the business database before starting the service.'
|
||||
docker run --rm --name sw2-migrate --label com.sw2.application=sw2 \
|
||||
--network "$NETWORK" --env-file "$BASE_DIR/runtime.env" \
|
||||
-v "$BASE_DIR/config:/data/conf:ro" -v "$BASE_DIR/log:/app/logs" \
|
||||
-v "$BASE_DIR/config:/data/conf" -v "$BASE_DIR/log:/app/logs" \
|
||||
"$BACKEND_IMAGE" /app/kratos-admin -conf /data/conf/config.yaml -migrate
|
||||
|
||||
created+=(sw2-backend)
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
// Package migrations contains the reviewed SQL bundled with the application.
|
||||
// SQL execution and database lifecycle remain in internal/data.
|
||||
package migrations
|
||||
|
||||
import (
|
||||
_ "embed"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
//go:embed legacy-framework.sql
|
||||
var frameworkSQL string
|
||||
|
||||
//go:embed cleanup-legacy.sql
|
||||
var cleanupSQL string
|
||||
|
||||
func FrameworkSQL() string { return frameworkSQL }
|
||||
func CleanupSQL() string { return cleanupSQL }
|
||||
|
||||
// Statements understands the DELIMITER directives in the bundled, trusted
|
||||
// scripts. It does not split stored-procedure bodies on their semicolons.
|
||||
func Statements(source string) ([]string, error) {
|
||||
delimiter := ";"
|
||||
var buffer strings.Builder
|
||||
var statements []string
|
||||
for _, line := range strings.Split(source, "\n") {
|
||||
trimmed := strings.TrimSpace(line)
|
||||
if strings.HasPrefix(trimmed, "DELIMITER ") {
|
||||
delimiter = strings.TrimSpace(strings.TrimPrefix(trimmed, "DELIMITER "))
|
||||
if delimiter == "" {
|
||||
return nil, fmt.Errorf("empty SQL delimiter")
|
||||
}
|
||||
continue
|
||||
}
|
||||
if buffer.Len() == 0 && (trimmed == "" || strings.HasPrefix(trimmed, "--")) {
|
||||
continue
|
||||
}
|
||||
buffer.WriteString(line)
|
||||
buffer.WriteByte('\n')
|
||||
if strings.HasSuffix(trimmed, delimiter) {
|
||||
statement := strings.TrimSuffix(strings.TrimSpace(buffer.String()), delimiter)
|
||||
statements = append(statements, statement)
|
||||
buffer.Reset()
|
||||
}
|
||||
}
|
||||
if strings.TrimSpace(buffer.String()) != "" {
|
||||
return nil, fmt.Errorf("unterminated SQL script")
|
||||
}
|
||||
return statements, nil
|
||||
}
|
||||
|
||||
func CleanupTables() []string {
|
||||
block := regexp.MustCompile(`(?s)INSERT INTO _sw_cleanup_targets_20261106\([^\n]+\) VALUES\s*(.*?);`).FindStringSubmatch(cleanupSQL)
|
||||
if len(block) != 2 {
|
||||
panic("cleanup table manifest is missing")
|
||||
}
|
||||
var names []string
|
||||
for _, match := range regexp.MustCompile(`\('([a-z0-9_]+)'`).FindAllStringSubmatch(block[1], -1) {
|
||||
names = append(names, match[1])
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
func LegacyArchives() map[string]string {
|
||||
archives := map[string]string{}
|
||||
for _, match := range regexp.MustCompile("`([a-z0-9_]+)` TO `(legacy_sw_20261103_[a-z0-9_]+)`").FindAllStringSubmatch(frameworkSQL, -1) {
|
||||
archives[match[1]] = match[2]
|
||||
}
|
||||
return archives
|
||||
}
|
||||
|
||||
func FrameworkCreatedTables() []string {
|
||||
var names []string
|
||||
for _, match := range regexp.MustCompile("(?m)^CREATE TABLE `?([a-z0-9_]+)`? ").FindAllStringSubmatch(frameworkSQL, -1) {
|
||||
names = append(names, match[1])
|
||||
}
|
||||
return names
|
||||
}
|
||||
@@ -0,0 +1,401 @@
|
||||
-- SW → Kra 自动升级后的旧表清理,兼容 MySQL 5.7。
|
||||
-- 程序会自动调用本脚本;手工执行仅用于预览或恢复已到清理阶段的升级。
|
||||
-- 自动流程已完成私有备份、结构/索引及业务角色校验后,才允许进入删除阶段。
|
||||
-- 使用独立 MySQL 会话完整执行本文件;客户端需支持 DELIMITER,遇错停止,不使用 --force。
|
||||
-- 默认仅预览。确认输出名单和目标库后,将下方 0 改为 1,再完整执行。
|
||||
-- DROP TABLE 隐式提交,不能用 ROLLBACK 恢复;中途失败可在排除原因后重跑。
|
||||
-- 删除名单为明确表名,绝不按 legacy_、bak_、sj_ 等前缀批量匹配。
|
||||
-- 保留业务表、sys_schema_migrations 和 sys_business_upgrade_state;旧映射与核验明细已自动归档。
|
||||
USE `sw10`;
|
||||
SET @sw_cleanup_execute = 0;
|
||||
SET NAMES utf8mb4;
|
||||
|
||||
DELIMITER $$
|
||||
DROP PROCEDURE IF EXISTS sw_kra_cleanup_20261106$$
|
||||
CREATE PROCEDURE sw_kra_cleanup_20261106()
|
||||
cleanup_run: BEGIN
|
||||
DECLARE v_db VARCHAR(64);
|
||||
DECLARE v_lock_name VARCHAR(64);
|
||||
DECLARE v_locked INT DEFAULT 0;
|
||||
DECLARE v_prepared INT DEFAULT 0;
|
||||
DECLARE v_timeout_changed INT DEFAULT 0;
|
||||
DECLARE v_old_lock_wait BIGINT UNSIGNED;
|
||||
DECLARE v_name VARCHAR(64);
|
||||
DECLARE v_last VARCHAR(64) DEFAULT '';
|
||||
DECLARE v_message VARCHAR(128);
|
||||
DECLARE v_present INT DEFAULT 0;
|
||||
DECLARE v_dropped INT DEFAULT 0;
|
||||
DECLARE v_started DATETIME(6);
|
||||
DECLARE v_step_started DATETIME(6);
|
||||
DECLARE EXIT HANDLER FOR SQLEXCEPTION
|
||||
BEGIN
|
||||
IF v_prepared=1 THEN DEALLOCATE PREPARE sw_cleanup_drop; END IF;
|
||||
IF v_timeout_changed=1 THEN SET SESSION lock_wait_timeout=v_old_lock_wait; END IF;
|
||||
DROP TEMPORARY TABLE IF EXISTS _sw_cleanup_targets_20261106;
|
||||
DROP TEMPORARY TABLE IF EXISTS _sw_cleanup_required_tables_20261106;
|
||||
DROP TEMPORARY TABLE IF EXISTS _sw_cleanup_required_versions_20261106;
|
||||
IF v_locked=1 THEN DO RELEASE_LOCK(v_lock_name); END IF;
|
||||
RESIGNAL;
|
||||
END;
|
||||
|
||||
SET v_started=SYSDATE(6), v_db=DATABASE();
|
||||
IF v_db IS NULL OR LOWER(v_db) IN ('oa','game_lx','game_hw','game_yq','mysql','sys','information_schema','performance_schema') THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Refusing cleanup of source/platform/system database';
|
||||
END IF;
|
||||
IF @sw_cleanup_execute IS NULL OR @sw_cleanup_execute NOT IN (0,1) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='sw_cleanup_execute must be 0 (preview) or 1 (delete)';
|
||||
END IF;
|
||||
IF @@SESSION.foreign_key_checks<>1 THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Enable foreign_key_checks before cleanup';
|
||||
END IF;
|
||||
|
||||
-- 与旧框架转换和 -migrate 使用同一把库级锁。
|
||||
SET v_lock_name=CONCAT('sw_to_kra:',MD5(v_db));
|
||||
SELECT GET_LOCK(v_lock_name,0) INTO v_locked;
|
||||
IF COALESCE(v_locked,0)<>1 THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Migration/cleanup already running for this database';
|
||||
END IF;
|
||||
|
||||
CREATE TEMPORARY TABLE _sw_cleanup_targets_20261106 (
|
||||
table_name VARCHAR(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin PRIMARY KEY,
|
||||
category VARCHAR(80) NOT NULL
|
||||
);
|
||||
INSERT INTO _sw_cleanup_targets_20261106(table_name,category) VALUES
|
||||
('legacy_sw_20261103_sys_client','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_config','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_dept','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_dict_data','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_dict_type','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_login_info','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_menu','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_notice','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_oper_log','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_oss','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_oss_config','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_post','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_role','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_role_dept','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_role_menu','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_social','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_user','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_user_post','旧框架归档'),
|
||||
('legacy_sw_20261103_sys_user_role','旧框架归档'),
|
||||
('bak_sw2kra_20261103_sj_job','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_alert_operation_log','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_alert_record','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_alert_rule','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_client','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_config','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_dept','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_dict_data','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_dict_type','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_login_info','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_menu','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_message','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_notice','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_oper_log','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_oss','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_oss_config','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_post','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_role','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_role_dept','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_role_menu','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_social','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_user','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_user_bind_platform','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_user_post','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_user_preference','迁移前备份'),
|
||||
('bak_sw2kra_20261103_sys_user_role','迁移前备份'),
|
||||
('sj_distributed_lock','旧 SnailJob 框架'),
|
||||
('sj_group_config','旧 SnailJob 框架'),
|
||||
('sj_job','旧 SnailJob 框架'),
|
||||
('sj_job_executor','旧 SnailJob 框架'),
|
||||
('sj_job_log_message','旧 SnailJob 框架'),
|
||||
('sj_job_summary','旧 SnailJob 框架'),
|
||||
('sj_job_task','旧 SnailJob 框架'),
|
||||
('sj_job_task_batch','旧 SnailJob 框架'),
|
||||
('sj_namespace','旧 SnailJob 框架'),
|
||||
('sj_notify_config','旧 SnailJob 框架'),
|
||||
('sj_notify_recipient','旧 SnailJob 框架'),
|
||||
('sj_retry','旧 SnailJob 框架'),
|
||||
('sj_retry_dead_letter','旧 SnailJob 框架'),
|
||||
('sj_retry_scene_config','旧 SnailJob 框架'),
|
||||
('sj_retry_summary','旧 SnailJob 框架'),
|
||||
('sj_retry_task','旧 SnailJob 框架'),
|
||||
('sj_retry_task_log_message','旧 SnailJob 框架'),
|
||||
('sj_server_node','旧 SnailJob 框架'),
|
||||
('sj_system_user','旧 SnailJob 框架'),
|
||||
('sj_system_user_permission','旧 SnailJob 框架'),
|
||||
('sj_workflow','旧 SnailJob 框架'),
|
||||
('sj_workflow_node','旧 SnailJob 框架'),
|
||||
('sj_workflow_task_batch','旧 SnailJob 框架'),
|
||||
('gen_table','旧代码生成与演示'),
|
||||
('gen_table_column','旧代码生成与演示'),
|
||||
('test_demo','旧代码生成与演示'),
|
||||
('test_tree','旧代码生成与演示'),
|
||||
('sys_alert_dedup_bak_20261020_105142_128391_5084329','旧预警去重备份'),
|
||||
('sys_alert_dedup_bak_20261020_111958_265207_5085137','旧预警去重备份'),
|
||||
('sw_kra_api_contract','已归档的迁移辅助表'),
|
||||
('sw_kra_checks','已归档的迁移辅助表'),
|
||||
('sw_kra_id_map','已归档的迁移辅助表'),
|
||||
('sw_kra_menu_permissions','已归档的迁移辅助表'),
|
||||
('sw_kra_state','已归档的迁移辅助表')
|
||||
;
|
||||
CREATE TEMPORARY TABLE _sw_cleanup_required_tables_20261106 (
|
||||
table_name VARCHAR(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin PRIMARY KEY
|
||||
);
|
||||
INSERT INTO _sw_cleanup_required_tables_20261106(table_name) VALUES
|
||||
('casbin_rule'),
|
||||
('jwt_blacklists'),
|
||||
('media_attachment_category'),
|
||||
('media_file_upload_and_downloads'),
|
||||
('media_uploads'),
|
||||
('media_upload_chunks'),
|
||||
('pay_events'),
|
||||
('pay_orders'),
|
||||
('sys_alert_channel_lock'),
|
||||
('sys_alert_exemption'),
|
||||
('sys_alert_gap'),
|
||||
('sys_alert_operation_log'),
|
||||
('sys_alert_record'),
|
||||
('sys_alert_rule'),
|
||||
('sys_announcements'),
|
||||
('sys_apis'),
|
||||
('sys_api_tokens'),
|
||||
('sys_authorities'),
|
||||
('sys_authority_apis'),
|
||||
('sys_authority_btns'),
|
||||
('sys_authority_departments'),
|
||||
('sys_authority_menus'),
|
||||
('sys_base_menus'),
|
||||
('sys_base_menu_btns'),
|
||||
('sys_base_menu_parameters'),
|
||||
('sys_business_alert_locks'),
|
||||
('sys_business_binding_locks'),
|
||||
('sys_business_channel_effectiveness'),
|
||||
('sys_business_effectiveness_state'),
|
||||
('sys_business_groups'),
|
||||
('sys_business_group_members'),
|
||||
('sys_business_group_state'),
|
||||
('sys_business_member_profiles'),
|
||||
('sys_business_password_codes'),
|
||||
('sys_business_password_resets'),
|
||||
('sys_business_performance_goals'),
|
||||
('sys_business_report_build_locks'),
|
||||
('sys_data_access_logs'),
|
||||
('sys_departments'),
|
||||
('sys_dictionaries'),
|
||||
('sys_dictionary_details'),
|
||||
('sys_error'),
|
||||
('sys_export_templates'),
|
||||
('sys_export_template_condition'),
|
||||
('sys_export_template_join'),
|
||||
('sys_ignore_apis'),
|
||||
('sys_integration_configs'),
|
||||
('sys_login_logs'),
|
||||
('sys_message'),
|
||||
('sys_operation_records'),
|
||||
('sys_params'),
|
||||
('sys_positions'),
|
||||
('sys_schema_migrations'),
|
||||
('sys_security_config'),
|
||||
('sys_timed_tasks'),
|
||||
('sys_timed_task_logs'),
|
||||
('sys_users'),
|
||||
('sys_user_authority'),
|
||||
('sys_user_bind_platform'),
|
||||
('sys_user_departments'),
|
||||
('sys_user_positions'),
|
||||
('sys_user_preference'),
|
||||
('sys_versions'),
|
||||
('tab_game'),
|
||||
('tab_game_server'),
|
||||
('tab_promote'),
|
||||
('tab_promote_apply'),
|
||||
('tab_spend'),
|
||||
('tab_sw_report_day'),
|
||||
('tab_sw_report_day_key'),
|
||||
('tab_sw_report_server_day'),
|
||||
('tab_sw_report_server_day_key'),
|
||||
('tab_user'),
|
||||
('tab_user_day_login'),
|
||||
('tab_user_play_info'),
|
||||
('sys_business_upgrade_state')
|
||||
;
|
||||
CREATE TEMPORARY TABLE _sw_cleanup_required_versions_20261106 (
|
||||
id VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin PRIMARY KEY
|
||||
);
|
||||
INSERT INTO _sw_cleanup_required_versions_20261106(id) VALUES
|
||||
('202608200001_data_infrastructure'),
|
||||
('202608200002_system_schema'),
|
||||
('202608200003_payment_schema'),
|
||||
('202608200004_payment_defaults'),
|
||||
('202608210001_communication_integration_defaults'),
|
||||
('202608210002_communication_surface'),
|
||||
('202608210003_communication_test_surface'),
|
||||
('202608220001_task_schema'),
|
||||
('202608290001_payment_event_schema'),
|
||||
('202608290002_payment_admin_surface'),
|
||||
('202608290003_payment_order_read_permissions'),
|
||||
('202608290004_kafka_integration_default'),
|
||||
('202608290005_communication_integration_default_upgrade'),
|
||||
('202609290001_sms_admin_surface'),
|
||||
('202611030001_business_preference'),
|
||||
('202611030002_business_user_fields'),
|
||||
('202611030003_business_bindings'),
|
||||
('202611030004_business_binding_locks'),
|
||||
('202611030005_business_password_reset'),
|
||||
('202611030006_business_alert_schema'),
|
||||
('202611030007_business_alert_attachments'),
|
||||
('202611040008_business_report_build_locks'),
|
||||
('202611040009_business_menu_icons'),
|
||||
('202611040010_business_alert_trigger_locks'),
|
||||
('202611040011_business_registration_index'),
|
||||
('202611040012_business_server_report_index'),
|
||||
('202611040013_business_query_indexes'),
|
||||
('202611040014_business_report_build_indexes'),
|
||||
('202611040015_business_approved_game_index'),
|
||||
('202611040016_business_other_settings_surface'),
|
||||
('202611040017_business_groups_goals_effectiveness'),
|
||||
('202611040018_business_effectiveness_query_index'),
|
||||
('202611040019_business_performance_surface'),
|
||||
('202611050020_business_alert_v2'),
|
||||
('202611050021_business_alert_surface'),
|
||||
('202611060022_business_promote_id_capacity'),
|
||||
('202611060023_business_binding_status_index'),
|
||||
('202611060024_business_role_identity'),
|
||||
('202611060025_automatic_upgrade_state')
|
||||
;
|
||||
|
||||
-- 缺任意必需表时,在执行第一条 DROP 前停止。
|
||||
SET v_name=(SELECT MIN(r.table_name) FROM _sw_cleanup_required_tables_20261106 r
|
||||
WHERE NOT EXISTS (SELECT 1 FROM information_schema.tables t
|
||||
WHERE t.table_schema=v_db AND BINARY t.table_name=BINARY r.table_name AND t.table_type='BASE TABLE'));
|
||||
IF v_name IS NOT NULL THEN
|
||||
SET v_message=CONCAT('Required current table is missing: ',v_name);
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT=v_message;
|
||||
END IF;
|
||||
IF NOT EXISTS (SELECT 1 FROM sys_business_upgrade_state WHERE name='legacy-sw'
|
||||
AND version='20261106_automatic_v1' AND phase IN ('cleanup','complete')
|
||||
AND verified_at IS NOT NULL AND LENGTH(verified_migrations)=64
|
||||
AND cleanup_backup_path<>'' AND LENGTH(cleanup_sha256)=64) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Automatic upgrade and backup verification must finish before cleanup';
|
||||
END IF;
|
||||
-- 初次删除复核旧记录;中断后旧记录可能已删除,改由持久化的自动升级状态证明已校验。
|
||||
IF EXISTS(SELECT 1 FROM information_schema.tables WHERE table_schema=v_db AND table_name='sw_kra_state')
|
||||
AND EXISTS(SELECT 1 FROM information_schema.tables WHERE table_schema=v_db AND table_name='sw_kra_checks') THEN
|
||||
IF NOT EXISTS (SELECT 1 FROM sw_kra_state WHERE id=1
|
||||
AND version='20261103_framework_migration_v5' AND phase='completed' AND completed_at IS NOT NULL) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Legacy framework migration has not completed';
|
||||
END IF;
|
||||
IF (SELECT COUNT(DISTINCT check_name) FROM sw_kra_checks WHERE stage='backup')<>52
|
||||
OR (SELECT COUNT(DISTINCT check_name) FROM sw_kra_checks WHERE stage='migration')<>24
|
||||
OR EXISTS (SELECT 1 FROM sw_kra_checks WHERE passed<>1 OR expected_value IS NULL
|
||||
OR actual_value IS NULL OR expected_value<>actual_value) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Migration verification records are incomplete or failed';
|
||||
END IF;
|
||||
END IF;
|
||||
IF (SELECT COUNT(*) FROM sys_authorities WHERE deleted_at IS NULL AND business_level BETWEEN 1 AND 3 AND authority_id<>888)<>3
|
||||
OR (SELECT COUNT(DISTINCT business_level) FROM sys_authorities WHERE deleted_at IS NULL AND business_level BETWEEN 1 AND 3 AND authority_id<>888)<>3
|
||||
OR EXISTS(SELECT 1 FROM sys_authorities WHERE business_level IS NOT NULL AND (business_level NOT BETWEEN 1 AND 3 OR authority_id=888)) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Stable business role bindings are incomplete';
|
||||
END IF;
|
||||
IF (SELECT COUNT(*) FROM information_schema.statistics WHERE table_schema=v_db AND table_name='sys_authorities'
|
||||
AND index_name='uk_authority_business_level')<>1
|
||||
OR NOT EXISTS(SELECT 1 FROM information_schema.statistics WHERE table_schema=v_db AND table_name='sys_authorities'
|
||||
AND index_name='uk_authority_business_level' AND column_name='business_level' AND seq_in_index=1 AND non_unique=0 AND sub_part IS NULL) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Business role unique index is missing or incompatible';
|
||||
END IF;
|
||||
IF EXISTS (SELECT 1 FROM _sw_cleanup_required_versions_20261106 r
|
||||
WHERE NOT EXISTS (SELECT 1 FROM sys_schema_migrations m WHERE BINARY m.id=BINARY r.id)) THEN
|
||||
SELECT r.id AS missing_migration FROM _sw_cleanup_required_versions_20261106 r
|
||||
WHERE NOT EXISTS (SELECT 1 FROM sys_schema_migrations m WHERE BINARY m.id=BINARY r.id) ORDER BY r.id;
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Run all incremental migrations before cleanup';
|
||||
END IF;
|
||||
-- 不能仅凭手工填写 completed 状态删除尚未归档的旧框架。
|
||||
IF EXISTS (SELECT 1 FROM information_schema.tables t WHERE t.table_schema=v_db AND t.table_name IN (
|
||||
'sys_social','sys_dept','sys_user','sys_post','sys_role','sys_menu','sys_user_role','sys_role_menu',
|
||||
'sys_role_dept','sys_user_post','sys_oper_log','sys_dict_type','sys_dict_data','sys_config',
|
||||
'sys_login_info','sys_notice','sys_oss','sys_oss_config','sys_client')) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Unarchived legacy framework tables remain; cleanup stopped';
|
||||
END IF;
|
||||
IF EXISTS (SELECT 1 FROM information_schema.tables t JOIN _sw_cleanup_targets_20261106 c
|
||||
ON BINARY c.table_name=BINARY t.table_name WHERE t.table_schema=v_db AND t.table_type<>'BASE TABLE') THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='A cleanup target is not a base table';
|
||||
END IF;
|
||||
|
||||
-- 保留外键检查;任何引用(包括其他库)都需先处理,避免删除一半才遇到约束。
|
||||
IF EXISTS (SELECT 1 FROM information_schema.key_column_usage k JOIN _sw_cleanup_targets_20261106 c
|
||||
ON BINARY k.referenced_table_name=BINARY c.table_name WHERE k.referenced_table_schema=v_db) THEN
|
||||
SELECT DISTINCT k.table_schema,k.table_name,k.constraint_name,k.referenced_table_name
|
||||
FROM information_schema.key_column_usage k JOIN _sw_cleanup_targets_20261106 c
|
||||
ON BINARY k.referenced_table_name=BINARY c.table_name WHERE k.referenced_table_schema=v_db;
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Foreign keys reference cleanup targets';
|
||||
END IF;
|
||||
-- 检查当前库的视图、触发器、事件和存储程序;定义不可见时也停止。
|
||||
IF EXISTS (SELECT 1 FROM information_schema.views v WHERE v.table_schema=v_db AND
|
||||
(v.view_definition IS NULL OR EXISTS (SELECT 1 FROM _sw_cleanup_targets_20261106 c
|
||||
WHERE LOWER(v.view_definition) REGEXP CONCAT('(^|[^0-9a-z_])',c.table_name,'([^0-9a-z_]|$)')))) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Review views referencing cleanup targets before deletion';
|
||||
END IF;
|
||||
IF EXISTS (SELECT 1 FROM information_schema.triggers t WHERE t.trigger_schema=v_db AND
|
||||
(t.action_statement IS NULL OR EXISTS (SELECT 1 FROM _sw_cleanup_targets_20261106 c
|
||||
WHERE BINARY t.event_object_table=BINARY c.table_name
|
||||
OR LOWER(t.action_statement) REGEXP CONCAT('(^|[^0-9a-z_])',c.table_name,'([^0-9a-z_]|$)')))) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Review triggers referencing cleanup targets before deletion';
|
||||
END IF;
|
||||
IF EXISTS (SELECT 1 FROM information_schema.events e WHERE e.event_schema=v_db AND
|
||||
(e.event_definition IS NULL OR EXISTS (SELECT 1 FROM _sw_cleanup_targets_20261106 c
|
||||
WHERE LOWER(e.event_definition) REGEXP CONCAT('(^|[^0-9a-z_])',c.table_name,'([^0-9a-z_]|$)')))) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Review events referencing cleanup targets before deletion';
|
||||
END IF;
|
||||
IF EXISTS (SELECT 1 FROM information_schema.routines r WHERE r.routine_schema=v_db
|
||||
AND r.routine_name<>'sw_kra_cleanup_20261106' AND
|
||||
(r.routine_definition IS NULL OR EXISTS (SELECT 1 FROM _sw_cleanup_targets_20261106 c
|
||||
WHERE LOWER(r.routine_definition) REGEXP CONCAT('(^|[^0-9a-z_])',c.table_name,'([^0-9a-z_]|$)')))) THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Review routines referencing cleanup targets before deletion';
|
||||
END IF;
|
||||
|
||||
SELECT COUNT(*) INTO v_present FROM information_schema.tables t JOIN _sw_cleanup_targets_20261106 c
|
||||
ON BINARY t.table_name=BINARY c.table_name WHERE t.table_schema=v_db;
|
||||
SELECT v_db AS target_database,IF(@sw_cleanup_execute=0,'PREVIEW','DELETE') AS mode,
|
||||
v_present AS existing_target_tables;
|
||||
SELECT c.table_name,c.category,IF(t.table_name IS NULL,'already absent','to delete') AS status,
|
||||
t.table_rows AS estimated_rows,COALESCE(t.data_length,0)+COALESCE(t.index_length,0) AS estimated_bytes
|
||||
FROM _sw_cleanup_targets_20261106 c LEFT JOIN information_schema.tables t
|
||||
ON t.table_schema=v_db AND BINARY t.table_name=BINARY c.table_name ORDER BY c.category,c.table_name;
|
||||
|
||||
IF @sw_cleanup_execute=1 THEN
|
||||
SET v_old_lock_wait=@@SESSION.lock_wait_timeout, v_timeout_changed=1;
|
||||
-- 只限制等待元数据锁的时间,避免遗留连接占表后一直卡住;实际删除不设总超时。
|
||||
SET SESSION lock_wait_timeout=15;
|
||||
drop_loop: LOOP
|
||||
SET v_name=(SELECT MIN(table_name) FROM _sw_cleanup_targets_20261106 WHERE table_name>v_last);
|
||||
IF v_name IS NULL THEN LEAVE drop_loop; END IF;
|
||||
SET v_last=v_name;
|
||||
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_schema=v_db AND BINARY table_name=BINARY v_name) THEN
|
||||
SET v_step_started=SYSDATE(6);
|
||||
SET @sw_cleanup_drop_sql=CONCAT('DROP TABLE IF EXISTS `',REPLACE(v_db,'`','``'),'`.`',REPLACE(v_name,'`','``'),'`');
|
||||
PREPARE sw_cleanup_drop FROM @sw_cleanup_drop_sql;
|
||||
SET v_prepared=1;
|
||||
EXECUTE sw_cleanup_drop;
|
||||
DEALLOCATE PREPARE sw_cleanup_drop;
|
||||
SET v_prepared=0, v_dropped=v_dropped+1;
|
||||
SELECT v_name AS dropped_table,TIMESTAMPDIFF(MICROSECOND,v_step_started,SYSDATE(6))/1000 AS elapsed_ms;
|
||||
END IF;
|
||||
END LOOP;
|
||||
SET SESSION lock_wait_timeout=v_old_lock_wait;
|
||||
SET v_timeout_changed=0;
|
||||
END IF;
|
||||
|
||||
DROP TEMPORARY TABLE _sw_cleanup_targets_20261106;
|
||||
DROP TEMPORARY TABLE _sw_cleanup_required_tables_20261106;
|
||||
DROP TEMPORARY TABLE _sw_cleanup_required_versions_20261106;
|
||||
DO RELEASE_LOCK(v_lock_name);
|
||||
SET v_locked=0, @sw_cleanup_drop_sql=NULL;
|
||||
SELECT IF(@sw_cleanup_execute=0,'Preview completed; no tables deleted','Cleanup completed') AS result,
|
||||
v_dropped AS dropped_tables,TIMESTAMPDIFF(MICROSECOND,v_started,SYSDATE(6))/1000 AS elapsed_ms;
|
||||
END cleanup_run$$
|
||||
CALL sw_kra_cleanup_20261106()$$
|
||||
DROP PROCEDURE sw_kra_cleanup_20261106$$
|
||||
DELIMITER ;
|
||||
@@ -0,0 +1,297 @@
|
||||
package migrations_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"os"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
mysql "github.com/go-sql-driver/mysql"
|
||||
"kra/internal/modules"
|
||||
)
|
||||
|
||||
func cleanupSQL(t *testing.T) string {
|
||||
t.Helper()
|
||||
raw, err := os.ReadFile("cleanup-legacy.sql")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
func cleanupManifest(t *testing.T, source, name string) []string {
|
||||
t.Helper()
|
||||
block := regexp.MustCompile(`(?s)INSERT INTO ` + regexp.QuoteMeta(name) + `\([^\n]+\) VALUES\s*(.*?);`).FindStringSubmatch(source)
|
||||
if len(block) != 2 {
|
||||
t.Fatalf("missing manifest %s", name)
|
||||
}
|
||||
matches := regexp.MustCompile(`\('([a-z0-9_]+)'`).FindAllStringSubmatch(block[1], -1)
|
||||
var values []string
|
||||
seen := map[string]bool{}
|
||||
for _, match := range matches {
|
||||
if seen[match[1]] {
|
||||
t.Fatalf("duplicate %s entry %s", name, match[1])
|
||||
}
|
||||
seen[match[1]] = true
|
||||
values = append(values, match[1])
|
||||
}
|
||||
return values
|
||||
}
|
||||
|
||||
func TestCleanupManifestMatchesMigrationAndRetainsCurrentTables(t *testing.T) {
|
||||
source := cleanupSQL(t)
|
||||
targets := cleanupManifest(t, source, "_sw_cleanup_targets_20261106")
|
||||
required := cleanupManifest(t, source, "_sw_cleanup_required_tables_20261106")
|
||||
versions := cleanupManifest(t, source, "_sw_cleanup_required_versions_20261106")
|
||||
if len(targets) != 79 || len(required) != 76 {
|
||||
t.Fatalf("cleanup manifest changed: %d targets, %d required", len(targets), len(required))
|
||||
}
|
||||
set := map[string]bool{}
|
||||
for _, name := range targets {
|
||||
if strings.HasPrefix(name, "tab_") || name == "sys_business_upgrade_state" {
|
||||
t.Fatalf("current data included: %s", name)
|
||||
}
|
||||
set[name] = true
|
||||
}
|
||||
for _, name := range required {
|
||||
if set[name] {
|
||||
t.Fatalf("required table selected for deletion: %s", name)
|
||||
}
|
||||
}
|
||||
legacy, err := os.ReadFile("legacy-framework.sql")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, pattern := range []string{"TO `(legacy_sw_20261103_[a-z0-9_]+)`", "CREATE TABLE `(bak_sw2kra_20261103_[a-z0-9_]+)`"} {
|
||||
for _, match := range regexp.MustCompile(pattern).FindAllStringSubmatch(string(legacy), -1) {
|
||||
if !set[match[1]] {
|
||||
t.Fatalf("migration archive omitted: %s", match[1])
|
||||
}
|
||||
}
|
||||
}
|
||||
known := map[string]bool{}
|
||||
for _, id := range versions {
|
||||
known[id] = true
|
||||
}
|
||||
steps := modules.Catalog().MigrationSteps()
|
||||
if len(steps) != len(versions) {
|
||||
t.Fatal("refresh the cleanup version gate for the current catalog")
|
||||
}
|
||||
for _, step := range steps {
|
||||
if !known[step.ID] {
|
||||
t.Fatalf("missing required migration %s", step.ID)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(source, "SET @sw_cleanup_execute = 0;") {
|
||||
t.Fatal("cleanup must default to preview")
|
||||
}
|
||||
}
|
||||
|
||||
// DELIMITER belongs to the mysql client. Keep one connection for the routine,
|
||||
// session variables, advisory lock and temporary manifests.
|
||||
func runCleanupSQL(ctx context.Context, conn *sql.Conn, source, database string, execute bool) error {
|
||||
source = strings.Replace(source, "USE `sw10`;", "USE `"+database+"`;", 1)
|
||||
if execute {
|
||||
source = strings.Replace(source, "SET @sw_cleanup_execute = 0;", "SET @sw_cleanup_execute = 1;", 1)
|
||||
}
|
||||
delimiter, buffer := ";", ""
|
||||
for _, line := range strings.Split(source, "\n") {
|
||||
trimmed := strings.TrimSpace(line)
|
||||
if strings.HasPrefix(trimmed, "DELIMITER ") {
|
||||
delimiter = strings.TrimSpace(strings.TrimPrefix(trimmed, "DELIMITER "))
|
||||
continue
|
||||
}
|
||||
buffer += line + "\n"
|
||||
if strings.HasSuffix(trimmed, delimiter) {
|
||||
statement := strings.TrimSuffix(strings.TrimSpace(buffer), delimiter)
|
||||
if _, err := conn.ExecContext(ctx, statement); err != nil {
|
||||
return err
|
||||
}
|
||||
buffer = ""
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestMySQLCleanupPreflightPreviewDeletionAndRetry(t *testing.T) {
|
||||
dsn := os.Getenv("SW_TEST_MYSQL_DSN")
|
||||
if dsn == "" {
|
||||
t.Skip("set SW_TEST_MYSQL_DSN to an isolated local MySQL")
|
||||
}
|
||||
cfg, err := mysql.ParseDSN(dsn)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Net != "tcp" || !strings.HasPrefix(cfg.Addr, "127.0.0.1:") {
|
||||
t.Fatal("cleanup tests require a local isolated MySQL")
|
||||
}
|
||||
cfg.DBName = ""
|
||||
admin, err := sql.Open("mysql", cfg.FormatDSN())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
name := fmt.Sprintf("sw_test_cleanup_%d", time.Now().UnixNano())
|
||||
if _, err := admin.Exec("CREATE DATABASE `" + name + "` CHARACTER SET utf8mb4"); err != nil {
|
||||
admin.Close()
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
_, err := admin.Exec("DROP DATABASE `" + name + "`")
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
admin.Close()
|
||||
})
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
|
||||
defer cancel()
|
||||
conn, err := admin.Conn(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer conn.Close()
|
||||
exec := func(query string, args ...any) {
|
||||
t.Helper()
|
||||
if _, err := conn.ExecContext(ctx, query, args...); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
exec("USE `" + name + "`")
|
||||
source := cleanupSQL(t)
|
||||
targets := cleanupManifest(t, source, "_sw_cleanup_targets_20261106")
|
||||
required := cleanupManifest(t, source, "_sw_cleanup_required_tables_20261106")
|
||||
versions := cleanupManifest(t, source, "_sw_cleanup_required_versions_20261106")
|
||||
for _, table := range append(targets, required...) {
|
||||
switch table {
|
||||
case "sw_kra_state", "sw_kra_checks", "sys_schema_migrations", "sys_business_upgrade_state", "sys_authorities":
|
||||
continue
|
||||
}
|
||||
exec("CREATE TABLE `" + table + "` (id BIGINT PRIMARY KEY) ENGINE=InnoDB")
|
||||
exec("INSERT INTO `" + table + "` VALUES (1)")
|
||||
}
|
||||
exec("CREATE TABLE sys_authorities(authority_id BIGINT PRIMARY KEY,business_level TINYINT UNSIGNED,deleted_at DATETIME,UNIQUE KEY uk_authority_business_level(business_level))")
|
||||
exec("INSERT INTO sys_authorities VALUES(101,1,NULL),(102,2,NULL),(103,3,NULL)")
|
||||
exec("CREATE TABLE sys_business_upgrade_state(name VARCHAR(64) PRIMARY KEY,version VARCHAR(64),phase VARCHAR(32),verified_at DATETIME,verified_migrations VARCHAR(64),cleanup_backup_path TEXT,cleanup_sha256 VARCHAR(64))")
|
||||
exec("INSERT INTO sys_business_upgrade_state VALUES('legacy-sw','20261106_automatic_v1','cleanup',NOW(),REPEAT('a',64),'verified-backup.sql.gz',REPEAT('b',64))")
|
||||
exec("CREATE TABLE sw_kra_state(id INT PRIMARY KEY,version VARCHAR(80),phase VARCHAR(40),completed_at DATETIME)")
|
||||
exec("INSERT INTO sw_kra_state VALUES(1,'20261103_framework_migration_v5','completed',NOW())")
|
||||
exec("CREATE TABLE sw_kra_checks(stage VARCHAR(32),check_name VARCHAR(160),expected_value BIGINT,actual_value BIGINT,passed TINYINT)")
|
||||
for stage, count := range map[string]int{"backup": 52, "migration": 24} {
|
||||
for i := 0; i < count; i++ {
|
||||
exec("INSERT INTO sw_kra_checks VALUES(?,?,?,?,?)", stage, fmt.Sprintf("check_%d", i), 1, 1, 1)
|
||||
}
|
||||
}
|
||||
exec("CREATE TABLE sys_schema_migrations(id VARCHAR(255) PRIMARY KEY)")
|
||||
for _, id := range versions {
|
||||
exec("INSERT INTO sys_schema_migrations VALUES(?)", id)
|
||||
}
|
||||
exec("CREATE TABLE bak_sw2kra_20261103_not_owned(id INT)")
|
||||
count := func(query string, args ...any) int {
|
||||
t.Helper()
|
||||
var n int
|
||||
if err := conn.QueryRowContext(ctx, query, args...).Scan(&n); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return n
|
||||
}
|
||||
has := func(table string) bool {
|
||||
return count("SELECT COUNT(*) FROM information_schema.tables WHERE table_schema=? AND table_name=?", name, table) == 1
|
||||
}
|
||||
assertTargets := func() {
|
||||
t.Helper()
|
||||
for _, table := range targets {
|
||||
if !has(table) {
|
||||
t.Fatalf("table deleted before all checks passed: %s", table)
|
||||
}
|
||||
}
|
||||
}
|
||||
fails := func(want string) {
|
||||
t.Helper()
|
||||
err := runCleanupSQL(ctx, conn, source, name, true)
|
||||
if err == nil || !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("want rejection %q, got %v", want, err)
|
||||
}
|
||||
assertTargets()
|
||||
}
|
||||
if err := runCleanupSQL(ctx, conn, source, name, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assertTargets()
|
||||
exec("UPDATE sw_kra_state SET phase='data_validated'")
|
||||
fails("has not completed")
|
||||
exec("UPDATE sw_kra_state SET phase='completed'")
|
||||
exec("UPDATE sw_kra_checks SET passed=0 WHERE stage='backup' AND check_name='check_0'")
|
||||
fails("verification records")
|
||||
exec("UPDATE sw_kra_checks SET passed=1")
|
||||
const attachments = "202611030007_business_alert_attachments"
|
||||
exec("DELETE FROM sys_schema_migrations WHERE id=?", attachments)
|
||||
fails("incremental migrations")
|
||||
exec("INSERT INTO sys_schema_migrations VALUES(?)", attachments)
|
||||
exec("RENAME TABLE sys_alert_rule TO user_saved_rule")
|
||||
fails("Required current table")
|
||||
exec("RENAME TABLE user_saved_rule TO sys_alert_rule")
|
||||
exec("CREATE TABLE sys_user(id INT)")
|
||||
fails("Unarchived legacy")
|
||||
exec("DROP TABLE sys_user")
|
||||
exec("CREATE TABLE retained_reference(id BIGINT,FOREIGN KEY(id) REFERENCES legacy_sw_20261103_sys_user(id))")
|
||||
fails("Foreign keys")
|
||||
exec("DROP TABLE retained_reference")
|
||||
exec("CREATE VIEW retained_view AS SELECT * FROM legacy_sw_20261103_sys_user")
|
||||
fails("views referencing")
|
||||
exec("DROP VIEW retained_view")
|
||||
exec("CREATE TRIGGER retained_trigger AFTER INSERT ON sys_users FOR EACH ROW INSERT INTO sj_job(id) VALUES(NEW.id)")
|
||||
fails("triggers referencing")
|
||||
exec("DROP TRIGGER retained_trigger")
|
||||
exec("CREATE PROCEDURE retained_routine() SELECT COUNT(*) FROM sj_job")
|
||||
fails("routines referencing")
|
||||
exec("DROP PROCEDURE retained_routine")
|
||||
exec("CREATE EVENT retained_event ON SCHEDULE EVERY 1 YEAR STARTS CURRENT_TIMESTAMP + INTERVAL 1 YEAR DISABLE DO INSERT INTO sj_job(id) VALUES(2)")
|
||||
fails("events referencing")
|
||||
exec("DROP EVENT retained_event")
|
||||
exec("SET SESSION foreign_key_checks=0")
|
||||
fails("foreign_key_checks")
|
||||
exec("SET SESSION foreign_key_checks=1")
|
||||
other, err := admin.Conn(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var lock string
|
||||
if err := conn.QueryRowContext(ctx, "SELECT CONCAT('sw_to_kra:',MD5(DATABASE()))").Scan(&lock); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var locked int
|
||||
if err := other.QueryRowContext(ctx, "SELECT GET_LOCK(?,0)", lock).Scan(&locked); err != nil || locked != 1 {
|
||||
t.Fatalf("lock: %d %v", locked, err)
|
||||
}
|
||||
fails("already running")
|
||||
if _, err := other.ExecContext(ctx, "DO RELEASE_LOCK(?)", lock); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
other.Close()
|
||||
oldTimeout := count("SELECT @@SESSION.lock_wait_timeout")
|
||||
if err := runCleanupSQL(ctx, conn, source, name, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, table := range targets {
|
||||
if has(table) {
|
||||
t.Fatalf("old table retained: %s", table)
|
||||
}
|
||||
}
|
||||
for _, table := range required {
|
||||
if !has(table) {
|
||||
t.Fatalf("current table deleted: %s", table)
|
||||
}
|
||||
}
|
||||
if !has("bak_sw2kra_20261103_not_owned") || count("SELECT COUNT(*) FROM sys_business_upgrade_state") != 1 || count("SELECT COUNT(*) FROM sys_users") != 1 || count("SELECT COUNT(*) FROM tab_user") != 1 {
|
||||
t.Fatal("non-target data changed")
|
||||
}
|
||||
if count("SELECT @@SESSION.lock_wait_timeout") != oldTimeout {
|
||||
t.Fatal("session lock timeout not restored")
|
||||
}
|
||||
if err := runCleanupSQL(ctx, conn, source, name, true); err != nil {
|
||||
t.Fatalf("retry after deletion: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
-- SW → Kra 旧框架转换:完整迁移必须继续执行当前后端的 -migrate 和 -check-schema。
|
||||
-- SW → Kra 旧框架转换:由程序首次连接旧库时自动执行,随后自动完成增量迁移、校验和清理。
|
||||
-- 服务设置 admin.system.disable_auto_migrate: true;目标库由下方 USE 配置。
|
||||
-- 新副本:备份源表、创建 Kra 表、转换框架数据及关联,并归档旧表。
|
||||
-- 已迁移副本:自动执行菜单调整,不重新导入账号,不重建已存在的表。
|
||||
@@ -42,7 +42,7 @@ IF COALESCE(v_lock,0)<>1 THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Migratio
|
||||
-- 一、已迁移库:确认目标表完整、迁移账本已完成,再调整菜单。
|
||||
IF EXISTS(SELECT 1 FROM information_schema.tables WHERE table_schema=DATABASE() AND table_name='sys_users') THEN
|
||||
IF (SELECT COUNT(*) FROM information_schema.tables WHERE table_schema=DATABASE()
|
||||
AND table_name IN ('sys_schema_migrations','sys_users','sys_authorities','sys_base_menus','sys_user_authority','sys_authority_menus','sys_base_menu_parameters','sys_apis','sys_ignore_apis','sys_authority_apis','casbin_rule','sys_base_menu_btns','sys_authority_btns','sys_departments','sys_positions','sys_user_departments','sys_user_positions','sys_authority_departments','sys_dictionaries','sys_dictionary_details','sys_params','sys_api_tokens','jwt_blacklists','sys_security_config','sys_versions','sys_export_templates','sys_export_template_condition','sys_export_template_join','sys_operation_records','sys_login_logs','sys_data_access_logs','sys_error','media_file_upload_and_downloads','media_attachment_category','media_uploads','media_upload_chunks','sys_announcements','sys_integration_configs','sys_timed_tasks','sys_timed_task_logs','pay_orders','pay_events','sys_user_bind_platform','sys_user_preference','sys_alert_rule','sys_alert_record','sys_alert_operation_log','sys_message','sw_kra_id_map') AND engine='InnoDB')<>49 THEN
|
||||
AND table_name IN ('sys_schema_migrations','sys_users','sys_authorities','sys_base_menus','sys_user_authority','sys_authority_menus','sys_base_menu_parameters','sys_apis','sys_ignore_apis','sys_authority_apis','casbin_rule','sys_base_menu_btns','sys_authority_btns','sys_departments','sys_positions','sys_user_departments','sys_user_positions','sys_authority_departments','sys_dictionaries','sys_dictionary_details','sys_params','sys_api_tokens','jwt_blacklists','sys_security_config','sys_versions','sys_export_templates','sys_export_template_condition','sys_export_template_join','sys_operation_records','sys_login_logs','sys_data_access_logs','sys_error','media_file_upload_and_downloads','media_attachment_category','media_uploads','media_upload_chunks','sys_announcements','sys_integration_configs','sys_timed_tasks','sys_timed_task_logs','pay_orders','pay_events','sys_user_bind_platform','sys_user_preference','sys_alert_rule','sys_alert_record','sys_alert_operation_log','sys_message') AND engine='InnoDB')<>48 THEN
|
||||
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='Kra target tables incomplete; refusing partial migration';
|
||||
END IF;
|
||||
IF EXISTS(SELECT 1 FROM information_schema.tables WHERE table_schema=DATABASE() AND table_name='sw_kra_state') THEN
|
||||
@@ -79,8 +79,10 @@ IF EXISTS(SELECT 1 FROM information_schema.tables WHERE table_schema=DATABASE()
|
||||
WHERE x.sys_menu_id=v_workbench OR b.sys_base_menu_id=v_workbench;
|
||||
DELETE FROM sys_authority_menus WHERE sys_base_menu_id=v_workbench;
|
||||
DELETE FROM sys_base_menu_parameters WHERE sys_base_menu_id=v_workbench;
|
||||
IF EXISTS(SELECT 1 FROM information_schema.tables WHERE table_schema=DATABASE() AND table_name='sw_kra_id_map') THEN
|
||||
DELETE m FROM sw_kra_id_map m LEFT JOIN sys_base_menu_btns b ON b.id=m.new_id
|
||||
WHERE m.entity='menu' AND (m.new_id=v_workbench OR b.sys_base_menu_id=v_workbench);
|
||||
END IF;
|
||||
DELETE FROM sys_base_menu_btns WHERE sys_base_menu_id=v_workbench;
|
||||
DELETE FROM sys_base_menus WHERE id=v_workbench;
|
||||
|
||||
@@ -689,6 +691,7 @@ INSERT INTO sw_kra_checks(stage,check_name,expected_value,actual_value,passed) V
|
||||
IF v_expected <> v_actual THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='sys_alert_operation_log backup bytes'; END IF;
|
||||
|
||||
COMMIT;
|
||||
SELECT '旧框架备份已校验' AS result;
|
||||
|
||||
-- 四、按 Kra 当前定义创建目标表,并增加商务账号和岗位扩展字段。
|
||||
|
||||
@@ -1731,6 +1734,7 @@ IF v_expected <> v_actual THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='positio
|
||||
UPDATE sw_kra_state SET phase='data_validated' WHERE id=1;
|
||||
|
||||
COMMIT;
|
||||
SELECT '框架数据已验证' AS result;
|
||||
|
||||
-- 十、数据核验通过后归档旧框架表,登记完成状态并释放迁移锁。
|
||||
|
||||
|
||||
Reference in New Issue
Block a user