chore: 移除剩余迁移验证脚本并说明汇总任务依赖

This commit is contained in:
Yvan
2026-11-04 14:49:34 +08:00
parent 297e80827d
commit 7764b03906
20 changed files with 40 additions and 1640 deletions
+1 -1
View File
@@ -50,6 +50,6 @@
## 文档与脚本清理
104 份阶段报告、重复 JSON、旧目录审查和独立菜单 SQL,以及 26 个旧脚本已逐文件复制到仓库外并校验 SHA-256 后移除。保留统一迁移 SQL、业务口径、综合验收索引和 14 个当前验证工具,补充维护入口与脚本说明。
首次清理将 104 份阶段报告、重复 JSON、旧目录审查和独立菜单 SQL,以及 26 个旧脚本移出项目。用户追加要求后,剩余 14 个验证工具、脚本说明和 Python 缓存也全部移出,`scripts` 目录已删除。所有移除文件均在仓库外归档并校验 SHA-256;统一迁移 SQL、业务口径与综合验收索引保留。
真实本地配置 `configs/config.yaml`、私有运行数据,以及其他会话的 V2 估算和 outputs 成果不纳入本次提交。未推送远端。
+1 -1
View File
@@ -4,6 +4,6 @@
- [当前接口审查](API_REVIEW.md):此次修复、验证范围及保留的技术风险。
- [后台说明](ADMIN_BACKEND.md)、[支付接入](PAYMENT.md)、[短信接入](SMS_INTEGRATION.md)。
- [OA 迁移计划](OA_MIGRATION_PLAN.md):独立 OA 工作的设计材料,不是当前商务迁移的完成证明。
- [验证脚本](../scripts/README.md):保留工具的用途和私有输入约定。
- [汇总任务说明](sw6-to-kra/report-build-source-notes.md):任务读写表及接口依赖;临时验证脚本已移出项目。
历史阶段记录已归档到仓库外;后续将最新结果更新到对应维护文档,不再每个接口生成一份日期报告。其他会话生成的商务 V2 估算及表格保持原状,本次清理和提交不包含这些文件。
+2 -2
View File
@@ -8,7 +8,7 @@
2. 在 [统一迁移 SQL](sw_to_kra_migrate.sql) 开头修改 `USE`,当前为 `USE sw9`。用支持 `DELIMITER` 的客户端在同一连接执行,遇错停止。
3. 启动当前 Go 版本,执行版本迁移,补齐业务锁、密码重置表、附件引用和查询索引。大表首次建索引需要等待完成。
4. 配置主库和 `lx`、`hw`、`yq`、`oa` 数据源。配置和凭据保存在仓库外;数据库连接信息不写入本目录。
5. 对照 [验收记录](validation.md) 与 [验证脚本说明](../../scripts/README.md) 复核接口和任务。迁入计划默认停用,启用前应确认业务所需运行时段。
5. 对照 [验收记录](validation.md) 与 [汇总任务说明](report-build-source-notes.md) 复核接口和任务。迁入计划默认停用,启用前应确认业务所需运行时段。
SQL 自动区分首次迁移、已完成迁移的菜单调整和异常中间状态;不会对完成迁移的库重新导入账号。重复执行及旧表已归档/清理的路径均有演练记录。
@@ -25,4 +25,4 @@ SQL 自动区分首次迁移、已完成迁移的菜单调整和异常中间状
SQL 生成器位于旧仓库 `script/sql/sw6_to_kra/build_sw7_migration.py`。修改生成器后同步本目录统一 SQL,并重新核对散列及迁移结果;不要恢复分阶段 SQL 补丁。
阶段报告、重复 JSON、旧 sw7 冒烟脚本已归档到本机仓库外 `D:/Users/Yvan/backups/sw9/20261104_rehearsal/cleanup-archive/`。该目录以及相邻的运行配置、原始响应、分区备份不纳入 Git。历史索引中 `reports` 项的文件名可在归档内查找。
阶段报告、重复 JSON、全部临时验证脚本已归档到本机仓库外 `D:/Users/Yvan/backups/sw9/20261104_rehearsal/cleanup-archive/`,最后保留的工具位于其中的 `scripts-final/`。项目不再保留 `scripts` 目录,Go 任务运行不依赖这些脚本。归档及相邻的运行配置、原始响应、分区备份不纳入 Git。历史索引中 `reports` 项的文件名可在归档内查找。
@@ -25,3 +25,38 @@
领域与存储测试覆盖身份/账号规则、重复构建、自然日筛选和身份明细写失败后的原子回滚。真实 TCP HTTP 使用隔离数据库和 Kra 原生任务创建、手动触发、执行日志,核对两种报表实际落库且任务仍为禁用。
当前验证状态统一见 [验收与性能记录](validation.md),不再维护阶段性运行报告。
## 任务、表与接口的对应关系
这些任务是 Go 内部注册的方法,与已删除的临时验证脚本无关。它们从当前主库读取已同步的原始数据,再生成统计结果;不负责将 SDK 平台数据同步进主库。当前运行服务主库为 sw9。
| 任务方法 | 日期范围 | 写入的统计表 |
|---|---|---|
| swReportDayTodayTask | 上海时区今天 | tab_sw_report_day、tab_sw_report_day_key |
| swReportDayYesterdayTask | 上海时区昨天 | 同上 |
| swReportDayBackfillTask | startDate 至 endDate,含两端 | 同上 |
| swReportServerDayTodayTask | 上海时区今天 | tab_sw_report_server_day、tab_sw_report_server_day_key |
| swReportServerDayYesterdayTask | 上海时区昨天 | 同上 |
| swReportServerDayBackfillTask | startDate 至 endDate,含两端 | 同上 |
商务日报读取 `tab_user`(主账号注册及身份资料)、`tab_user_day_login`(登录)、`tab_spend`(成功充值)、`tab_game`(游戏名)。区服日报读取 `tab_user_play_info`(创角 create_time、活跃 play_time)、`tab_spend`、`tab_user`、`tab_game`。
`tab_sw_report_day` 按日期、平台、渠道、游戏、去重模式保存注册数、活跃人数、付费人数、新增付费人数和充值金额。`tab_sw_report_server_day` 在相同维度上增加区服,注册数代表创角玩家数,不包含新增付费人数。
两个 `_key` 表保存参与统计的身份键,供跨日期、渠道或游戏聚合时重新 `COUNT(DISTINCT identity_key)`。例如同一玩家连续两天活跃,查询这两天的独立活跃人数应为 1,不能直接将两个日活人数相加。`dedup_mode=0` 使用账号 ID,`dedup_mode=1` 使用规范化身份证;空身份证不计入模式 1。`metric` 为注册 1、活跃 2、付费 3,商务日报另有新增付费 4;金额查询只读模式 0,避免两种模式重复累加。
每次重建会在一个事务内锁定“报表类型+日期”,读取原始数据,删除统计表及对应 key 表该 `stat_date` 的旧行,批量写入新结果。读取或写入失败、取消时回滚;每个日期单独提交,回跑中途失败不会撤销此前已完成日期。除统计表外,会使用 `sys_business_report_build_locks` 保存互斥锁键,任务执行框架将结果记入 `sys_timed_task_logs`。原始用户、充值和登录记录不会被这些任务改写。
| 使用方 | 接口 | 依赖 |
|---|---|---|
| 渠道数据统计 | GET /sw/channelData/list | 商务日报两表 |
| 游戏数据 | GET /sw/gameData/list | 商务日报两表 |
| 三类排行榜 | GET /sw/rank/channel、/business、/game | 商务日报两表 |
| 仪表盘/工作台 | GET /sw/workbench/summary、/registerTop、/payRank | 商务日报两表;汇总中的新增渠道另查渠道表 |
| 商务画像 | GET /sw/business/:userId/portrait、/sw/business/bindPlatform/portrait | 商务日报两表 |
| 渠道画像及游戏分布 | GET /sw/promote/portrait、/sw/promote/game-distribution | 商务日报两表;付费分层还会读取原始充值 |
| 区服分析 | GET /sw/serverAnalysis/list | 区服日报两表 |
普通渠道列表、注册明细、充值明细、渠道激活、工作台未登录榜,以及游戏/区服下拉选项不直接使用这四张汇总表。预警检测 `alertTriggerTask` 另行读取原始注册/充值数据,不由这些汇总任务生成预警。
今天任务用于当天持续刷新,昨天任务用于补算昨日完整数据,backfill 用于补历史或纠正迟到数据。2026-11-04 核实时六条计划全部停用:不自动刷新汇总,但已有历史结果仍可查询;若原始数据继续同步,依赖汇总的指标会与明细逐渐产生时间差。
+1 -1
View File
@@ -65,4 +65,4 @@
## 页面
渠道激活页不再在发起分页请求时清零 total;排行榜与仪表盘四榜并排、付费率用百分比、头部两项偏好直接勾选;三级商务保留商务列表/绑定列表切换。完整页面检查脚本仍保留,运行截图和带业务数据的响应仅存仓库外。
渠道激活页不再在发起分页请求时清零 total;排行榜与仪表盘四榜并排、付费率用百分比、头部两项偏好直接勾选;三级商务保留商务列表/绑定列表切换。页面检查脚本、运行截图和带业务数据的响应均已保存在仓库外。
-49
View File
@@ -1,49 +0,0 @@
# 商务验证工具
仅保留仍用于当前 sw8/sw9 验证的工具。旧 sw7、固定端口 18000、重复页面夹具脚本已归档;接口边界回归优先运行相应 Go 测试。
## 环境
- Python 3.10+;数据库工具需要 PyMySQL、PyYAML,Excel 对比需要 openpyxl。
- 页面检查使用 Node 和 Playwright;通过 `SW_PLAYWRIGHT_MODULE` 指定已有模块路径。
- `--runtime` 必须指向仓库外的私有目录。配置、令牌、实际响应、导出文件和 SQL 日志不得提交。
- 旧 Java 连接 sw8,新 Go 连接 sw9。运行命令时明确传入端口,避免误连日常服务。任务保持停用。
通用输入:`legacy-<identity>.token`、`new-<identity>.token`、`legacy-client.json`、`id-maps.json`;写入测试还需要专门创建的三级测试身份及对应凭据。不要把业务正常账号当作可随意清理的测试身份。
## 工具用途
| 工具 | 用途与写入范围 |
|---|---|
| start-sw-rehearsal.ps1 | 从私有配置启动本机 Go 服务,临时注入 SDK 兼容参数,日志留在 runtime |
| sw_database_rehearsal.py | 数据库盘点、备份、迁移及验证;同时提供其他脚本依赖的 connect/query |
| sw_api_compare.py | 新旧读取接口逐字段对比;输入 case JSON 明确路径、参数、允许的协议转换 |
| sw_write_verify.py | 规则与预警状态流程测试;提供其他写入工具的 Rehearsal 支持 |
| sw_account_create_verify.py | 创建临时下级、验证层级/角色并清理;不删除预先存在的同名账号 |
| sw_account_write_verify.py | 测试账号资料/状态并恢复原值 |
| sw_preference_compare.py | 四种包含下级/身份证去重组合,结束恢复偏好 |
| sw_export_compare.py | 新旧注册和充值 Excel 逐单元格对比 |
| sw_sdk_write_verify.py | 独立平台夹具上的绑定、重提、解绑与渠道创建;受控 OA 审核 |
| sw_report_rehearsal.py | 指定日报分区备份、比较和恢复 |
| sw_run_report_tasks.py | 要求已有日期匹配的备份;创建停用的临时计划、手工执行并清理计划 |
| sw_endpoint_benchmark.py | 接口耗时、完整响应摘要及 SQL 采集,只读 |
| sw_endpoint_parity.py | 优化前后接口及偏好矩阵对比;即使 HTTP 恢复失败也尝试还原数据库偏好 |
| sw_dashboard_layout_smoke.cjs | 激活分页、偏好和榜单布局检查,截图留在指定私有目录 |
先通过各 Python 工具的 `--help` 查看必需参数。日报、预警及写入工具会改动测试数据;使用其备份/恢复流程,并核对最终表计数与任务状态。
## 读取对比示例
```powershell
python scripts/sw_api_compare.py --runtime D:/private/sw-review --cases D:/private/sw-review/cases.json --old-url http://127.0.0.1:18080 --new-url http://127.0.0.1:19004 --output D:/private/sw-review/result.json
```
`--recompare` 只比较已保存响应;`--refresh-target` 使用已保存的同参数旧响应、重新请求新服务。报告会标记 `source_capture_reused`,不能把快照对比描述为双方重新实测。
本地 MySQL 回归只接受 `SW_TEST_MYSQL_DSN` 指向 `127.0.0.1` 的隔离实例:
```powershell
go test ./internal/data/business -run MySQL -count=1
```
这些测试创建并清理自己的随机数据库,不在 sw8/sw9 上运行测试建表和删除。
-37
View File
@@ -1,37 +0,0 @@
param(
[Parameter(Mandatory = $true)][string]$RuntimePath,
[string]$ConfigName = 'config-next.yaml',
[string]$BinaryPath = '.cache/sw89/sw-new-complete.exe'
)
$ErrorActionPreference = 'Stop'
$workspaceRoot = (Resolve-Path (Join-Path $PSScriptRoot '..')).Path
$runtimeRoot = (Resolve-Path -LiteralPath $RuntimePath).Path
$configPath = (Resolve-Path -LiteralPath (Join-Path $runtimeRoot $ConfigName)).Path
if (-not [System.IO.Path]::IsPathRooted($BinaryPath)) {
$BinaryPath = Join-Path $workspaceRoot $BinaryPath
}
$binary = (Resolve-Path -LiteralPath $BinaryPath).Path
$compat = Get-Content -LiteralPath (Join-Path $runtimeRoot 'sdk-compatibility.json') -Raw | ConvertFrom-Json
$previous = @{}
$settings = @{
SW_SDK_AUTH_CODE = $compat.AUTH_CODE
SW_SDK_MIGRATE_AUTH_CODE = $compat.MIGRATE_AUTH_CODE
SW_SDK_TABLE_PREFIX = $compat.TABLE_PREFIX
}
try {
foreach ($name in $settings.Keys) {
$previous[$name] = [Environment]::GetEnvironmentVariable($name, 'Process')
[Environment]::SetEnvironmentVariable($name, $settings[$name], 'Process')
}
$process = Start-Process -FilePath $binary -ArgumentList @('-conf', ('"' + $configPath + '"')) `
-WorkingDirectory $workspaceRoot -WindowStyle Hidden `
-RedirectStandardOutput (Join-Path $runtimeRoot 'rehearsal.stdout.log') `
-RedirectStandardError (Join-Path $runtimeRoot 'rehearsal.stderr.log') -PassThru
$process.Id | Set-Content -LiteralPath (Join-Path $runtimeRoot 'rehearsal.pid')
Write-Output "Rehearsal process: $($process.Id)"
} finally {
foreach ($name in $previous.Keys) {
[Environment]::SetEnvironmentVariable($name, $previous[$name], 'Process')
}
}
-73
View File
@@ -1,73 +0,0 @@
"""在 sw8/sw9 用专用二级账号创建临时三级账号,核对层级和角色后清理。"""
import argparse
from pathlib import Path
from sw_write_verify import Rehearsal
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--config", type=Path, required=True)
parser.add_argument("--old-url", default="http://127.0.0.1:18080")
parser.add_argument("--new-url", default="http://127.0.0.1:19002")
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
if args.runtime.resolve().is_relative_to(Path(__file__).resolve().parents[1]):
raise ValueError("测试凭据必须位于仓库外")
run = Rehearsal(args)
username = "mig_recheck_" + run.suffix
password = (args.runtime / "synthetic-test-password.txt").read_text(encoding="utf-8").strip()
source_roles = run.query("SELECT role_id FROM sw8.sys_user_role WHERE user_id=%s", (run.owner("legacy", 3),))
expected_roles = run.query("SELECT sys_authority_authority_id id FROM sw9.sys_user_authority WHERE sys_user_id=%s", (run.owner("new", 3),))
created_sides = set()
try:
run.check("测试角色存在", len(source_roles) == 1 and len(expected_roles) == 1)
for side, database, table, column in (("legacy", "sw8", "sys_user", "user_name"), ("new", "sw9", "sys_users", "username")):
run.check("临时账号不存在 " + side, not run.query(f"SELECT 1 FROM {database}.{table} WHERE {column}=%s", (username,)))
payload = {"userName": username, "password": password, "nickName": "迁移复核临时账号", "realName": "迁移复核临时账号"}
run.call("legacy", "testlevel2", "POST", "/system/user", {**payload, "roleIds": [str(source_roles[0]["role_id"])]})
created_sides.add("sw8")
run.call("new", "testlevel2", "POST", "/sw/business", {**payload, "parentId": 0, "level": 1, "authorityId": 888})
created_sides.add("sw9")
old = run.query("SELECT user_id,level,parent_id,id_path,status FROM sw8.sys_user WHERE user_name=%s", (username,))[0]
new = run.query("SELECT id,level,parent_id,id_path,business_status,authority_id,enable FROM sw9.sys_users WHERE username=%s", (username,))[0]
run.check("新旧创建三级且挂靠当前二级", old["level"] == new["level"] == 3 and str(old["parent_id"]) == str(run.owner("legacy", 2)) and new["parent_id"] == run.owner("new", 2))
path = ",".join(run.mapping.get(part, part) for part in str(old["id_path"]).split(","))
run.check("祖先路径按用户映射一致", path == new["id_path"])
run.check("默认状态正常且角色不能由请求提权", str(old["status"]) == str(new["business_status"]) == "0" and new["enable"] == 1 and new["authority_id"] == expected_roles[0]["id"] and new["authority_id"] != 888)
old_roles = run.query("SELECT role_id FROM sw8.sys_user_role WHERE user_id=%s", (old["user_id"],))
new_roles = run.query("SELECT sys_authority_authority_id id FROM sw9.sys_user_authority WHERE sys_user_id=%s", (new["id"],))
run.check("关联角色正确", old_roles == source_roles and new_roles == expected_roles)
values = []
for database, user_id in (("sw8", old["user_id"]), ("sw9", new["id"])):
values.append(run.query(f"SELECT include_sub_promote,idcard_dedup FROM {database}.sys_user_preference WHERE user_id=%s", (user_id,)))
run.check("默认统计偏好一致", values[0] == values[1] == [{"include_sub_promote": 1, "idcard_dedup": 1}])
finally:
# 只清理本次唯一名称创建的临时账号及其角色/偏好,不触碰原有测试链。
for database, table, id_col, name_col in (("sw8", "sys_user", "user_id", "user_name"), ("sw9", "sys_users", "id", "username")):
if database not in created_sides:
continue
rows = run.query(f"SELECT {id_col} id FROM {database}.{table} WHERE {name_col}=%s", (username,))
for row in rows:
user_id = row["id"]
run.db.begin()
try:
run.query(f"DELETE FROM {database}.sys_user_preference WHERE user_id=%s", (user_id,))
if database == "sw8":
for relation in ("sys_user_role", "sys_user_post"):
run.query(f"DELETE FROM sw8.{relation} WHERE user_id=%s", (user_id,))
else:
run.query("DELETE FROM sw9.sys_user_authority WHERE sys_user_id=%s", (user_id,))
run.query(f"DELETE FROM {database}.{table} WHERE {id_col}=%s AND {name_col}=%s", (user_id, username))
run.db.commit()
except Exception:
run.db.rollback()
raise
run.check("临时账号清理完成 " + database, not run.query(f"SELECT 1 FROM {database}.{table} WHERE {name_col}=%s", (username,)))
run.db.close()
if __name__ == "__main__":
main()
-75
View File
@@ -1,75 +0,0 @@
"""核对专用二级测试账号的资料更新、四种商务状态及恢复,不修改真实商务账号。"""
import argparse
import json
import time
from pathlib import Path
from sw_write_verify import Rehearsal
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--config", type=Path, default=Path("configs/config.yaml"))
parser.add_argument("--old-url", default="http://127.0.0.1:18080")
parser.add_argument("--new-url", default="http://127.0.0.1:19001")
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
run = Rehearsal(args)
account = run.users[1]
if not account["username"].startswith("mig_verify_"):
raise ValueError("仅允许修改本轮创建的专用测试账号")
originals = {
"legacy": run.query("SELECT real_name,status,reject_reason,update_time FROM sw8.sys_user WHERE user_id=%s AND user_name=%s", (account["source_id"], account["username"]))[0],
"new": run.query("SELECT real_name,business_status,enable,reject_reason,updated_at FROM sw9.sys_users WHERE id=%s AND username=%s", (account["target_id"], account["username"]))[0],
}
(args.runtime / "test-account-before-write.json").write_text(json.dumps(originals, default=str, indent=2), encoding="utf-8")
def update(side, status, name, reason):
payload = {"userId": account["source_id" if side == "legacy" else "target_id"], "userName": account["username"], "realName": name,
"status": str(status), "rejectReason": reason or "", "roleIds": ["2087435436573081601"]}
return run.call(side, "admin", "PUT", "/system/user" if side == "legacy" else "/sw/business", payload)
try:
started = time.monotonic()
for index, status in enumerate((0, 3, 2, 1, 0)):
if index == 4:
# 旧控制器对相同请求体有五秒重复提交限制。
time.sleep(max(0, 5.2 - (time.monotonic() - started)))
name = "迁移核对二级资料更新"
reason = "迁移核对驳回" if status == 2 else ""
for side in ("legacy", "new"):
update(side, status, name, reason)
path = "/system/user/" + str(account["source_id"]) if side == "legacy" else "/sw/business/" + str(account["target_id"])
detail = run.call(side, "admin", "GET", path)["data"]
if side == "legacy":
detail = detail["user"]
run.check(f"资料及状态 {status} {side}", detail["realName"] == name and str(detail["status"]) == str(status) and detail["level"] == 2)
enabled = run.query("SELECT enable FROM sw9.sys_users WHERE id=%s", (account["target_id"],))[0]["enable"]
run.check(f"商务状态 {status} 与启停一致", enabled == (1 if status == 0 else 0))
replies = {side: run.call(side, "testlevel2", "GET", run.prefix(side) + "/user/preference", success=False) for side in ("legacy", "new")}
if status != 0:
run.check(f"状态 {status} 拒绝旧会话", replies["new"]["code"] != 0,
legacy_session_allowed=replies["legacy"]["code"] == 200, accepted_change="新系统每次请求核对商务状态,阻止停用/待审/驳回账号沿用旧会话")
else:
run.check("正常账号会话可用", replies["new"]["code"] == 0 and replies["legacy"]["code"] == 200)
finally:
for side in ("legacy", "new"):
original = originals[side]
update(side, original["status" if side == "legacy" else "business_status"], original["real_name"], original["reject_reason"])
old, new = originals["legacy"], originals["new"]
run.query("UPDATE sw8.sys_user SET real_name=%s,status=%s,reject_reason=%s,update_time=%s WHERE user_id=%s AND user_name=%s",
(old["real_name"], old["status"], old["reject_reason"], old["update_time"], account["source_id"], account["username"]))
run.query("UPDATE sw9.sys_users SET real_name=%s,business_status=%s,enable=%s,reject_reason=%s,updated_at=%s WHERE id=%s AND username=%s",
(new["real_name"], new["business_status"], new["enable"], new["reject_reason"], new["updated_at"], account["target_id"], account["username"]))
actual = {
"legacy": run.query("SELECT real_name,status,reject_reason,update_time FROM sw8.sys_user WHERE user_id=%s", (account["source_id"],))[0],
"new": run.query("SELECT real_name,business_status,enable,reject_reason,updated_at FROM sw9.sys_users WHERE id=%s", (account["target_id"],))[0],
}
run.check("测试账号原始资料和状态已恢复", actual == originals)
run.db.close()
if __name__ == "__main__":
main()
-201
View File
@@ -1,201 +0,0 @@
"""请求隔离的新旧商务实例;原始响应和令牌留在仓库外,报告只记录校验结果。"""
import argparse
import concurrent.futures
import decimal
import hashlib
import json
import re
import time
import urllib.error
import urllib.parse
import urllib.request
from pathlib import Path
def fetch(base, token, path, params, legacy, client_id="", timeout=120):
url = base.rstrip("/") + path
if params:
url += "?" + urllib.parse.urlencode(params, doseq=True)
headers = {"Authorization": "Bearer " + token} if legacy else {"x-token": token}
if legacy:
headers["clientid"] = client_id
request = urllib.request.Request(url, headers=headers)
started = time.monotonic()
try:
with urllib.request.urlopen(request, timeout=timeout) as response:
body = json.load(response)
return {"http": response.status, "elapsed_ms": round((time.monotonic() - started) * 1000), "body": body}
except (urllib.error.URLError, TimeoutError, ValueError) as error:
return {"http": None, "elapsed_ms": round((time.monotonic() - started) * 1000), "error": type(error).__name__}
def data_of(reply):
body = reply.get("body", {})
if "rows" in body:
return {key: value for key, value in body.items() if key not in ("code", "msg")}
return body.get("data")
def shape(value):
if isinstance(value, dict):
return {key: (len(item) if isinstance(item, list) else type(item).__name__) for key, item in value.items()}
return {"type": type(value).__name__, "length": len(value) if isinstance(value, list) else None}
def expected_relative_pay_rate(row):
"""用户确认的新口径:由旧的本期付费率及百分点差推导相对环比。"""
if row.get("momValue") is None:
return row
current = decimal.Decimal(str(row["metricValue"]))
difference = decimal.Decimal(str(row["momValue"]))
previous = current - difference
result = dict(row)
if previous <= 0:
result.update(momValue=None, momStatus=None)
else:
change = (difference / previous * 100).quantize(decimal.Decimal("0.01"), rounding=decimal.ROUND_HALF_UP)
result.update(momValue=format(abs(change) if change == 0 else change, ".2f"), momStatus=0 if change == 0 else 1 if change > 0 else -1)
return result
def compare_data(source, target, case, users):
differences = []
omitted = {"source": case.get("source_omit_fields", []), "target": case.get("target_omit_fields", [])}
# 两边通过 HTTP 分别创建的专用测试账号,其创建时刻天然不同。
# 只忽略显式列出的夹具账号时间,真实迁移账号的所有时间字段仍须相等。
generated = {str(value) for value in case.get("generated_user_ids", [])}
generated_ids = {"source": generated, "target": {users.get(value, value) for value in generated}}
user_fields = set(case.get("user_fields", ["userId", "parentId", "creatorId", "businessId", "operatorId", "parentUserId"]))
def normalize(value, side, field="", path=""):
if isinstance(value, dict):
if side == "source" and field == "payRateRankList" and case.get("relative_pay_rate"):
value = expected_relative_pay_rate(value)
if side == "target" and value.get("available") is False:
differences.append({"path": path + ".available", "kind": "unavailable"})
if side == "source" and case.get("normalize_root_path") and value.get("level") in (0, 1, "0", "1") and value.get("idPath") is None:
value = {**value, "idPath": "0"}
excluded = set(omitted[side])
if str(value.get("userId")) in generated_ids[side]:
excluded.add("createTime")
return {key: normalize(item, side, key, path + "." + key) for key, item in value.items() if key not in excluded}
if isinstance(value, list):
rows = [normalize(item, side, field, path) for item in value]
keys = case.get("row_key")
if keys and rows and all(isinstance(row, dict) and all(key in row for key in keys) for row in rows):
rows.sort(key=lambda row: tuple(str(row[key]) for key in keys))
return rows
if value is None and field in case.get("null_empty_fields", []):
return ""
if side == "source" and case.get("map_users", True) and value is not None:
if field in user_fields:
value = users.get(str(value), str(value))
elif field == "idPath":
value = ",".join(users.get(item, item) for item in str(value).split(","))
return value
def walk(left, right, path="data"):
# 只兼容数值与 JSON 数字字符串;账号、手机号、区服编号等字符串保留前导零。
numeric_field = path.rsplit(".", 1)[-1] in case.get("numeric_fields", [])
if not isinstance(left, bool) and not isinstance(right, bool) and (numeric_field or isinstance(left, (int, float)) or isinstance(right, (int, float))):
try:
if decimal.Decimal(str(left)) == decimal.Decimal(str(right)):
return
except decimal.InvalidOperation:
pass
if type(left) is not type(right):
differences.append({"path": path, "kind": "type", "source": type(left).__name__, "target": type(right).__name__})
elif isinstance(left, dict):
for key in sorted(left.keys() | right.keys()):
if key not in left or key not in right:
differences.append({"path": path + "." + key, "kind": "missing_field", "side": "source" if key not in left else "target"})
else:
walk(left[key], right[key], path + "." + key)
elif isinstance(left, list):
if len(left) != len(right):
differences.append({"path": path, "kind": "length", "source": len(left), "target": len(right)})
for i, (a, b) in enumerate(zip(left, right)):
walk(a, b, path + f"[{i}]")
elif left != right:
# 数据值只留在仓库外;公开报告保留字段路径和摘要供复核。
differences.append({"path": path, "kind": "value", "source_sha256": hashlib.sha256(str(left).encode()).hexdigest(), "target_sha256": hashlib.sha256(str(right).encode()).hexdigest()})
walk(normalize(source, "source"), normalize(target, "target"))
return {"semantic_equal": not differences, "difference_count": len(differences), "differences": differences[:30],
"normalization": {"user_ids_mapped": case.get("map_users", True), "numeric_representation": True,
"user_fields": sorted(user_fields), "numeric_string_fields": case.get("numeric_fields", []),
"normalize_root_path": case.get("normalize_root_path", False),
"expected_relative_pay_rate": case.get("relative_pay_rate", False),
"generated_user_creation_times": len(generated),
"row_key": case.get("row_key"), "omitted_fields": omitted, "null_empty_fields": case.get("null_empty_fields", [])}}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--cases", type=Path, required=True)
parser.add_argument("--identity", default="admin")
parser.add_argument("--old-url", default="http://127.0.0.1:18080")
parser.add_argument("--new-url", default="http://127.0.0.1:19000")
parser.add_argument("--output", type=Path, required=True)
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--recompare", action="store_true", help="使用已保存的同参数 HTTP 响应重新核对,不重复访问数据库")
mode.add_argument("--refresh-target", action="store_true", help="保留已采集的同参数旧响应,重新请求优化后的新实现;新增用例仍请求两边")
args = parser.parse_args()
root = Path(__file__).resolve().parents[1]
if args.runtime.resolve().is_relative_to(root):
raise ValueError("原始业务响应必须保存在仓库外")
tokens = [(args.runtime / (prefix + "-" + args.identity + ".token")).read_text(encoding="utf-8").strip() for prefix in ("legacy", "new")]
client_id = json.loads((args.runtime / "legacy-client.json").read_text(encoding="utf-8"))["client_id"]
users = {str(row["old_id"]): str(row["new_id"]) for row in json.loads((args.runtime / "id-maps.json").read_text(encoding="utf-8")) if row["entity"] == "user"}
cases = json.loads(args.cases.read_text(encoding="utf-8"))
reports = []
folder = args.runtime / ("responses-" + args.identity)
folder.mkdir(exist_ok=True)
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as pool:
for case in cases:
capture = folder / (case["name"] + ".json")
reused_source = args.refresh_target and capture.exists()
if args.recompare or reused_source:
saved = json.loads(capture.read_text(encoding="utf-8"))
if any(saved["case"].get(key) != case.get(key) for key in ("source", "target", "params", "source_params", "target_params")):
raise ValueError("已有 HTTP 响应的请求条件不符:" + case["name"])
replies = [saved["source"], saved["target"]]
if args.refresh_target:
replies[1] = fetch(args.new_url, tokens[1], case["target"], case.get("target_params", case.get("params", {})), False, timeout=case.get("timeout_seconds", 120))
capture.write_text(json.dumps({"case": case, "source": replies[0], "target": replies[1]}, ensure_ascii=False, indent=2), encoding="utf-8")
else:
pairs = [pool.submit(fetch, base, tokens[i], case[side], case.get(side + "_params", case.get("params", {})), i == 0, client_id, case.get("timeout_seconds", 120))
for i, (base, side) in enumerate(((args.old_url, "source"), (args.new_url, "target")))]
replies = [job.result() for job in pairs]
capture.write_text(json.dumps({"case": case, "source": replies[0], "target": replies[1]}, ensure_ascii=False, indent=2), encoding="utf-8")
values = [data_of(reply) for reply in replies]
for i, side in enumerate(("source", "target")):
for key in case.get(side + "_data_path", []):
values[i] = values[i][key]
codes = [reply.get("body", {}).get("code") for reply in replies]
report = {"case": case["name"], "source": case["source"], "target": case["target"], "codes": codes,
"source_capture_reused": reused_source,
"http_passed": codes == [200, 0], "elapsed_ms": [reply["elapsed_ms"] for reply in replies],
"shapes": [shape(value) for value in values],
"exact_data_equal": values[0] == values[1]}
if all(isinstance(value, dict) and "total" in value for value in values):
report["totals"] = [value["total"] for value in values]
report["totals_equal"] = str(values[0]["total"]) == str(values[1]["total"])
reports.append(report)
report.update(compare_data(values[0], values[1], case, users))
report["passed"] = report["http_passed"] and report["semantic_equal"]
if case.get("expected_error_codes"):
messages = [reply.get("body", {}).get("msg") for reply in replies]
report["rejection_equal"] = codes == case["expected_error_codes"] and messages[0] == messages[1]
report["expected_error_codes"] = case["expected_error_codes"]
report["passed"] = report["rejection_equal"]
print(json.dumps(report, ensure_ascii=False), flush=True)
args.output.write_text(json.dumps({"identity": args.identity, "checks": reports}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
if not all(report["passed"] for report in reports):
raise SystemExit(1)
if __name__ == "__main__":
main()
-218
View File
@@ -1,218 +0,0 @@
// 真实开发服务:检查 OA 数据、统计口径持久化与响应式布局。
// 临时修改当前测试账号的统计口径,结束时恢复;截图只写入指定的私有目录。
const fs = require('node:fs/promises')
const path = require('node:path')
const assert = require('node:assert/strict')
const { chromium } = require(process.env.SW_PLAYWRIGHT_MODULE || 'playwright')
;(async () => {
const base = process.env.SW_UI_BASE_URL || 'http://127.0.0.1:8080'
const output = process.env.SW_UI_OUTPUT_DIR
assert.ok(output && process.env.SW_TEST_TOKEN_FILE, 'Set SW_UI_OUTPUT_DIR and SW_TEST_TOKEN_FILE')
const token = (await fs.readFile(process.env.SW_TEST_TOKEN_FILE, 'utf8')).trim()
const browser = await chromium.launch({ channel: 'chrome', headless: true })
const context = await browser.newContext({ viewport: { width: 1600, height: 1000 } })
const headers = { 'x-token': token }
const api = async (method, endpoint, data) => {
const response = await context.request.fetch(`${base}/api${endpoint}`, { method, headers, data })
assert.ok(response.ok())
const body = await response.json()
assert.equal(body.code, 0)
return body.data
}
let original
const report = { checks: [], pageErrors: [] }
try {
await fs.mkdir(output, { recursive: true })
original = await api('GET', '/sw/user/preference')
await fs.writeFile(path.join(output, 'original-preference.json'), JSON.stringify(original))
await context.addInitScript(value => localStorage.setItem('token', value), token)
const page = await context.newPage()
page.on('pageerror', e => report.pageErrors.push(e.message))
const activationPath = '/sw/promoteActivePool/bindList'
await page.goto(`${base}/#/layout/sw-BusMgmt/ChannelActivationList`)
await page.locator('.el-table__body-wrapper tr').first().waitFor()
const activation = await api('GET', `${activationPath}?pageNum=1&pageSize=10&platformNames=lx,hw,yq`)
assert.equal(activation.available, true)
assert.ok(activation.total > 0)
assert.equal(await page.locator('.el-table__body-wrapper tr').count(), activation.rows.length)
assert.equal(await page.locator('.el-alert--warning').count(), 0)
report.checks.push({ name: 'actual OA activation rows', total: activation.total })
await page.screenshot({ path: path.join(output, 'activation.png'), animations: 'disabled' })
const pagination = page.locator('.kra-pagination')
const pageRequests = []
const trackPage = request => {
if (request.url().includes(activationPath)) pageRequests.push(Number(new URL(request.url()).searchParams.get('pageNum')))
}
page.on('request', trackPage)
const movePage = async (target, action) => {
pageRequests.length = 0
const [response] = await Promise.all([
page.waitForResponse(r => r.url().includes(activationPath) && Number(new URL(r.url()).searchParams.get('pageNum')) === target),
action()
])
const expected = (await response.json()).data
await page.waitForFunction(() => !document.querySelector('.el-loading-mask'))
assert.equal((await pagination.locator('.el-pager .is-active').innerText()).trim(), String(target))
assert.equal(await page.locator('.el-table__body-wrapper tr').count(), expected.rows.length)
const names = await page.locator('.el-table__body-wrapper tr td:nth-child(2) .cell').allTextContents()
assert.deepEqual(names.map(name => name.trim()), expected.rows.map(row => row.promoteAccount))
assert.deepEqual(pageRequests, [target], 'pagination must not trigger a second request back to page 1')
}
assert.ok(activation.total > 10, 'Pagination check needs more than one page')
await movePage(2, () => pagination.locator('.btn-next').click())
const lastPage = Math.ceil(activation.total / 10)
if (lastPage > 2) await movePage(lastPage, () => pagination.locator('.el-pager .number').filter({ hasText: new RegExp(`^${lastPage}$`) }).click())
await page.screenshot({ path: path.join(output, 'activation-last-page.png'), animations: 'disabled' })
await movePage(lastPage - 1, () => pagination.locator('.btn-prev').click())
if (lastPage - 1 !== 1) await movePage(1, () => pagination.locator('.el-pager .number').filter({ hasText: /^\s*1\s*$/ }).click())
page.off('request', trackPage)
report.checks.push({ name: 'activation pagination: next, last, previous, first; response rows match and no page reset', total: activation.total, lastPage })
const preference = page.getByRole('group', { name: '统计口径', exact: true })
for (const [key, label] of [['includeSubPromote', '包含下级渠道'], ['idcardDedup', '身份证去重']]) {
const checkbox = preference.getByRole('checkbox', { name: label, exact: true })
await page.waitForFunction(label => [...document.querySelectorAll('.business-preferences input')].some(el => el.closest('label')?.textContent.includes(label) && !el.disabled), label)
assert.equal(await checkbox.isChecked(), original[key])
const [saved] = await Promise.all([
page.waitForResponse(r => r.url().includes('/sw/user/preference') && r.request().method() === 'PUT'),
page.waitForResponse(r => r.url().includes(activationPath)),
preference.getByText(label, { exact: true }).click()
])
assert.equal((await saved.json()).code, 0)
assert.equal((await api('GET', '/sw/user/preference'))[key], !original[key])
await page.reload()
await preference.waitFor()
await page.waitForFunction(() => [...document.querySelectorAll('.business-preferences input')].every(el => !el.disabled))
assert.equal(await preference.getByRole('checkbox', { name: label, exact: true }).isChecked(), !original[key])
report.checks.push({ name: `${key}: visible, persisted, page refreshed` })
}
await api('PUT', '/sw/user/preference', original)
// 保存失败不能让勾选状态与服务端失去同步。
await page.reload()
await page.waitForFunction(() => document.querySelectorAll('.business-preferences input').length === 2 && [...document.querySelectorAll('.business-preferences input')].every(el => !el.disabled))
const rejectSave = route => route.request().method() === 'PUT'
? route.fulfill({ json: { code: 7, msg: '测试保存失败', data: {} } }) : route.continue()
await page.route('**/api/sw/user/preference', rejectSave)
await Promise.all([
page.waitForResponse(r => r.url().includes('/sw/user/preference') && r.request().method() === 'PUT'),
preference.getByText('包含下级渠道', { exact: true }).click()
])
await page.waitForFunction(() => [...document.querySelectorAll('.business-preferences input')].every(el => !el.disabled))
assert.equal(await preference.getByRole('checkbox', { name: '包含下级渠道', exact: true }).isChecked(), original.includeSubPromote)
await page.unroute('**/api/sw/user/preference', rejectSave)
report.checks.push({ name: 'failed preference save preserves previous value' })
const search = () => page.locator('.kra-search-box').getByRole('button', { name: '查询', exact: true }).click()
const activationPattern = '**/api/sw/promoteActivePool/bindList?*'
await page.route(activationPattern, route => route.fulfill({ json: { code: 7, msg: '测试查询失败', data: {} } }))
await search()
await page.getByText('查询失败,请重试', { exact: true }).waitFor()
assert.equal(await page.locator('.el-table__body-wrapper tr').count(), 0)
await page.unroute(activationPattern)
await page.route(activationPattern, route => route.fulfill({ json: { code: 0, data: { available: false, rows: [], total: 0 } } }))
await search()
await page.getByText('OA 数据源暂不可用', { exact: true }).waitFor()
await page.unroute(activationPattern)
await search()
await page.locator('.el-table__body-wrapper tr').first().waitFor()
report.checks.push({ name: 'activation error and unavailable states differ from empty results; retry recovers' })
for (const [name, route, grid] of [
['ranking', 'sw-DataAnalysis/RankingAnalysis', '.ranking-panels'],
['dashboard', 'dashboard', '.business-ranks']
]) {
await page.goto(`${base}/#/layout/${route}`)
await page.locator(grid).waitFor()
await page.waitForFunction(() => !document.querySelector('.el-loading-mask'))
await page.locator('.el-message').evaluateAll(nodes => nodes.forEach(node => node.querySelector('.el-message__closeBtn')?.click()))
if (name === 'ranking') {
for (const panel of await page.locator('.ranking-panel').all()) {
assert.match(await panel.innerText(), /上期环比/)
assert.doesNotMatch(await panel.innerText(), /百分点|\bpp\b/)
}
} else {
assert.equal(await page.getByText('快捷功能', { exact: true }).count(), 0)
assert.equal(await page.getByText('公告', { exact: true }).count(), 0)
assert.equal(await page.getByText('商务数据', { exact: true }).count(), 0)
assert.ok(await page.getByText('数据平台', { exact: true }).isVisible())
for (const index of [0, 1]) {
const panel = page.locator('.business-ranks > .el-card').nth(index)
await panel.locator('.business-rank-period').click()
const optionsId = await panel.getByRole('combobox').getAttribute('aria-controls')
await Promise.all([
page.waitForResponse(r => /\/sw\/workbench\/(registerTop|payRank)/.test(r.url()) && r.url().includes('period=last7')),
page.locator(`[id="${optionsId}"]`).getByRole('option', { name: '近7天', exact: true }).click()
])
await panel.locator('.ranking-table tbody tr').first().waitFor()
assert.equal(await panel.locator('.ranking-empty').count(), 0)
}
report.checks.push({ name: 'dashboard period filters refresh real data in compact tables' })
}
for (const [width, columns] of [[1920, 4], [1600, 4], [1366, 4], [1024, 2], [390, 1]]) {
await page.setViewportSize({ width, height: 1000 })
if (width === 390) await page.getByRole('button', { name: '打开菜单', exact: true }).waitFor()
await page.waitForFunction(({grid, columns}) => getComputedStyle(document.querySelector(grid)).gridTemplateColumns.split(' ').length === columns, { grid, columns })
const boxes = await page.locator(`${grid} > *`).evaluateAll(nodes => nodes.map(n => { const r = n.getBoundingClientRect(); return { x: r.x, y: r.y, right: r.right } }))
if (columns === 4) assert.ok(boxes.every(box => Math.abs(box.y - boxes[0].y) < 1))
assert.ok(boxes.every(box => box.x >= 0 && box.right <= width + 1), `panel overflow: ${name}, ${width}`)
const overflow = await page.locator(`${grid} .el-table__body-wrapper, ${grid} .ranking-table-wrap`).evaluateAll(nodes => nodes.some(n => n.querySelector('table')?.clientWidth > n.clientWidth + 1))
assert.equal(overflow, false, `table horizontal overflow: ${name}, ${width}`)
if (name === 'ranking') {
const tallRows = await page.locator('.ranking-table tr').evaluateAll(nodes => nodes.some(n => n.getBoundingClientRect().height > 40))
assert.equal(tallRows, false, `ranking rows should stay compact: ${width}`)
}
const clippedNumbers = await page.locator(`${grid} tbody .ranking-numeric, ${grid} .ranking-number`).evaluateAll(nodes => nodes.some(n => n.scrollWidth > n.clientWidth + 1))
assert.equal(clippedNumbers, false, `numeric values must be fully visible: ${name}, ${width}`)
const prefBox = await preference.boundingBox()
assert.ok(prefBox && prefBox.x >= 0 && prefBox.x + prefBox.width <= width, `preference overflow at ${width}`)
const toolbarOverflow = await page.getByRole('button', { name: '打开菜单', exact: true }).count() ? await page.locator('.business-preferences').evaluate(el => {
const header = el.closest('.fixed')
return header.scrollWidth > header.clientWidth + 1
}) : false
assert.equal(toolbarOverflow, false, `mobile header overflow: ${width}`)
await page.screenshot({ path: path.join(output, `${name}-${width}.png`), animations: 'disabled' })
report.checks.push({ name: `${name} layout`, width, columns, noOverflow: true })
}
await page.setViewportSize({ width: 1600, height: 1000 })
}
await page.goto(`${base}/#/layout/sw-BusMgmt/BusinessList`)
await page.getByRole('button', { name: '添加子商务', exact: true }).waitFor()
assert.equal(await page.locator('.business-view-switch').count(), 0)
assert.equal(await page.getByRole('tab', { name: '商务列表', exact: true }).count(), 0)
await page.screenshot({ path: path.join(output, 'business-list-admin.png'), animations: 'disabled' })
// 只替换展示身份核对三级专属切换器;读取接口仍用授权测试身份,不做业务写入。
await page.route('**/api/user/getUserInfo', async route => {
const response = await route.fetch()
const body = await response.json()
Object.assign(body.data.userInfo, { level: 3, authorityId: 903 })
await route.fulfill({ response, json: body })
})
await page.reload()
await page.getByRole('radiogroup', { name: '商务管理视图', exact: true }).waitFor()
assert.equal(await page.getByRole('button', { name: '添加子商务', exact: true }).count(), 0)
const viewSwitch = page.getByRole('radiogroup', { name: '商务管理视图', exact: true })
await Promise.all([
page.waitForResponse(r => r.url().includes('/sw/business/bindPlatform')),
viewSwitch.getByText('绑定列表', { exact: true }).click()
])
assert.equal(await page.getByPlaceholder('请输入账号', { exact: true }).isVisible(), false)
await viewSwitch.getByText('商务列表', { exact: true }).click()
assert.ok(await page.getByPlaceholder('请输入账号', { exact: true }).isVisible())
await page.screenshot({ path: path.join(output, 'business-list-level3.png'), animations: 'disabled' })
report.checks.push({ name: 'business list: no redundant single tab; controlled level-3 switch works' })
assert.deepEqual(report.pageErrors, [])
report.passed = true
} catch (e) {
const page = context.pages()[0]
if (page) await page.screenshot({ path: path.join(output, 'failure.png'), animations: 'disabled' }).catch(() => {})
throw e
} finally {
if (original) await api('PUT', '/sw/user/preference', original)
await fs.writeFile(path.join(output, 'ui-checks.json'), JSON.stringify(report, null, 2) + '\n')
await browser.close()
}
console.log(JSON.stringify(report))
})().catch(e => { console.error(e.stack); process.exitCode = 1 })
-188
View File
@@ -1,188 +0,0 @@
"""在同一测试服务器核对 SW 源副本、备份目标库并执行迁移;凭据只读取本地配置。"""
import argparse
import datetime as dt
import gzip
import hashlib
import json
import re
import shutil
import subprocess
import tempfile
from pathlib import Path
import pymysql
import yaml
def identifier(value):
if not re.fullmatch(r"[A-Za-z0-9_]+", value):
raise ValueError("数据库和表名只能包含字母、数字、下划线")
return "`" + value + "`"
def write_json(path, data):
path.write_text(json.dumps(data, ensure_ascii=False, indent=2, default=str) + "\n", encoding="utf-8")
def connect(config):
return pymysql.connect(host=config["host"], port=int(config["port"]), user=config["user"],
password=config["password"], charset="utf8mb4", connect_timeout=10,
read_timeout=180, autocommit=True, cursorclass=pymysql.cursors.DictCursor)
def query(conn, sql, args=None):
with conn.cursor() as cursor:
cursor.execute(sql, args)
return cursor.fetchall()
def source_tables(sql):
tables = re.findall(r"CREATE TABLE `bak_sw2kra_20261103_([^`]+)` LIKE", sql)
if len(tables) != 26 or len(set(tables)) != 26:
raise ValueError("迁移源表清单发生变化,请先复核备份范围")
return tables
def fingerprint(conn, database, table):
columns = query(conn, "SELECT column_name,data_type FROM information_schema.columns WHERE table_schema=%s AND table_name=%s ORDER BY ordinal_position", (database, table))
keys = query(conn, "SELECT column_name FROM information_schema.statistics WHERE table_schema=%s AND table_name=%s AND index_name='PRIMARY' ORDER BY seq_in_index", (database, table))
if not keys:
raise ValueError("备份核对要求稳定主键:" + table)
order = ",".join(identifier(item["column_name"]) for item in keys)
digest = hashlib.sha256()
rows = query(conn, f"SELECT * FROM {identifier(database)}.{identifier(table)} ORDER BY {order}")
json_columns = [item["column_name"] for item in columns if item["data_type"] == "json"]
for row in rows:
# JSON 的内部二进制布局在导入后可能不同,比较实际业务值而非 CHECKSUM 的存储字节。
for name in json_columns:
if row[name] is not None:
row[name] = json.loads(row[name])
digest.update(json.dumps(row, ensure_ascii=False, sort_keys=True, default=str, separators=(",", ":")).encode())
digest.update(b"\n")
return {"rows": len(rows), "sha256": digest.hexdigest()}
def client_options(config, folder):
# 使用独立选项文件,避免密码出现在进程命令行或输出中。
def escape(value):
return str(value).replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n").replace("\r", "\\r")
handle = tempfile.NamedTemporaryFile(mode="w", encoding="utf-8", suffix=".cnf", dir=folder, delete=False)
with handle:
handle.write("[client]\n")
for key in ("host", "port", "user", "password"):
handle.write(f'{key}="{escape(config[key])}"\n')
handle.write("default-character-set=utf8mb4\n")
return Path(handle.name)
def prepare(args, config, sql):
if (args.state / (args.target + "-migration-source.sql.gz")).exists():
raise ValueError("目标备份已存在;请使用新的状态目录,保留原始备份")
tables = source_tables(sql)
manifest = {"source": args.source, "target": args.target, "created_at": dt.datetime.now(dt.timezone.utc),
"sql_sha256": hashlib.sha256(sql.encode()).hexdigest(), "checks": []}
with connect(config) as conn:
manifest["server"] = query(conn, "SELECT @@hostname hostname,@@version version")[0]
if query(conn, "SELECT 1 FROM information_schema.tables WHERE table_schema=%s AND table_name='sys_users'", (args.target,)):
raise ValueError("目标已迁移,不能覆盖原始基线")
for table in tables:
checksums = query(conn, f"CHECKSUM TABLE {identifier(args.source)}.{identifier(table)}, {identifier(args.target)}.{identifier(table)} EXTENDED")
values = [row["Checksum"] for row in checksums]
fingerprints = [fingerprint(conn, db, table) for db in (args.source, args.target)]
item = {"table": table, "rows": [item["rows"] for item in fingerprints], "checksums": values,
"fingerprints": fingerprints, "passed": fingerprints[0] == fingerprints[1]}
manifest["checks"].append(item)
print(json.dumps(item, ensure_ascii=False), flush=True)
if not all(item["passed"] for item in manifest["checks"]):
write_json(args.state / "baseline.json", manifest)
raise RuntimeError("源库与目标库框架数据不一致,未执行迁移")
manifest["tables_before"] = query(conn, "SELECT table_name,engine FROM information_schema.tables WHERE table_schema=%s ORDER BY table_name", (args.target,))
schemas = {}
for row in manifest["tables_before"]:
table = row["table_name"]
schemas[table] = query(conn, f"SHOW CREATE TABLE {identifier(args.target)}.{identifier(table)}")[0]["Create Table"]
write_json(args.state / "schema-before.json", schemas)
dump = Path(shutil.which("mysqldump") or Path(shutil.which("mysql")).with_name("mysqldump.exe"))
options = client_options(config, args.state)
backup = args.state / (args.target + "-migration-source.sql.gz")
try:
command = [str(dump), "--defaults-extra-file=" + str(options), "--single-transaction", "--quick", "--hex-blob",
"--set-gtid-purged=OFF", "--skip-lock-tables", args.target, *tables]
with (args.state / "backup.stderr.log").open("wb") as errors:
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=errors)
with gzip.open(backup, "wb") as output:
shutil.copyfileobj(process.stdout, output)
if process.wait() != 0:
raise RuntimeError("备份失败,详情仅保存在本地日志")
# 完整解压检查,避免把未写完的压缩文件当作成功备份。
with gzip.open(backup, "rb") as stream:
while stream.read(1024 * 1024):
pass
manifest["backup"] = {"path": str(backup.resolve()), "bytes": backup.stat().st_size,
"sha256": hashlib.sha256(backup.read_bytes()).hexdigest()}
write_json(args.state / "baseline.json", manifest)
print(json.dumps({"backup": manifest["backup"], "passed": True}), flush=True)
finally:
options.unlink(missing_ok=True)
def migrate(args, config, sql):
manifest = json.loads((args.state / "baseline.json").read_text(encoding="utf-8"))
if (manifest["source"], manifest["target"]) != (args.source, args.target):
raise ValueError("备份对应的数据库与本次选择不符")
backup = Path(manifest["backup"]["path"])
if hashlib.sha256(backup.read_bytes()).hexdigest() != manifest["backup"]["sha256"]:
raise ValueError("备份文件校验失败")
statements = re.findall(r"(?m)^USE `[^`]+`;", sql)
if len(statements) != 1:
raise ValueError("SQL 必须只有一个显式 USE 入口")
prepared = sql.replace(statements[0], "USE " + identifier(args.target) + ";")
(args.state / "executed-migration.sql").write_text(prepared, encoding="utf-8")
with connect(config) as conn:
actual_server = query(conn, "SELECT @@hostname hostname,@@version version")[0]
if actual_server != manifest["server"]:
raise ValueError("连接服务器与备份时不同")
# 仅首次执行需要核对原始基线;重复执行由迁移 SQL 的完成账本保护。
migrated = query(conn, "SELECT 1 FROM information_schema.tables WHERE table_schema=%s AND table_name='sys_users'", (args.target,))
if not migrated:
for item in manifest["checks"]:
if fingerprint(conn, args.target, item["table"]) != item["fingerprints"][1]:
raise ValueError("目标数据在备份后变化:" + item["table"])
options = client_options(config, args.state)
try:
with (args.state / "migration.stdout.log").open("wb") as output, (args.state / "migration.stderr.log").open("wb") as errors:
result = subprocess.run([shutil.which("mysql"), "--defaults-extra-file=" + str(options), "--batch", "--binary-mode"],
input=prepared.encode("utf-8"), stdout=output, stderr=errors)
print(json.dumps({"target": args.target, "exit_code": result.returncode,
"sql_sha256": hashlib.sha256(prepared.encode()).hexdigest()}), flush=True)
if result.returncode:
raise RuntimeError("迁移 SQL 失败,请检查本地日志;不会自动覆盖部分迁移状态")
finally:
options.unlink(missing_ok=True)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("action", choices=("prepare", "migrate"))
parser.add_argument("--config", type=Path, default=Path("configs/config.yaml"))
parser.add_argument("--sql", type=Path, default=Path("docs/sw6-to-kra/sw_to_kra_migrate.sql"))
parser.add_argument("--source", default="sw8")
parser.add_argument("--target", default="sw9")
parser.add_argument("--state", type=Path, required=True)
args = parser.parse_args()
identifier(args.source)
identifier(args.target)
if args.source == args.target or args.target in ("sw", "sw6"):
raise ValueError("演练目标必须为独立测试副本")
args.state = args.state.resolve()
if args.state.is_relative_to(Path(__file__).resolve().parents[1]):
raise ValueError("含账号数据的备份必须保存在仓库外")
args.state.mkdir(parents=True, exist_ok=True)
config = yaml.safe_load(args.config.read_text(encoding="utf-8"))["data"]["database"]
sql = args.sql.read_text(encoding="utf-8")
{"prepare": prepare, "migrate": migrate}[args.action](args, config, sql)
if __name__ == "__main__":
main()
-76
View File
@@ -1,76 +0,0 @@
"""只读请求商务接口,记录耗时、完整数据摘要及 SQL;敏感内容必须存放仓库外。"""
import argparse
import hashlib
import json
import re
import statistics
import time
import urllib.parse
import urllib.request
from pathlib import Path
def log_offsets(directory):
return {p: p.stat().st_size for p in directory.rglob("*.log")}
def read_queries(directory, offsets):
queries = []
files = list(directory.rglob("*.log"))
files = [file for file in files if file.parent.name == "sql"] or files
for file in files:
with file.open("rb") as stream:
stream.seek(offsets.get(file, 0))
for line in stream:
try:
row = json.loads(line)
except (ValueError, UnicodeDecodeError):
continue
if row.get("gorm_logger") and "sql" in row:
queries.append(row)
return queries
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--base-url", default="http://127.0.0.1:19004")
parser.add_argument("--phase", required=True)
parser.add_argument("--repeat", type=int, default=2)
parser.add_argument("--cases", type=Path)
args = parser.parse_args()
if args.runtime.resolve().is_relative_to(Path(__file__).resolve().parents[1]):
raise ValueError("完整接口响应和 SQL 必须留在仓库外")
output = args.runtime / args.phase
output.mkdir(exist_ok=True)
cases = json.loads((args.cases or args.runtime / "cases.json").read_text(encoding="utf-8"))
token = (args.runtime / "new-admin.token").read_text(encoding="utf-8").strip()
report = []
for case in cases:
elapsed = []
for round_number in range(args.repeat):
offsets = log_offsets(args.runtime / "sql-logs")
url = args.base_url + case["path"] + "?" + urllib.parse.urlencode(case["params"], doseq=True)
request = urllib.request.Request(url, headers={"x-token": token, "X-Request-Id": f"perf-{args.phase}-{case['name']}-{round_number}"})
started = time.monotonic()
with urllib.request.urlopen(request, timeout=300) as response:
body = json.load(response)
elapsed.append(round((time.monotonic() - started) * 1000, 2))
if body.get("code") != 0:
raise RuntimeError("请求失败:" + case["name"])
queries = read_queries(args.runtime / "sql-logs", offsets)
data = body["data"]
capture = {"case": case, "data": data, "elapsedMs": elapsed[-1], "queries": queries}
(output / f"{case['name']}-{round_number}.json").write_text(json.dumps(capture, ensure_ascii=False, indent=2), encoding="utf-8")
digest = hashlib.sha256(json.dumps(data, sort_keys=True, ensure_ascii=False).encode()).hexdigest()
slow = sorted(queries, key=lambda row: row.get("elapsed_ms", 0), reverse=True)[:5]
item = {"case": case["name"], "round": round_number, "elapsedMs": elapsed[-1], "dataSha256": digest, "queryCount": len(queries), "slowQueries": [{"elapsedMs": q.get("elapsed_ms"), "rows": q.get("rows"), "tables": list(dict.fromkeys(re.findall(r"(?:FROM|JOIN)\s+`?([\w.]+)", q["sql"], re.I)))} for q in slow]}
report.append(item)
(output / "report.json").write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
print(json.dumps(item), flush=True)
print(json.dumps({"case": case["name"], "medianMs": statistics.median(elapsed)}), flush=True)
if __name__ == "__main__":
main()
-79
View File
@@ -1,79 +0,0 @@
"""在同一个 sw9 快照上核对优化前后接口,涵盖分页/排序及统计偏好。"""
import argparse
import json
import urllib.parse
import urllib.request
from pathlib import Path
import yaml
from sw_database_rehearsal import connect, query
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--before-url", default="http://127.0.0.1:19002")
parser.add_argument("--after-url", default="http://127.0.0.1:19004")
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
if args.runtime.resolve().is_relative_to(Path(__file__).resolve().parents[1]):
raise ValueError("原始响应必须保存在仓库外")
config = yaml.safe_load((args.runtime / "config.yaml").read_text(encoding="utf-8"))["data"]["database"]
if config["name"] != "sw9":
raise ValueError("统计偏好测试仅允许 sw9")
folder = args.runtime / "parity"
folder.mkdir(exist_ok=True)
reports = []
def api(base, token, path, params=None, payload=None):
url = base + path + ("?" + urllib.parse.urlencode(params, doseq=True) if params else "")
request = urllib.request.Request(url, headers={"x-token": token, "Content-Type": "application/json"}, data=json.dumps(payload).encode() if payload is not None else None, method="PUT" if payload is not None else "GET")
with urllib.request.urlopen(request, timeout=300) as response:
data = json.load(response)
if data.get("code") != 0:
raise RuntimeError("接口请求失败:" + path)
return data["data"]
def compare(case, token, group):
values = [api(base, token, case["path"], case["params"]) for base in (args.before_url, args.after_url)]
name = group + "-" + case["name"]
(folder / (name + ".json")).write_text(json.dumps({"case": case, "before": values[0], "after": values[1]}, ensure_ascii=False, indent=2), encoding="utf-8")
row = {"case": name, "passed": values[0] == values[1]}
reports.append(row)
args.output.write_text(json.dumps({"checks": reports, "passed": all(row["passed"] for row in reports)}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print(json.dumps(row), flush=True)
if not row["passed"]:
raise AssertionError("接口数据不一致:" + name)
admin = (args.runtime / "new-admin.token").read_text(encoding="utf-8").strip()
for case in json.loads((args.runtime / "parity-cases.json").read_text(encoding="utf-8")):
compare(case, admin, "admin")
token = (args.runtime / "new-level3.token").read_text(encoding="utf-8").strip()
user = api(args.before_url, token, "/user/getUserInfo")["userInfo"]["ID"]
original_preference = api(args.before_url, token, "/sw/user/preference")
cases = json.loads((args.runtime / "cases.json").read_text(encoding="utf-8"))
with connect(config) as db:
original = query(db, "SELECT * FROM sw9.sys_user_preference WHERE user_id=%s", (user,))
try:
for include, dedup in ((True, True), (True, False), (False, True), (False, False)):
api(args.before_url, token, "/sw/user/preference", payload={"includeSubPromote": include, "idcardDedup": dedup})
for case in cases:
compare(case, token, f"level3-sub{int(include)}-dedup{int(dedup)}")
finally:
try:
api(args.before_url, token, "/sw/user/preference", payload=original_preference)
finally:
# Restore the row even if the service stopped during verification.
if original:
row = original[0]
query(db, "UPDATE sw9.sys_user_preference SET include_sub_promote=%s,idcard_dedup=%s,update_time=%s WHERE user_id=%s", (row["include_sub_promote"], row["idcard_dedup"], row["update_time"], user))
else:
query(db, "DELETE FROM sw9.sys_user_preference WHERE user_id=%s", (user,))
assert query(db, "SELECT * FROM sw9.sys_user_preference WHERE user_id=%s", (user,)) == original
if __name__ == "__main__":
main()
-76
View File
@@ -1,76 +0,0 @@
"""以同一主账号的实际数据请求新旧明细导出,并逐单元格核对,文件保留在仓库外。"""
import argparse
import decimal
import io
import json
import time
import urllib.parse
import urllib.request
from pathlib import Path
import openpyxl
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--old-url", default="http://127.0.0.1:18080")
parser.add_argument("--new-url", default="http://127.0.0.1:19001")
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
if args.runtime.resolve().is_relative_to(Path(__file__).resolve().parents[1]):
raise ValueError("导出文件必须保存在仓库外")
client = json.loads((args.runtime / "legacy-client.json").read_text(encoding="utf-8"))["client_id"]
headers = [{"Authorization": "Bearer " + (args.runtime / "legacy-admin.token").read_text().strip(), "clientid": client},
{"x-token": (args.runtime / "new-admin.token").read_text().strip()}]
reports = []
for kind, time_key in (("register", "Register"), ("spend", "Pay")):
params = {"sourceDb": "lx", "pageSize": 10, "begin" + time_key + "Time": "2026-11-03", "end" + time_key + "Time": "2026-11-03"}
request = urllib.request.Request(args.new_url + "/sw/" + kind + "/list?" + urllib.parse.urlencode(params), headers=headers[1])
with urllib.request.urlopen(request, timeout=60) as response:
body = json.load(response)
if body.get("code") != 0:
raise RuntimeError("读取导出样本失败:" + kind)
sample = next(row for row in body["data"]["rows"] if row.get("userId") is not None)
params["userId"] = sample["userId"]
sheets, durations = [], []
for i, base in enumerate((args.old_url, args.new_url)):
payload = urllib.parse.urlencode(params).encode()
request = urllib.request.Request(base + ("/system/" if i == 0 else "/sw/") + kind + "/export", data=payload,
headers={**headers[i], "Content-Type": "application/x-www-form-urlencoded"}, method="POST")
started = time.monotonic()
with urllib.request.urlopen(request, timeout=120) as response:
raw = response.read()
durations.append(round((time.monotonic() - started) * 1000))
if not raw.startswith(b"PK"):
raise RuntimeError("导出未返回工作簿:" + kind + "/" + str(i))
(args.runtime / (("legacy-" if i == 0 else "new-") + kind + "-export.xlsx")).write_bytes(raw)
workbook = openpyxl.load_workbook(io.BytesIO(raw), read_only=True, data_only=True)
sheets.append(list(workbook.worksheets[0].values))
workbook.close()
differences = []
for row_number, (old, new) in enumerate(zip(*sheets), 1):
if len(old) != len(new):
differences.append({"row": row_number, "kind": "columns"})
for col, (a, b) in enumerate(zip(old, new), 1):
if a == b or (a in (None, "") and b in (None, "")):
continue
if isinstance(a, (int, float)) or isinstance(b, (int, float)):
try:
if decimal.Decimal(str(a)) == decimal.Decimal(str(b)):
continue
except decimal.InvalidOperation:
pass
differences.append({"row": row_number, "column": col, "kind": "value"})
rows = [len(sheet) - 1 for sheet in sheets]
reports.append({"kind": kind, "rows": rows, "columns": [len(sheet[0]) for sheet in sheets], "elapsed_ms": durations,
"differences": differences, "passed": rows[0] > 0 and rows[0] == rows[1] and not differences})
print(json.dumps(reports[-1], ensure_ascii=False), flush=True)
args.output.write_text(json.dumps({"checks": reports, "passed": all(row["passed"] for row in reports)}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
if not all(row["passed"] for row in reports):
raise SystemExit(1)
if __name__ == "__main__":
main()
-61
View File
@@ -1,61 +0,0 @@
"""用已有真实绑定的三级账号逐项切换统计偏好,对比报表后恢复原始偏好行。"""
import argparse
import json
import subprocess
import sys
from pathlib import Path
from sw_write_verify import Rehearsal
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--config", type=Path, default=Path("configs/config.yaml"))
parser.add_argument("--old-url", default="http://127.0.0.1:18080")
parser.add_argument("--new-url", default="http://127.0.0.1:19001")
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
run = Rehearsal(args)
identity = json.loads((args.runtime / "test-business-identities.json").read_text())["level3"]
ids = {"legacy": identity["user_id"], "new": run.mapping[str(identity["user_id"])]}
originals, preferences = {}, {}
for side, db in (("legacy", "sw8"), ("new", "sw9")):
originals[side] = run.query(f"SELECT * FROM {db}.sys_user_preference WHERE user_id=%s", (ids[side],))
preferences[side] = run.call(side, "level3", "GET", run.prefix(side) + "/user/preference")["data"]
(args.runtime / "preferences-before-matrix.json").write_text(json.dumps(originals, ensure_ascii=False, indent=2, default=str), encoding="utf-8")
templates = json.loads((args.runtime / "level3-business-cases.json").read_text())
templates = [row for row in templates if row["name"] in ("channelData-list", "gameData-list", "rank-game", "workbench-summary")]
try:
for include, dedup in ((True, True), (True, False), (False, True), (False, False)):
mode = f"sub{int(include)}-dedup{int(dedup)}"
value = {"includeSubPromote": include, "idcardDedup": dedup}
for side in ("legacy", "new"):
run.call(side, "level3", "PUT", run.prefix(side) + "/user/preference", value)
actual = run.call(side, "level3", "GET", run.prefix(side) + "/user/preference")["data"]
run.check("偏好写入 " + mode + " " + side, actual == value)
cases = [{**row, "name": mode + "-" + row["name"]} for row in templates]
cases_file = args.runtime / ("preference-cases-" + mode + ".json")
cases_file.write_text(json.dumps(cases), encoding="utf-8")
output = args.output.with_name(args.output.stem + "-" + mode + args.output.suffix)
with (args.runtime / ("preference-" + mode + ".log")).open("wb") as log:
completed = subprocess.run([sys.executable, str(Path(__file__).with_name("sw_api_compare.py")), "--runtime", str(args.runtime), "--cases", str(cases_file),
"--identity", "level3", "--old-url", args.old_url, "--new-url", args.new_url, "--output", str(output)], stdout=log, stderr=subprocess.STDOUT)
run.check("偏好模式报表一致 " + mode, completed.returncode == 0, report=output.name)
finally:
for side, db in (("legacy", "sw8"), ("new", "sw9")):
run.call(side, "level3", "PUT", run.prefix(side) + "/user/preference", preferences[side])
if not originals[side]:
run.query(f"DELETE FROM {db}.sys_user_preference WHERE user_id=%s", (ids[side],))
else:
row = originals[side][0]
run.query(f"UPDATE {db}.sys_user_preference SET include_sub_promote=%s,idcard_dedup=%s,update_time=%s WHERE user_id=%s",
(row["include_sub_promote"], row["idcard_dedup"], row["update_time"], ids[side]))
actual = run.query(f"SELECT * FROM {db}.sys_user_preference WHERE user_id=%s", (ids[side],))
run.check("偏好原始行已恢复 " + side, actual == originals[side])
run.db.close()
if __name__ == "__main__":
main()
-119
View File
@@ -1,119 +0,0 @@
"""备份/比较/恢复 sw8 与 sw9 指定自然日的四张报表,不备份或改写原始业务流水。"""
import argparse
import datetime as dt
import gzip
import hashlib
import json
import shutil
import subprocess
from pathlib import Path
import yaml
from sw_database_rehearsal import client_options, connect, query, write_json
TABLES = ("tab_sw_report_day", "tab_sw_report_day_key", "tab_sw_report_server_day", "tab_sw_report_server_day_key")
def snapshot(conn, database, day, logical=False):
result = {}
for table in TABLES:
rows = query(conn, f"SELECT * FROM `{database}`.`{table}` WHERE stat_date=%s ORDER BY id", (day,))
if logical:
rows = [{key: value for key, value in row.items() if key not in ("id", "created_at", "updated_at")} for row in rows]
serialized = [json.dumps(row, ensure_ascii=False, sort_keys=True, separators=(",", ":"), default=str) for row in rows]
if logical:
serialized.sort()
result[table] = {"rows": len(rows), "sha256": hashlib.sha256("\n".join(serialized).encode()).hexdigest()}
return result
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("action", choices=("backup", "verify", "restore"))
parser.add_argument("--config", type=Path, default=Path("configs/config.yaml"))
parser.add_argument("--date", default="2026-11-03")
parser.add_argument("--state", type=Path, required=True)
parser.add_argument("--output", type=Path)
args = parser.parse_args()
dt.date.fromisoformat(args.date)
args.state = args.state.resolve()
if args.state.is_relative_to(Path(__file__).resolve().parents[1]):
raise ValueError("报表备份和身份键必须保存在仓库外")
args.state.mkdir(parents=True, exist_ok=True)
config = yaml.safe_load(args.config.read_text(encoding="utf-8"))["data"]["database"]
manifest_path = args.state / "report-baseline.json"
if args.action == "backup":
if manifest_path.exists():
raise ValueError("已有基线备份,禁止覆盖")
manifest = {"date": args.date, "databases": {}}
with connect(config) as conn:
manifest["server"] = query(conn, "SELECT @@hostname hostname,@@version version")[0]
for database in ("sw8", "sw9"):
baseline = snapshot(conn, database, args.date)
options = client_options(config, args.state)
file = args.state / (database + "-report-partition.sql.gz")
try:
with (args.state / (database + "-dump.stderr.log")).open("wb") as log:
process = subprocess.Popen([shutil.which("mysqldump"), "--defaults-extra-file=" + str(options), "--single-transaction", "--quick", "--hex-blob",
"--no-create-info", "--skip-triggers", "--skip-add-locks", "--skip-disable-keys", "--complete-insert", "--set-gtid-purged=OFF",
"--where=stat_date='" + args.date + "'", database, *TABLES], stdout=subprocess.PIPE, stderr=log)
with gzip.open(file, "wb") as target:
shutil.copyfileobj(process.stdout, target)
if process.wait() != 0:
raise RuntimeError("报表备份失败")
with gzip.open(file, "rb") as stream:
while stream.read(1024 * 1024):
pass
manifest["databases"][database] = {"tables": baseline, "backup": str(file), "sha256": hashlib.sha256(file.read_bytes()).hexdigest()}
finally:
options.unlink(missing_ok=True)
write_json(manifest_path, manifest)
print(json.dumps({"date": args.date, "tables": {db: value["tables"] for db, value in manifest["databases"].items()}}, ensure_ascii=False), flush=True)
return
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
if manifest["date"] != args.date:
raise ValueError("日期与备份不符")
with connect(config) as conn:
if query(conn, "SELECT @@hostname hostname,@@version version")[0] != manifest["server"]:
raise ValueError("数据库服务器与备份不符")
if args.action == "restore":
for database, value in manifest["databases"].items():
if database not in ("sw8", "sw9"):
raise ValueError("未知数据库")
file = Path(value["backup"])
if hashlib.sha256(file.read_bytes()).hexdigest() != value["sha256"]:
raise ValueError("备份校验失败")
content = gzip.decompress(file.read_bytes())
if b"LOCK TABLES" in content or b"ALTER TABLE" in content or b"CREATE TABLE" in content:
raise ValueError("分区恢复文件不允许包含 DDL 或隐式提交")
deletes = "\n".join(f"DELETE FROM `{table}` WHERE stat_date='{args.date}';" for table in TABLES)
script = ("START TRANSACTION;\n" + deletes + "\n").encode() + content + b"\nCOMMIT;\n"
options = client_options(config, args.state)
try:
completed = subprocess.run([shutil.which("mysql"), "--defaults-extra-file=" + str(options), "--batch", database], input=script, capture_output=True)
(args.state / (database + "-restore.stderr.log")).write_bytes(completed.stderr)
if completed.returncode:
raise RuntimeError("分区恢复失败")
finally:
options.unlink(missing_ok=True)
with connect(config) as conn:
checks = [{"database": db, "passed": snapshot(conn, db, args.date) == value["tables"]} for db, value in manifest["databases"].items()]
else:
with connect(config) as conn:
source, target = [snapshot(conn, db, args.date, logical=True) for db in ("sw8", "sw9")]
checks = [{"table": table, "source": source[table], "target": target[table], "passed": source[table] == target[table]} for table in TABLES]
for db in ("sw8", "sw9"):
for table in TABLES:
rows = query(conn, f"SELECT * FROM `{db}`.`{table}` WHERE stat_date=%s ORDER BY id", (args.date,))
write_json(args.state / (db + "-" + table + "-rebuilt.json"), rows)
result = {"action": args.action, "date": args.date, "checks": checks, "passed": all(item["passed"] for item in checks)}
write_json(args.output or args.state / (args.action + ".json"), result)
print(json.dumps(result, ensure_ascii=False), flush=True)
if not result["passed"]:
raise SystemExit(1)
if __name__ == "__main__":
main()
-58
View File
@@ -1,58 +0,0 @@
"""通过新商务原生任务 HTTP 重跑指定日期,要求已有分区备份并保持任务禁用。"""
import argparse
import json
import time
from pathlib import Path
from sw_write_verify import Rehearsal
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--config", type=Path, default=Path("configs/config.yaml"))
parser.add_argument("--old-url", default="http://127.0.0.1:18080")
parser.add_argument("--new-url", default="http://127.0.0.1:19001")
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--date", default="2026-11-03")
parser.add_argument("--backup", type=Path, required=True)
parser.add_argument("--methods", nargs="+", choices=("swReportDayBackfillTask", "swReportServerDayBackfillTask", "alertTriggerTask"),
default=["swReportDayBackfillTask", "swReportServerDayBackfillTask"])
args = parser.parse_args()
backup = json.loads(args.backup.read_text(encoding="utf-8"))
if backup["date"] != args.date:
raise ValueError("已有备份的日期与任务不符")
run = Rehearsal(args)
ids = []
try:
for method in args.methods:
name = "迁移核对-" + method + "-" + str(int(time.time()))
run.call("new", "admin", "POST", "/timedTask/createTimedTask", {"name": name, "spec": "@daily", "executorType": "method", "methodName": method,
"params": {"startDate": args.date, "endDate": args.date, "_execution": {"timeoutSeconds": 1800}}, "enabled": False})
task = run.query("SELECT id,enabled FROM sw9.sys_timed_tasks WHERE name=%s AND deleted_at IS NULL", (name,))[0]
ids.append(task["id"])
(args.runtime / "report-rehearsal-task-ids.json").write_text(json.dumps(ids), encoding="utf-8")
run.check(method + " 创建且保持禁用", task["enabled"] == 0)
started = time.monotonic()
run.call("new", "admin", "POST", "/timedTask/triggerTimedTask", {"id": task["id"]})
while time.monotonic() - started < 1830:
value = run.call("new", "admin", "GET", f"/timedTask/getTimedTaskLogList?taskId={task['id']}&page=1&pageSize=1")["data"]
logs = value.get("list", [])
if logs and logs[0].get("status") not in ("running", "pending"):
(args.runtime / (method + "-execution.json")).write_text(json.dumps(logs[0], ensure_ascii=False, indent=2), encoding="utf-8")
run.check(method + " 真实执行完成", logs[0]["status"] == "success", status=logs[0]["status"], elapsed_seconds=round(time.monotonic() - started, 2))
break
time.sleep(2)
else:
run.check(method + " 等待执行完成", False, reason="timeout")
except Exception as error:
if all(check["passed"] for check in run.checks):
run.check("报表重跑执行异常", False, error_type=type(error).__name__)
raise
finally:
run.db.close()
if __name__ == "__main__":
main()
-118
View File
@@ -1,118 +0,0 @@
"""主库使用 sw8/sw9,平台写入使用独立本地 SDK 夹具;核对验密、重提、解绑与新增渠道。"""
import argparse
import json
import time
from pathlib import Path
import pymysql
from sw_api_compare import compare_data
from sw_write_verify import Rehearsal
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--config", type=Path, default=Path("configs/config.yaml"))
parser.add_argument("--old-url", default="http://127.0.0.1:18081")
parser.add_argument("--new-url", default="http://127.0.0.1:19002")
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
run = Rehearsal(args)
fixture = json.loads((args.runtime / "sdk-fixture/fixture.json").read_text())
owner = fixture["owner_id"]
local = pymysql.connect(host="127.0.0.1", port=13309, user="root", autocommit=True, cursorclass=pymysql.cursors.DictCursor)
channel = "migchannel" + str(int(time.time()))
def query(sql, params=()):
with local.cursor() as cursor:
cursor.execute(sql, params)
return cursor.fetchall()
def api_path(side, path):
return ("/system/user" if side == "legacy" else "/sw/business") + path
def bind(side, password=None, success=True, identity="testlevel3"):
return run.call(side, identity, "POST", api_path(side, "/bindPlatform"),
{"sourceDb": "lx", "account": fixture["account"], "password": password or fixture["password"]}, success=success)
def snapshot(label, status):
values = []
for side, database in (("legacy", "sw8"), ("new", "sw9")):
rows = run.query(f"SELECT user_id AS userId,source_db AS sourceDb,platform_busier_id AS platformBusierId,platform_busier_account AS platformBusierAccount,status,reject_reason AS rejectReason,review_account AS reviewAccount FROM {database}.sys_user_bind_platform WHERE source_db='lx' AND platform_busier_id=%s", (owner,))
run.check(label + " 行数和状态 " + side, len(rows) == 1 and rows[0]["status"] == status)
values.append(rows)
compared = compare_data(*values, {}, run.mapping)
run.check(label + " 字段一致", compared["semantic_equal"], differences=compared["differences"])
try:
for database in ("sw_test_sdk_old", "sw_test_sdk_new"):
for table in ("tab_promote", "tab_promote_settlement_time"):
run.check("本地平台写入夹具为空 " + database + "." + table, query(f"SELECT COUNT(*) n FROM {database}.{table}")[0]["n"] == 0)
query(f"ALTER TABLE {database}.{table} AUTO_INCREMENT=1")
for side, database in (("legacy", "sw8"), ("new", "sw9")):
run.check("夹具绑定键未占用 " + side, not run.query(f"SELECT user_id FROM {database}.sys_user_bind_platform WHERE source_db='lx' AND platform_busier_id=%s", (owner,)))
denied = bind(side, password="invalid-password", success=False)
run.check("错误密码不允许绑定 " + side, denied["code"] != (200 if side == "legacy" else 0))
denied = bind(side, success=False, identity="testlevel1")
run.check("仅三级允许绑定 " + side, denied["code"] != (200 if side == "legacy" else 0))
started = time.monotonic()
for side in ("legacy", "new"):
bind(side)
snapshot("首次绑定待审核", "3")
for side in ("legacy", "new"):
denied = bind(side, success=False)
run.check("待审核重复提交被拒绝 " + side, denied["code"] != (200 if side == "legacy" else 0))
# 仅在主库夹具模拟 OA 的审核结果,不调用外部 OA 写接口。
for database in ("sw8", "sw9"):
run.query(f"UPDATE {database}.sys_user_bind_platform SET status='2',reject_reason='测试驳回',review_account='migration-test',review_time=NOW() WHERE source_db='lx' AND platform_busier_id=%s", (owner,))
time.sleep(max(0, 5.2 - (time.monotonic() - started)))
for side in ("legacy", "new"):
bind(side)
snapshot("驳回后重提并清空旧审核信息", "3")
for database in ("sw8", "sw9"):
run.query(f"UPDATE {database}.sys_user_bind_platform SET status='0',review_account='migration-test',review_time=NOW() WHERE source_db='lx' AND platform_busier_id=%s", (owner,))
snapshot("审核通过绑定", "0")
payload = {"sourceDb": "lx", "platformBusierId": owner, "account": channel, "password": "Channel168", "registerType": 0}
for side in ("legacy", "new"):
denied = run.call(side, "testlevel1", "POST", run.prefix(side) + "/promote", payload, success=False)
run.check("保留原一级角色的新增渠道权限限制 " + side, denied["code"] != (200 if side == "legacy" else 0))
run.call(side, "admin", "POST", run.prefix(side) + "/promote", payload)
saved, settlements = [], []
for side, database in (("legacy", "sw_test_sdk_old"), ("new", "sw_test_sdk_new")):
rows = query(f"SELECT id,account,password,status,busier_id,register_type,promote_level,parent_id,top_promote_id,game_ids2 FROM {database}.tab_promote WHERE account=%s", (channel,))
run.check("源平台渠道只创建一条 " + side, len(rows) == 1)
saved.append(rows[0])
settlement = query(f"SELECT * FROM {database}.tab_promote_settlement_time WHERE promote_id=%s", (rows[0]["id"],))
run.check("七日结算及时间关系 " + side, len(settlement) == 1 and settlement[0]["day_period"] == 7 and settlement[0]["time_type"] == 0 and settlement[0]["next_count_time"] - settlement[0]["create_time"] == 7 * 86400)
relation = query(f"SELECT promote_ids FROM {database}.tab_promote_business WHERE id=%s", (owner,))[0]["promote_ids"]
run.check("平台商务渠道关系已追加 " + side, str(rows[0]["id"]) in (relation or "").split(","))
run.check("新渠道属性及兼容密码完全一致", saved[0] == saved[1] and saved[0]["status"] == -3 and saved[0]["promote_level"] == 1)
for side, database in (("legacy", "sw8"), ("new", "sw9")):
denied = run.call(side, "admin", "POST", run.prefix(side) + "/promote", payload, success=False)
run.check("重复渠道账号被拒绝 " + side, denied["code"] != (200 if side == "legacy" else 0))
run.call(side, "testlevel3", "DELETE", api_path(side, "/bindPlatform"), {"sourceDb": "lx", "platformBusierId": owner})
run.check("解绑只删除当前绑定 " + side, not run.query(f"SELECT user_id FROM {database}.sys_user_bind_platform WHERE user_id=%s AND source_db='lx' AND platform_busier_id=%s", (run.owner(side, 3), owner)))
except Exception as error:
if all(check["passed"] for check in run.checks):
run.check("SDK 验证执行异常", False, error_type=type(error).__name__)
raise
finally:
for side, database in (("legacy", "sw8"), ("new", "sw9")):
run.query(f"DELETE FROM {database}.sys_user_bind_platform WHERE user_id=%s AND source_db='lx' AND platform_busier_id=%s AND platform_busier_account=%s", (run.owner(side, 3), owner, fixture["account"]))
run.query("DELETE FROM sw9.sys_business_binding_locks WHERE source_db='lx' AND platform_busier_id=%s", (owner,))
for database in ("sw_test_sdk_old", "sw_test_sdk_new"):
rows = query(f"SELECT id FROM {database}.tab_promote WHERE account=%s", (channel,))
for row in rows:
query(f"DELETE FROM {database}.tab_promote_settlement_time WHERE promote_id=%s", (row["id"],))
query(f"DELETE FROM {database}.tab_promote WHERE account=%s", (channel,))
query(f"UPDATE {database}.tab_promote_business SET promote_ids='' WHERE id=%s AND account=%s", (owner, fixture["account"]))
remaining = sum(run.query(f"SELECT COUNT(*) n FROM {db}.sys_user_bind_platform WHERE source_db='lx' AND platform_busier_id=%s", (owner,))[0]["n"] for db in ("sw8", "sw9"))
run.check("主库 SDK 绑定夹具已清理", remaining == 0)
local.close()
run.db.close()
if __name__ == "__main__":
main()
-207
View File
@@ -1,207 +0,0 @@
"""在 sw8/sw9 专用测试账号下核对写流程;临时渠道、绑定、预警及规则在 finally 中清理。"""
import argparse
import json
import time
import urllib.error
import urllib.parse
import urllib.request
from pathlib import Path
import pymysql
import yaml
from sw_api_compare import compare_data
class Rehearsal:
def __init__(self, args):
self.args = args
self.checks = []
args.output.write_text(json.dumps({"checks": [], "passed": False, "status": "running"}) + "\n", encoding="utf-8")
self.root = args.runtime
self.users = json.loads((self.root / "synthetic-test-users.json").read_text(encoding="utf-8"))
self.tokens = {(side, identity): (self.root / f"{side}-{identity}.token").read_text().strip()
for side in ("legacy", "new") for identity in ("admin", "level3", "testlevel1", "testlevel2", "testlevel3")}
self.client = json.loads((self.root / "legacy-client.json").read_text())["client_id"]
maps = json.loads((self.root / "id-maps.json").read_text())
self.mapping = {str(row["old_id"]): str(row["new_id"]) for row in maps if row["entity"] == "user"}
config = yaml.safe_load(args.config.read_text(encoding="utf-8"))["data"]["database"]
self.db = pymysql.connect(host=config["host"], port=int(config["port"]), user=config["user"], password=config["password"],
autocommit=True, charset="utf8mb4", read_timeout=60, cursorclass=pymysql.cursors.DictCursor)
self.rule_ids = {}
self.suffix = str(int(time.time()))
self.rule_type = "mig_verify_" + self.suffix
self.account = "mig_verify_" + self.suffix
self.promote_id = 2000000000 + int(self.suffix[-7:])
self.alert_id = 9007199254740993 + int(self.suffix[-7:])
self.fixtures = []
def query(self, sql, params=()):
with self.db.cursor() as cursor:
cursor.execute(sql, params)
return cursor.fetchall()
def check(self, name, passed, **details):
self.checks.append({"check": name, "passed": bool(passed), **details})
self.args.output.write_text(json.dumps({"checks": self.checks, "passed": all(item["passed"] for item in self.checks)}, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
print(json.dumps(self.checks[-1], ensure_ascii=False), flush=True)
if not passed:
raise AssertionError(name)
def call(self, side, identity, method, path, payload=None, success=True):
old = side == "legacy"
headers = {"Content-Type": "application/json"}
headers.update({"Authorization": "Bearer " + self.tokens[side, identity], "clientid": self.client} if old else {"x-token": self.tokens[side, identity]})
request = urllib.request.Request((self.args.old_url if old else self.args.new_url) + path,
data=json.dumps(payload).encode() if payload is not None else None, method=method, headers=headers)
try:
with urllib.request.urlopen(request, timeout=60) as response:
raw = response.read()
status = response.status
except urllib.error.HTTPError as response:
raw = response.read()
status = response.code
try:
reply = json.loads(raw)
except (ValueError, UnicodeDecodeError):
self.check(path + " " + side, False, http=status, response_bytes=len(raw), error="响应不是 JSON")
if success and reply.get("code") != (200 if old else 0):
self.check(path + " " + side, False, code=reply.get("code"), message=reply.get("msg"))
return reply
def owner(self, side, level):
return self.users[level - 1]["source_id" if side == "legacy" else "target_id"]
def prefix(self, side):
return "/system" if side == "legacy" else "/sw"
def rule_snapshot(self, name):
values = [self.call(side, "testlevel1", "GET", self.prefix(side) + "/alertRule/" + str(self.rule_ids[side]))["data"] for side in ("legacy", "new")]
options = {"source_omit_fields": ["id", "createdAt", "updatedAt"], "target_omit_fields": ["id", "createdAt", "updatedAt"],
"null_empty_fields": ["momDirectionLabel"]}
result = compare_data(*values, options, self.mapping)
self.check(name, result["semantic_equal"], differences=result["differences"])
return values
def rules(self):
self.rule = {"ruleName": "迁移核对临时规则", "ruleType": self.rule_type, "monitorMetric": 3, "unloginDays": 2, "monitorScope": 1, "warningColor": "#f56c6c"}
for side, database in (("legacy", "sw8"), ("new", "sw9")):
self.call(side, "testlevel1", "POST", self.prefix(side) + "/alertRule/add", self.rule)
rows = self.query(f"SELECT id FROM {database}.sys_alert_rule WHERE creator_id=%s AND rule_type=%s", (self.owner(side, 1), self.rule_type))
self.check("新增规则 " + side, len(rows) == 1)
self.rule_ids[side] = rows[0]["id"]
self.rule_snapshot("新增规则字段一致")
for side in ("legacy", "new"):
self.call(side, "testlevel1", "PUT", self.prefix(side) + f"/alertRule/toggle/{self.rule_ids[side]}?enabled=0")
values = self.rule_snapshot("规则禁用一致")
self.check("禁用状态生效", all(row["enabled"] == 0 for row in values))
self.rule.update(monitorMetric=1, statPeriodDays=3, momDirection=2, momThreshold=12.5)
self.rule.pop("unloginDays")
for side in ("legacy", "new"):
self.call(side, "testlevel1", "PUT", self.prefix(side) + "/alertRule/edit", {**self.rule, "id": self.rule_ids[side]})
values = self.rule_snapshot("编辑保留启停和可选历史字段")
self.check("编辑未重启规则且保留未登录天数", all(row["enabled"] == 0 and row["unloginDays"] == 2 for row in values))
for side in ("legacy", "new"):
rejected = self.call(side, "testlevel2", "PUT", self.prefix(side) + f"/alertRule/toggle/{self.rule_ids[side]}?enabled=1", success=False)
self.check("二级不能修改一级规则 " + side, rejected.get("code") != (200 if side == "legacy" else 0))
self.call(side, "testlevel1", "PUT", self.prefix(side) + f"/alertRule/toggle/{self.rule_ids[side]}?enabled=1")
self.rule_snapshot("重新启用规则一致")
def prepare_alert(self):
for side, database in (("legacy", "sw8"), ("new", "sw9")):
existing = self.query(f"SELECT id FROM {database}.tab_promote WHERE id=%s AND source_db='lx'", (self.promote_id,))
self.check("临时渠道 ID 未占用 " + side, not existing)
existing = self.query(f"SELECT id FROM {database}.sys_alert_record WHERE id=%s", (self.alert_id,))
self.check("临时预警 ID 未占用 " + side, not existing)
self.fixtures.append({"side": side, "database": database, "promote_id": self.promote_id, "alert_id": self.alert_id, "account": self.account,
"owner": self.owner(side, 1), "user": self.owner(side, 3)})
(self.root / "write-fixtures.json").write_text(json.dumps(self.fixtures, indent=2), encoding="utf-8")
self.query(f"INSERT INTO {database}.tab_promote(id,source_db,account,busier_id,promote_level,status) VALUES(%s,'lx',%s,%s,1,1)", (self.promote_id, self.account, self.promote_id))
self.query(f"INSERT INTO {database}.sys_user_bind_platform(user_id,source_db,platform_busier_id,platform_busier_account,status,create_time) VALUES(%s,'lx',%s,%s,'0',NOW())", (self.owner(side, 3), self.promote_id, self.account))
self.query(f"INSERT INTO {database}.sys_alert_record(id,platform_code,platform_name,rule_id,rule_name,rule_type,monitor_metric,target_promote_id,target_promote_name,business_id,business_name,alert_content,status,alert_date,created_at,updated_at) VALUES(%s,'lx','龙翔',%s,%s,%s,1,%s,%s,%s,%s,'迁移核对临时预警',1,CURDATE(),NOW(),NOW())",
(self.alert_id, self.rule_ids[side], self.rule["ruleName"], self.rule_type, self.promote_id, self.account, self.owner(side, 1), self.users[0]["username"]))
def alert_snapshot(self, label, expected_status):
values = [self.call(side, "testlevel1", "GET", self.prefix(side) + f"/alertRecord/detail/{self.alert_id}")["data"] for side in ("legacy", "new")]
for row in values:
row["alertRecord"].pop("ruleId", None)
options = {"source_omit_fields": ["id", "createdAt", "updatedAt"], "target_omit_fields": ["id", "createdAt", "updatedAt"],
"null_empty_fields": ["actionDetail", "noteContent"]}
# 日志主键独立生成;按原操作序列比较业务字段及操作人映射。
result = compare_data(*values, options, self.mapping)
self.check(label, result["semantic_equal"] and all(row["alertRecord"]["status"] == expected_status for row in values),
status=expected_status, log_counts=[len(row["operationLogs"]) for row in values], differences=result["differences"])
def workflow(self):
self.prepare_alert()
self.alert_snapshot("临时预警初始状态一致", 1)
for side in ("legacy", "new"):
rejected = self.call(side, "level3", "GET", self.prefix(side) + f"/alertRecord/detail/{self.alert_id}", success=False)
self.check("其他商务不可查看临时预警 " + side, rejected.get("code") != (200 if side == "legacy" else 0))
steps = [("startProcess", "testlevel3", 2), ("submitAudit", "testlevel3", 3), ("reject", "testlevel1", 4),
("reProcess", "testlevel3", 2), ("submitAudit", "testlevel3", 3), ("approve", "testlevel1", 5)]
for index, (action, identity, status) in enumerate(steps):
for side in ("legacy", "new"):
self.call(side, identity, "POST", self.prefix(side) + "/alertRecord/" + action, {"alertId": str(self.alert_id), "note": "迁移核对步骤" + str(index)})
self.alert_snapshot(action + " 状态及日志一致", status)
if index == 1:
for side in ("legacy", "new"):
rejected = self.call(side, "testlevel3", "POST", self.prefix(side) + "/alertRecord/approve", {"alertId": str(self.alert_id)}, success=False)
self.check("三级商务不能审核 " + side, rejected.get("code") != (200 if side == "legacy" else 0))
for side in ("legacy", "new"):
self.call(side, "testlevel3", "POST", self.prefix(side) + "/alertRecord/addNote", {"alertId": str(self.alert_id), "noteContent": "迁移核对独立备注", "attachmentIds": []})
self.alert_snapshot("已处理记录追加备注不改变状态", 5)
def remove_rule(self):
for side, database in (("legacy", "sw8"), ("new", "sw9")):
self.call(side, "testlevel1", "DELETE", self.prefix(side) + "/alertRule/" + str(self.rule_ids[side]))
row = self.query(f"SELECT is_deleted FROM {database}.sys_alert_rule WHERE id=%s", (self.rule_ids[side],))[0]
self.check("删除规则保留历史行 " + side, row["is_deleted"] == 1)
def cleanup(self):
for fixture in reversed(self.fixtures):
database = fixture["database"]
self.query(f"DELETE FROM {database}.sys_alert_operation_log WHERE alert_id=%s", (fixture["alert_id"],))
self.query(f"DELETE FROM {database}.sys_alert_record WHERE id=%s AND target_promote_name=%s", (fixture["alert_id"], fixture["account"]))
self.query(f"DELETE FROM {database}.sys_user_bind_platform WHERE user_id=%s AND source_db='lx' AND platform_busier_id=%s AND platform_busier_account=%s", (fixture["user"], fixture["promote_id"], fixture["account"]))
self.query(f"DELETE FROM {database}.tab_promote WHERE id=%s AND source_db='lx' AND account=%s", (fixture["promote_id"], fixture["account"]))
for side, database in (("legacy", "sw8"), ("new", "sw9")):
self.query(f"DELETE FROM {database}.sys_alert_rule WHERE creator_id=%s AND rule_type=%s", (self.owner(side, 1), self.rule_type))
remaining = 0
for fixture in self.fixtures:
database = fixture["database"]
for table, condition, params in (
("sys_alert_record", "id=%s", (fixture["alert_id"],)),
("sys_alert_operation_log", "alert_id=%s", (fixture["alert_id"],)),
("tab_promote", "id=%s AND source_db='lx'", (fixture["promote_id"],)),
("sys_user_bind_platform", "user_id=%s AND source_db='lx' AND platform_busier_id=%s", (fixture["user"], fixture["promote_id"])),
):
remaining += self.query(f"SELECT COUNT(*) n FROM {database}.{table} WHERE {condition}", params)[0]["n"]
for side, database in (("legacy", "sw8"), ("new", "sw9")):
remaining += self.query(f"SELECT COUNT(*) n FROM {database}.sys_alert_rule WHERE creator_id=%s AND rule_type=%s", (self.owner(side, 1), self.rule_type))[0]["n"]
self.check("临时预警渠道绑定和规则已清理", remaining == 0, remaining=remaining)
self.db.close()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--config", type=Path, default=Path("configs/config.yaml"))
parser.add_argument("--old-url", default="http://127.0.0.1:18080")
parser.add_argument("--new-url", default="http://127.0.0.1:19001")
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
if args.runtime.resolve().is_relative_to(Path(__file__).resolve().parents[1]):
raise ValueError("测试状态必须保存在仓库外")
run = Rehearsal(args)
try:
run.rules()
run.workflow()
run.remove_rule()
finally:
run.cleanup()
if __name__ == "__main__":
main()