forked from wangziqi/gongxue-base
feat: add pocketbase dry-run report
This commit is contained in:
18
README.md
18
README.md
@@ -157,8 +157,10 @@ npm run check:refactor
|
||||
npm run check:api
|
||||
npm run check:worker
|
||||
npm run check:importer
|
||||
npm run pb:import:dry-run
|
||||
npm run pb:import:validate
|
||||
npm run test:readiness
|
||||
npm run test:pb:dry-run
|
||||
npm run test:api
|
||||
npm run test:worker:crm
|
||||
npm run test:worker:commerce
|
||||
@@ -183,6 +185,22 @@ npm run readiness:production:db
|
||||
|
||||
这个检查会阻断默认弱密钥、`CORS=*`、mock 短信、legacy 身份头、local_dev 存储、对象存储未配置、CRM insecure localhost 等生产风险;带 `:db` 的版本还会检查租户 provider 公开配置是否混入密钥、活跃短信/OAuth/支付 provider 是否缺少 `app_private.tenant_secrets`、域名是否未验证。
|
||||
|
||||
## PocketBase 迁移 Dry-Run
|
||||
|
||||
把旧 PocketBase 导出的集合 JSON 放到仓库根目录 `pb_export/` 后,先执行不写数据库的静态 dry-run:
|
||||
|
||||
```bash
|
||||
npm run pb:import:dry-run
|
||||
```
|
||||
|
||||
需要给 CI 或脚本读取时:
|
||||
|
||||
```bash
|
||||
npm run pb:import:dry-run -- --json
|
||||
```
|
||||
|
||||
dry-run 会检查导出目录、JSON 形态、核心集合缺失、重复/缺失旧 ID、敏感字段、旧 schema 关系断裂和未映射集合。存在 blocker 时命令返回非 0;所有 blocker 处理完后,再执行 `npm run pb:import:json` 和 `npm run pb:import:validate`。
|
||||
|
||||
## API 模块
|
||||
|
||||
当前 API 目录:
|
||||
|
||||
@@ -158,7 +158,8 @@
|
||||
|
||||
| 能力 | 状态 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| PocketBase schema 分析 | 可联调 | `scripts/import-pocketbase` |
|
||||
| PocketBase schema/导出分析 | 可联调 | `scripts/import-pocketbase` 支持 schema summary/risk、`npm run pb:import:dry-run` 导出目录静态迁移报告 |
|
||||
| PocketBase JSON dry-run | 可联调 | 不写数据库,检查导出目录、JSON 形态、核心集合、旧 ID、敏感字段、schema relation、未映射集合和关键业务计数 |
|
||||
| 题目 JSON preview/import | 可联调 | 后端负责规范化、issue、幂等、审计 |
|
||||
| 公共题库采纳、手动同步和自动同步 | 可联调 | 平台授权后,租户可采纳公共题库并复制已发布题目快照;同步 API 和 `public-banks` worker 支持新增/更新题目、重新校验授权、跨租户拒绝、审计记录和租户自改冲突保护;已覆盖跨租户、重复采纳、采纳后组卷、同步新增题、冲突不覆盖和 worker 自动同步测试 |
|
||||
| 单词 JSON preview/import | 可联调 | 兼容旧模板 |
|
||||
@@ -179,6 +180,7 @@ npx supabase db reset
|
||||
npm audit --audit-level=high
|
||||
npm run check:refactor
|
||||
npm run test:readiness
|
||||
npm run test:pb:dry-run
|
||||
npm run test:worker:imports
|
||||
npm run test:worker:crm
|
||||
npm run test:worker:commerce
|
||||
|
||||
@@ -32,7 +32,7 @@
|
||||
| 会员与订单 | 可联调 | 下单、订单详情/状态轮询、优惠券领取/抵扣、零元订单自动开通、手工确认权限保护、激活码预检查/兑换、微信支付、支付宝、微信/支付宝发起退款、微信/支付宝退款查询确认、微信/支付宝退款通知 webhook、支付/退款补偿 worker、权益发放 | 完整资金流水对账、异常订单运营台 |
|
||||
| 登录认证 | 可联调 | 短信 mock、阿里云/腾讯云短信 adapter、迁移期 session、Supabase Auth JWT、微信小程序登录、微信网页登录、QQ 登录、手机号绑定/换绑、OAuth 配置表 | 真实生产账号和回调域名联调 |
|
||||
| 销售/代理/CRM | 基础完成 | 邀请码、首绑保护、团队关系、销售统计、CRM 入队 | 小程序码真实生成、分佣结算、钉钉/飞书/企微 worker |
|
||||
| 内容导入 | 可联调 | 题目、单词、知识手册、分数线、视频 JSON/CSV/Excel preview/import、issue、job、审计、幂等、`executionMode=async`、imports worker、导入后复检、模板下载和字段映射 API | 字段映射 UI、真实数据 dry-run 和导入性能压测 |
|
||||
| 内容导入 | 可联调 | 题目、单词、知识手册、分数线、视频 JSON/CSV/Excel preview/import、issue、job、审计、幂等、`executionMode=async`、imports worker、导入后复检、模板下载、字段映射 API、PocketBase JSON dry-run 报告 | 字段映射 UI、真实数据 dry-run 执行验收和导入性能压测 |
|
||||
| 数据看板 | 可联调 | 租户 dashboard 聚合接口,收益、注册、学习、内容、激活码、反馈、趋势、24h 活跃、套餐销量和运营动态 | 预聚合 worker、缓存、慢 SQL 监控和销售转化看板 |
|
||||
| AI 择校推荐 | 未开始 | 暂无 | 数据上下文、AI JSON schema、报告渲染、PDF 生成 |
|
||||
| Taro 前端 | 未开始 | 旧 Web 已有新 API 适配雏形 | `apps/taro`、跨端 API client、H5/小程序页面和端到端测试 |
|
||||
@@ -77,7 +77,7 @@
|
||||
|
||||
- 生产鉴权:API 已支持 Supabase Auth JWT;继续做真实云端 Auth/JWKS 回归、RLS 深测,并在生产关闭 `x-user-id` 与 `x-platform-admin-key` 兼容入口。
|
||||
- 对象存储:上传/下载签名已接入阿里云 OSS、腾讯云 COS、Supabase Storage;上传确认、PDF/图片预览签名和 assets worker 复检已完成,继续补 PDF 渲染、视频播放防盗链、杀毒扫描和水印。
|
||||
- 真实数据 dry-run:导出 PocketBase 用户、题库、单词、知识手册、分数线、订单、权益,跑迁移和校验报告。
|
||||
- 真实数据 dry-run:导出 PocketBase 用户、题库、单词、知识手册、分数线、订单、权益,先跑 `npm run pb:import:dry-run`,再跑迁移和校验报告。
|
||||
- 生产环境配置:`.env.example` 和 `npm run readiness:production` / `npm run readiness:production:db` 已补;继续补数据库迁移流程、备份恢复、日志、告警和 API 容器部署说明。
|
||||
- Taro scaffold:建立 `apps/taro`,先完成租户解析、首页、题库、背单词、知识手册、个人中心主链路。
|
||||
|
||||
|
||||
@@ -378,6 +378,6 @@ GET /api/tenant-content/imports/post-check?jobId=<jobId>
|
||||
|
||||
## 下一步
|
||||
|
||||
- 用真实 PocketBase 全量导出数据做多轮 dry-run,并把复检报告作为上线验收材料。
|
||||
- 已补 `npm run pb:import:dry-run` 静态报告工具;拿到真实 PocketBase 全量导出数据后做多轮 dry-run,并把 dry-run 报告、正式导入报告和复检报告作为上线验收材料。
|
||||
- 前端补可视化字段映射、模板下载按钮、导入 job 轮询、逐行 issue 展示和复检结果面板。
|
||||
- 后续按大租户数据量补导入任务分页预览、抽样校验和导入性能压测。
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
| 旧 PocketBase/React 项目 | `F:\project\参考\旧题库项目` | 保留作为功能参照和迁移来源 |
|
||||
| 新 Node API | `F:\project\apps\api` | 已按 `core/features` 分层重构 |
|
||||
| Supabase/PostgreSQL 迁移 | `F:\project\supabase\migrations` | 已建立多租户和业务域表 |
|
||||
| PocketBase 数据导入 | `F:\project\scripts\import-pocketbase` | 已支持多类旧数据归一化导入与校验 |
|
||||
| PocketBase 数据导入 | `F:\project\scripts\import-pocketbase` | 已支持多类旧数据归一化导入、dry-run 报告与校验 |
|
||||
| 共享包 | `F:\project\packages\config`、`F:\project\packages\db`、`F:\project\packages\domain` | 已建立基础共享层 |
|
||||
| 旧 Web 到新 API 适配 | `F:\project\参考\旧题库项目\src\services\supabaseApi.ts` | 仅作参考;后续应在 `apps/taro` 或共享包中建立新的跨端 API client |
|
||||
|
||||
@@ -39,7 +39,7 @@
|
||||
| 租户后台 | 已建品牌、域名、设置、支付账户、登录 provider、私密密钥表、成员、审计日志、资源台账、导入台账、内容导航台账 | 不适用 | 概览、品牌、设置、域名、支付账户、登录配置、密钥掩码、活动内容、兑换码/优惠券、成员管理、权限矩阵、审计查询、内容入口/分类树/题目集合/练习蓝图维护、资源管理、题目/单词/知识手册/分数线/视频 JSON/CSV/Excel 同步/异步导入已实现 | 核心 API 集成测试含角色/权限/租户隔离/密钥不泄露/导航/组卷/资源与导入断言 | 租户配置与运营闭环可用,前端权限 UI、字段映射操作台和导入复检结果面板待补 |
|
||||
| 平台后台 | 已建 SaaS 套餐、订阅、账单、服务费、用量 | 不适用 | 租户管理、账单、收款确认、用量记录、平台管理员 Supabase JWT 鉴权已实现 | API 集成测试 | 平台收费链路骨架可用,平台审计报表/自动计费待补 |
|
||||
| 登录认证 | 已建短信验证码、会话、OAuth provider 配置表,并支持 `auth_user_id` 映射 | 旧用户映射已预留 | 短信 mock 登录、迁移期 session、Supabase JWT 验签映射、微信小程序登录主链路、微信网页登录、QQ 登录、手机号绑定/换绑已实现 | API 集成测试 | H5 Supabase Auth 可联调;真实短信/OAuth 生产账号和回调域名联调待补 |
|
||||
| 数据导入 | 已建立 importer、risk report、validate | 已覆盖多类旧集合 | 命令行导入/校验 | `pb:import:validate` | 基础工具可用,需用真实完整数据做多轮 dry-run |
|
||||
| 数据导入 | 已建立 importer、risk report、dry-run report、validate | 已覆盖多类旧集合 | 命令行 dry-run/导入/校验 | `pb:import:dry-run`、`pb:import:validate` | 基础工具可用,需拿真实完整数据执行多轮 dry-run 和导入回归 |
|
||||
| 测试体系 | 不适用 | 不适用 | 不适用 | 已新增核心 API 集成测试、租户隔离测试、权限矩阵测试、资源/题目导入测试、导入校验 | 还不是完整覆盖,支付幂等、真实导入回归、前端端到端测试仍需补 |
|
||||
|
||||
## 已实现 API 范围
|
||||
@@ -284,7 +284,7 @@ platform-admin:
|
||||
|
||||
为了先把旧项目核心业务补齐,再进入支付/短信等商用关键模块,建议按下面顺序继续:
|
||||
|
||||
1. 完善内容导入和文件上传:字段映射 UI、真实数据 dry-run、CDN 防盗链、杀毒扫描。
|
||||
1. 完善内容导入和文件上传:字段映射 UI、真实数据 dry-run 执行验收、CDN 防盗链、杀毒扫描。
|
||||
2. 补公共题库版本通知/冲突操作台、租户套餐地区/科目/题库范围限制、主题模板系统。
|
||||
3. 补学习统计增强:排行榜防刷/预聚合、断点续练、专项练习策略和更细题型分析。
|
||||
4. 补视频商用控制:深度防盗链、动态水印和播放统计。
|
||||
|
||||
@@ -98,7 +98,7 @@
|
||||
1. 排行榜增强:刷题、模考、背单词、积分排行榜主接口已有;还需防刷、日/周榜预聚合、运营后台排名看板。
|
||||
2. 账号设置完整流:绑定/更换手机号基础 API 已完成;仍缺头像上传、微信/QQ 账号合并、密码/邮箱能力。
|
||||
3. 题库导出:服务端 JSON/试卷 payload 导出、权限审计和答案脱敏已补;仍缺 PDF/Word 二进制生成、水印、资料发布和后台导出操作台。
|
||||
4. 导入扩展:题目/单词/知识手册/分数线/视频已支持 JSON、CSV 和 Excel 预览导入,并可用 `executionMode=async` 进入 imports worker;导入后复检、模板下载和字段映射 API 已补,仍缺前端字段映射 UI 和真实数据 dry-run。
|
||||
4. 导入扩展:题目/单词/知识手册/分数线/视频已支持 JSON、CSV 和 Excel 预览导入,并可用 `executionMode=async` 进入 imports worker;导入后复检、模板下载、字段映射 API 和 PocketBase JSON dry-run 报告已补,仍缺前端字段映射 UI 和真实数据执行验收。
|
||||
5. 公共题库商业化:平台公共/地区题库授权、租户快照采纳、手动同步、自动同步 worker、冲突查询和租户自改冲突保护已完成基础闭环;还需版本通知、冲突处理操作台和运营后台 UI。
|
||||
6. CRM/销售结算:CRM worker、分佣规则、结算单、审核和打款状态基础闭环已完成;仍缺轮询/定向分配、打款导出、凭证和销售结算看板。
|
||||
7. 题目反馈增强:处理通知、消息提醒、问题聚合统计和内容修复闭环。
|
||||
@@ -117,7 +117,7 @@
|
||||
|
||||
1. 微信/支付宝支付和 webhook 幂等。
|
||||
2. 对象存储 PDF 预览、视频深度防盗链、动态水印。
|
||||
3. 字段映射 UI、真实数据 dry-run 和导入复检结果操作台。
|
||||
3. 字段映射 UI、真实数据 dry-run 执行验收和导入复检结果操作台。
|
||||
4. 数据看板预聚合 worker、销售/代理转化看板和分佣结算。
|
||||
5. 公共题库版本通知、冲突处理操作台和租户确认/跳过策略。
|
||||
|
||||
|
||||
@@ -109,10 +109,13 @@ PORT=8787
|
||||
把 PocketBase 导出的集合 JSON 放到仓库根目录的 `pb_export` 文件夹后执行:
|
||||
|
||||
```bash
|
||||
npm run pb:import:dry-run
|
||||
npm run pb:import:json
|
||||
npm run pb:import:validate
|
||||
```
|
||||
|
||||
`pb:import:dry-run` 不写数据库,会先检查 JSON 格式、核心集合、旧 ID、敏感字段、关系断裂和未映射集合。存在 blocker 时先处理问题,不要直接正式导入。
|
||||
|
||||
导入真实密钥时必须明确打开开关,且只允许进入 `app_private.tenant_secrets`:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -52,7 +52,7 @@
|
||||
|
||||
3. 真实导入 dry-run
|
||||
- 从 PocketBase 导出现有用户、题库、单词、知识手册、分数线、订单、权益数据。
|
||||
- 跑 `scripts/import-pocketbase`,生成迁移报告。
|
||||
- 已补 `npm run pb:import:dry-run` 静态迁移报告工具;拿到真实导出后先跑 dry-run,再跑 `pb:import:json` 和 `pb:import:validate`。
|
||||
- 对题目 JSON、单词、知识手册、分数线、视频走后端 preview/import API 做二次验证。
|
||||
|
||||
4. 部署配置
|
||||
@@ -78,7 +78,7 @@
|
||||
3. 导入体系扩展
|
||||
- 已完成题目、单词、知识手册、分数线、视频的 CSV/Excel 到规范 JSON 解析适配。
|
||||
- 已完成大批量异步导入 worker、重试、导入后复检和审计。
|
||||
- 模板下载、字段映射 API 和导入后复检已补;继续补字段映射 UI、真实数据 dry-run、导入前抽样校验和复检结果操作台。
|
||||
- 模板下载、字段映射 API、导入后复检和 PocketBase JSON dry-run 报告工具已补;继续补字段映射 UI、真实数据执行验收、导入前抽样校验和复检结果操作台。
|
||||
|
||||
4. 题库导出增强
|
||||
- 已完成服务端 JSON、`paper_json`、`print_payload` 基础导出和导出 job 台账。
|
||||
|
||||
@@ -67,7 +67,30 @@ pb_export/
|
||||
questions.json
|
||||
```
|
||||
|
||||
执行导入:
|
||||
先执行不写数据库的 dry-run:
|
||||
|
||||
```bash
|
||||
npm run import:dry-run
|
||||
```
|
||||
|
||||
根目录也可以执行:
|
||||
|
||||
```bash
|
||||
npm run pb:import:dry-run
|
||||
```
|
||||
|
||||
dry-run 会检查:
|
||||
|
||||
- 导出目录和 JSON 文件格式。
|
||||
- 核心集合是否缺失。
|
||||
- 旧记录 `id` 是否缺失或重复。
|
||||
- 旧 `pb_schema.json` 中的 relation 是否能在导出数据里找到目标记录。
|
||||
- `settings`、`crm_config`、用户资料等位置是否存在敏感字段。
|
||||
- 哪些集合当前已有规范化 mapper,哪些集合仍未映射。
|
||||
|
||||
存在 `blocker` 时不要执行正式导入;先修导出、补 mapper 或记录业务处理结论。
|
||||
|
||||
执行正式导入:
|
||||
|
||||
```bash
|
||||
npm run import:json
|
||||
@@ -82,6 +105,7 @@ npm run import:validate
|
||||
根目录也可以执行:
|
||||
|
||||
```bash
|
||||
npm run pb:import:dry-run
|
||||
npm run pb:import:json
|
||||
npm run pb:import:validate
|
||||
```
|
||||
|
||||
@@ -22,7 +22,7 @@
|
||||
"check:api": "npm --workspace @tiku-saas/api run check",
|
||||
"check:importer": "npm --workspace @tiku-saas/import-pocketbase run check",
|
||||
"check:worker": "npm --workspace @tiku-saas/worker run check",
|
||||
"check:refactor": "npm run check:api && npm run check:worker && npm run check:importer && npm run pb:import:validate && npm run test:readiness && npm run test:api",
|
||||
"check:refactor": "npm run check:api && npm run check:worker && npm run check:importer && npm run pb:import:validate && npm run test:readiness && npm run test:pb:dry-run && npm run test:api",
|
||||
"docker:api:build": "docker compose -f docker-compose.api.yml build",
|
||||
"docker:api:up": "docker compose -f docker-compose.api.yml up api",
|
||||
"docker:api:down": "docker compose -f docker-compose.api.yml down",
|
||||
@@ -39,11 +39,13 @@
|
||||
"test:worker:imports": "npm run db:smoke-seed && npm run build:worker && node scripts/import-worker-integration-test.js",
|
||||
"test:worker:public-banks": "npm run db:smoke-seed && npm run build:worker && node scripts/public-bank-worker-integration-test.js",
|
||||
"test:readiness": "node scripts/production-readiness-check-test.js",
|
||||
"test:pb:dry-run": "node scripts/pb-dry-run-report-test.js",
|
||||
"readiness:production": "node scripts/production-readiness-check.js --skip-db",
|
||||
"readiness:production:db": "node scripts/production-readiness-check.js --check-db",
|
||||
"test:api:remote": "node scripts/api-integration-test.js",
|
||||
"pb:schema:summary": "npm --workspace @tiku-saas/import-pocketbase run schema:summary",
|
||||
"pb:schema:risk": "npm --workspace @tiku-saas/import-pocketbase run schema:risk",
|
||||
"pb:import:dry-run": "npm --workspace @tiku-saas/import-pocketbase run import:dry-run --",
|
||||
"pb:import:json": "npm --workspace @tiku-saas/import-pocketbase run import:json",
|
||||
"pb:import:validate": "npm --workspace @tiku-saas/import-pocketbase run import:validate"
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
"scripts": {
|
||||
"schema:summary": "tsx src/analyze-schema.ts",
|
||||
"schema:risk": "tsx src/risk-report.ts",
|
||||
"import:dry-run": "tsx src/dry-run-report.ts",
|
||||
"import:json": "tsx src/import-json.ts",
|
||||
"import:validate": "tsx src/validate-import.ts",
|
||||
"check": "tsc -p tsconfig.json --noEmit"
|
||||
|
||||
523
scripts/import-pocketbase/src/dry-run-report.ts
Normal file
523
scripts/import-pocketbase/src/dry-run-report.ts
Normal file
@@ -0,0 +1,523 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { fieldsOf, readPocketBaseSchema, type PocketBaseCollection, type PocketBaseField } from './pb-schema.js';
|
||||
import { loadEnv } from './env.js';
|
||||
|
||||
loadEnv();
|
||||
|
||||
type JsonRecord = Record<string, unknown> & { id?: unknown };
|
||||
type Severity = 'blocker' | 'warning' | 'info';
|
||||
|
||||
interface DryRunIssue {
|
||||
severity: Severity;
|
||||
code: string;
|
||||
collection?: string;
|
||||
file?: string;
|
||||
fieldPath?: string;
|
||||
message: string;
|
||||
sample?: string;
|
||||
count?: number;
|
||||
}
|
||||
|
||||
interface CollectionReport {
|
||||
collection: string;
|
||||
file: string;
|
||||
recordCount: number;
|
||||
missingIdCount: number;
|
||||
duplicateIdCount: number;
|
||||
fieldCount: number;
|
||||
sampleIds: string[];
|
||||
supported: boolean;
|
||||
}
|
||||
|
||||
interface DryRunReport {
|
||||
exportDir: string;
|
||||
schemaPath?: string;
|
||||
generatedAt: string;
|
||||
summary: {
|
||||
files: number;
|
||||
collections: number;
|
||||
records: number;
|
||||
supportedCollections: number;
|
||||
unsupportedCollections: number;
|
||||
blockers: number;
|
||||
warnings: number;
|
||||
infos: number;
|
||||
};
|
||||
businessCounts: Record<string, number>;
|
||||
collections: CollectionReport[];
|
||||
issues: DryRunIssue[];
|
||||
}
|
||||
|
||||
const args = new Set(process.argv.slice(2));
|
||||
const jsonOutput = args.has('--json');
|
||||
const failOnWarnings = args.has('--fail-on-warnings') || process.env.PB_DRY_RUN_FAIL_ON_WARNINGS === 'true';
|
||||
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../..');
|
||||
const exportDir = path.resolve(repoRoot, process.env.PB_EXPORT_DIR || 'pb_export');
|
||||
|
||||
const supportedCollections = new Set([
|
||||
'announcements',
|
||||
'app_assets',
|
||||
'audit_logs',
|
||||
'badges',
|
||||
'banners',
|
||||
'categories',
|
||||
'code_batches',
|
||||
'codes',
|
||||
'coupons',
|
||||
'coupon_redemptions',
|
||||
'crm_config',
|
||||
'crm_webhook_log',
|
||||
'crm_webhook_queue',
|
||||
'dashboard_daily_stats',
|
||||
'exam_dates',
|
||||
'faqs',
|
||||
'handbook_chapters',
|
||||
'handbook_entries',
|
||||
'handbook_subjects',
|
||||
'images',
|
||||
'majors',
|
||||
'module_nodes',
|
||||
'orders',
|
||||
'products',
|
||||
'question_type_groups',
|
||||
'question_videos',
|
||||
'questions',
|
||||
'recent_practices',
|
||||
'referral_tracks',
|
||||
'region_modules',
|
||||
'regions',
|
||||
'reports',
|
||||
'revenue_daily_stats',
|
||||
'schools',
|
||||
'scoreline_fields',
|
||||
'scoreline_majors',
|
||||
'scoreline_records',
|
||||
'scoreline_schools',
|
||||
'settings',
|
||||
'subject_shares',
|
||||
'subjects',
|
||||
'svip_plans',
|
||||
'timelines',
|
||||
'user_badges',
|
||||
'user_word_favorites',
|
||||
'user_word_progress',
|
||||
'users',
|
||||
'video_explanations',
|
||||
'vocabulary',
|
||||
'vocabulary_units',
|
||||
]);
|
||||
|
||||
const coreCollections = [
|
||||
'users',
|
||||
'questions',
|
||||
'subjects',
|
||||
'categories',
|
||||
'orders',
|
||||
'svip_plans',
|
||||
'codes',
|
||||
'vocabulary_units',
|
||||
'vocabulary',
|
||||
'handbook_subjects',
|
||||
'handbook_chapters',
|
||||
'handbook_entries',
|
||||
];
|
||||
|
||||
const businessCollections = [
|
||||
'users',
|
||||
'questions',
|
||||
'orders',
|
||||
'svip_plans',
|
||||
'codes',
|
||||
'vocabulary_units',
|
||||
'vocabulary',
|
||||
'handbook_subjects',
|
||||
'handbook_chapters',
|
||||
'handbook_entries',
|
||||
'scoreline_schools',
|
||||
'scoreline_majors',
|
||||
'scoreline_records',
|
||||
'video_explanations',
|
||||
'question_videos',
|
||||
'referral_tracks',
|
||||
'coupons',
|
||||
'badges',
|
||||
];
|
||||
|
||||
const sensitiveKeyPattern =
|
||||
/(password|token|secret|privatekey|sessionkey|accesskey|appkey|apikey|api_v3_key|notifytoken|aeskey|openid|unionid|wxaccesstoken|wechatsessionkey|smscode|verifycode|verificationcode|captcha)/i;
|
||||
const highRiskCollections = new Set(['settings', 'crm_config', 'smscodes']);
|
||||
|
||||
function issue(issues: DryRunIssue[], nextIssue: DryRunIssue) {
|
||||
issues.push(nextIssue);
|
||||
}
|
||||
|
||||
function asRecords(input: unknown): JsonRecord[] | null {
|
||||
if (Array.isArray(input)) return input as JsonRecord[];
|
||||
if (input && typeof input === 'object') {
|
||||
const candidate = input as { items?: unknown; records?: unknown };
|
||||
if (Array.isArray(candidate.items)) return candidate.items as JsonRecord[];
|
||||
if (Array.isArray(candidate.records)) return candidate.records as JsonRecord[];
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function collectionNameFromFile(fileName: string) {
|
||||
return fileName.replace(/\.json$/i, '');
|
||||
}
|
||||
|
||||
function text(value: unknown) {
|
||||
if (value === null || value === undefined) return '';
|
||||
return String(value).trim();
|
||||
}
|
||||
|
||||
function readJsonFile(filePath: string) {
|
||||
return JSON.parse(fs.readFileSync(filePath, 'utf8')) as unknown;
|
||||
}
|
||||
|
||||
function scalarSample(value: unknown) {
|
||||
if (value === null || value === undefined) return '';
|
||||
if (typeof value === 'object') return JSON.stringify(value).slice(0, 160);
|
||||
return String(value).slice(0, 160);
|
||||
}
|
||||
|
||||
function walkSensitiveFields(
|
||||
issues: DryRunIssue[],
|
||||
collection: string,
|
||||
value: unknown,
|
||||
pathParts: string[] = [],
|
||||
depth = 0,
|
||||
) {
|
||||
if (!value || typeof value !== 'object' || depth > 8) return;
|
||||
if (Array.isArray(value)) {
|
||||
value.slice(0, 10).forEach((item, index) => walkSensitiveFields(issues, collection, item, [...pathParts, String(index)], depth + 1));
|
||||
return;
|
||||
}
|
||||
|
||||
for (const [key, childValue] of Object.entries(value as Record<string, unknown>)) {
|
||||
const fieldPath = [...pathParts, key].join('.');
|
||||
if (sensitiveKeyPattern.test(key) && childValue !== null && childValue !== undefined && childValue !== '') {
|
||||
issue(issues, {
|
||||
severity: highRiskCollections.has(collection) ? 'blocker' : 'warning',
|
||||
code: 'sensitive_field_in_export',
|
||||
collection,
|
||||
fieldPath,
|
||||
message: `Legacy export contains sensitive-looking field "${fieldPath}". It must not be copied into public normalized tables.`,
|
||||
sample: scalarSample(childValue),
|
||||
});
|
||||
}
|
||||
walkSensitiveFields(issues, collection, childValue, [...pathParts, key], depth + 1);
|
||||
}
|
||||
}
|
||||
|
||||
function fieldValues(record: JsonRecord, field: PocketBaseField) {
|
||||
const value = record[field.name];
|
||||
if (value === null || value === undefined || value === '') return [];
|
||||
if (Array.isArray(value)) return value.map(item => text(item)).filter(Boolean);
|
||||
return [text(value)].filter(Boolean);
|
||||
}
|
||||
|
||||
function relationTargetName(field: PocketBaseField, byId: Map<string, PocketBaseCollection>) {
|
||||
if (!field.collectionId) return '';
|
||||
return byId.get(field.collectionId)?.name || '';
|
||||
}
|
||||
|
||||
function validateRelations(
|
||||
issues: DryRunIssue[],
|
||||
collection: PocketBaseCollection,
|
||||
records: JsonRecord[],
|
||||
idIndex: Map<string, Set<string>>,
|
||||
byId: Map<string, PocketBaseCollection>,
|
||||
) {
|
||||
const relationFields = fieldsOf(collection).filter(field => field.type === 'relation');
|
||||
for (const field of relationFields) {
|
||||
const targetName = relationTargetName(field, byId);
|
||||
if (!targetName) {
|
||||
issue(issues, {
|
||||
severity: 'warning',
|
||||
code: 'relation_target_schema_missing',
|
||||
collection: collection.name,
|
||||
fieldPath: field.name,
|
||||
message: `Relation field "${field.name}" points to an unknown collection id in pb_schema.json.`,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const targetIds = idIndex.get(targetName);
|
||||
if (!targetIds) {
|
||||
const hasValues = records.some(record => fieldValues(record, field).length > 0);
|
||||
if (hasValues) {
|
||||
issue(issues, {
|
||||
severity: coreCollections.includes(collection.name) ? 'blocker' : 'warning',
|
||||
code: 'relation_target_export_missing',
|
||||
collection: collection.name,
|
||||
fieldPath: field.name,
|
||||
message: `Relation field "${field.name}" references "${targetName}", but ${targetName}.json is not present in export.`,
|
||||
});
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
let unresolved = 0;
|
||||
let sample = '';
|
||||
for (const record of records) {
|
||||
for (const value of fieldValues(record, field)) {
|
||||
if (!targetIds.has(value)) {
|
||||
unresolved += 1;
|
||||
if (!sample) sample = `${text(record.id) || '(missing id)'} -> ${value}`;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (unresolved > 0) {
|
||||
issue(issues, {
|
||||
severity: coreCollections.includes(collection.name) ? 'blocker' : 'warning',
|
||||
code: 'relation_unresolved',
|
||||
collection: collection.name,
|
||||
fieldPath: field.name,
|
||||
message: `Relation field "${field.name}" has ${unresolved} unresolved reference(s) to "${targetName}".`,
|
||||
count: unresolved,
|
||||
sample,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function loadSchema(issues: DryRunIssue[]) {
|
||||
try {
|
||||
return readPocketBaseSchema();
|
||||
} catch (error) {
|
||||
issue(issues, {
|
||||
severity: 'warning',
|
||||
code: 'schema_unavailable',
|
||||
message: `pb_schema.json could not be loaded; relation checks will be skipped. ${error instanceof Error ? error.message : ''}`.trim(),
|
||||
});
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function collectFieldNames(records: JsonRecord[]) {
|
||||
const fields = new Set<string>();
|
||||
for (const record of records.slice(0, 500)) {
|
||||
Object.keys(record).forEach(key => fields.add(key));
|
||||
}
|
||||
return fields;
|
||||
}
|
||||
|
||||
function buildReport(): DryRunReport {
|
||||
const issues: DryRunIssue[] = [];
|
||||
const schema = loadSchema(issues);
|
||||
const collectionByName = schema?.byName || new Map<string, PocketBaseCollection>();
|
||||
const files = fs.existsSync(exportDir) ? fs.readdirSync(exportDir).filter(file => file.toLowerCase().endsWith('.json')) : [];
|
||||
const collections: CollectionReport[] = [];
|
||||
const recordsByCollection = new Map<string, JsonRecord[]>();
|
||||
const idIndex = new Map<string, Set<string>>();
|
||||
|
||||
if (!fs.existsSync(exportDir)) {
|
||||
issue(issues, {
|
||||
severity: 'blocker',
|
||||
code: 'export_dir_missing',
|
||||
message: `Export dir not found: ${exportDir}`,
|
||||
});
|
||||
}
|
||||
|
||||
if (files.length === 0) {
|
||||
issue(issues, {
|
||||
severity: 'blocker',
|
||||
code: 'export_files_missing',
|
||||
message: `No PocketBase collection JSON files found in ${exportDir}`,
|
||||
});
|
||||
}
|
||||
|
||||
for (const file of files.sort()) {
|
||||
const collection = collectionNameFromFile(file);
|
||||
const filePath = path.join(exportDir, file);
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = readJsonFile(filePath);
|
||||
} catch (error) {
|
||||
issue(issues, {
|
||||
severity: 'blocker',
|
||||
code: 'json_parse_failed',
|
||||
collection,
|
||||
file,
|
||||
message: `Failed to parse ${file}: ${error instanceof Error ? error.message : String(error)}`,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const records = asRecords(parsed);
|
||||
if (!records) {
|
||||
issue(issues, {
|
||||
severity: 'blocker',
|
||||
code: 'collection_shape_invalid',
|
||||
collection,
|
||||
file,
|
||||
message: `${file} must be an array or an object with items/records array.`,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const ids = new Set<string>();
|
||||
const duplicateIds = new Set<string>();
|
||||
let missingIdCount = 0;
|
||||
for (const record of records) {
|
||||
const id = text(record.id);
|
||||
if (!id) {
|
||||
missingIdCount += 1;
|
||||
continue;
|
||||
}
|
||||
if (ids.has(id)) duplicateIds.add(id);
|
||||
ids.add(id);
|
||||
}
|
||||
|
||||
recordsByCollection.set(collection, records);
|
||||
idIndex.set(collection, ids);
|
||||
const fields = collectFieldNames(records);
|
||||
collections.push({
|
||||
collection,
|
||||
file,
|
||||
recordCount: records.length,
|
||||
missingIdCount,
|
||||
duplicateIdCount: duplicateIds.size,
|
||||
fieldCount: fields.size,
|
||||
sampleIds: [...ids].slice(0, 5),
|
||||
supported: supportedCollections.has(collection),
|
||||
});
|
||||
|
||||
if (!supportedCollections.has(collection) && !collection.startsWith('_')) {
|
||||
issue(issues, {
|
||||
severity: 'warning',
|
||||
code: 'unsupported_collection',
|
||||
collection,
|
||||
file,
|
||||
message: `${collection}.json is not currently normalized by import-json.ts. It will be preserved only if a future mapper is added.`,
|
||||
count: records.length,
|
||||
});
|
||||
}
|
||||
if (missingIdCount > 0) {
|
||||
issue(issues, {
|
||||
severity: coreCollections.includes(collection) ? 'blocker' : 'warning',
|
||||
code: 'record_id_missing',
|
||||
collection,
|
||||
file,
|
||||
message: `${collection} has ${missingIdCount} record(s) without id.`,
|
||||
count: missingIdCount,
|
||||
});
|
||||
}
|
||||
if (duplicateIds.size > 0) {
|
||||
issue(issues, {
|
||||
severity: 'blocker',
|
||||
code: 'record_id_duplicate',
|
||||
collection,
|
||||
file,
|
||||
message: `${collection} has duplicate legacy id(s).`,
|
||||
count: duplicateIds.size,
|
||||
sample: [...duplicateIds].slice(0, 5).join(', '),
|
||||
});
|
||||
}
|
||||
|
||||
records.slice(0, 200).forEach(record => walkSensitiveFields(issues, collection, record));
|
||||
}
|
||||
|
||||
for (const collection of coreCollections) {
|
||||
if (!recordsByCollection.has(collection)) {
|
||||
issue(issues, {
|
||||
severity: 'warning',
|
||||
code: 'core_collection_missing',
|
||||
collection,
|
||||
message: `${collection}.json is not present. If old production has this collection, export it before migration.`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (schema) {
|
||||
for (const [collectionName, records] of recordsByCollection.entries()) {
|
||||
const collection = collectionByName.get(collectionName);
|
||||
if (collection) validateRelations(issues, collection, records, idIndex, schema.byId);
|
||||
}
|
||||
}
|
||||
|
||||
const businessCounts = Object.fromEntries(
|
||||
businessCollections.map(collection => [collection, recordsByCollection.get(collection)?.length || 0]),
|
||||
);
|
||||
|
||||
const supportedCount = collections.filter(item => item.supported).length;
|
||||
const unsupportedCount = collections.length - supportedCount;
|
||||
const recordCount = collections.reduce((sum, item) => sum + item.recordCount, 0);
|
||||
const blockers = issues.filter(item => item.severity === 'blocker').length;
|
||||
const warnings = issues.filter(item => item.severity === 'warning').length;
|
||||
const infos = issues.filter(item => item.severity === 'info').length;
|
||||
|
||||
return {
|
||||
exportDir,
|
||||
schemaPath: schema?.absPath,
|
||||
generatedAt: new Date().toISOString(),
|
||||
summary: {
|
||||
files: files.length,
|
||||
collections: collections.length,
|
||||
records: recordCount,
|
||||
supportedCollections: supportedCount,
|
||||
unsupportedCollections: unsupportedCount,
|
||||
blockers,
|
||||
warnings,
|
||||
infos,
|
||||
},
|
||||
businessCounts,
|
||||
collections: collections.sort((a, b) => a.collection.localeCompare(b.collection)),
|
||||
issues: issues.sort((a, b) => {
|
||||
const order: Record<Severity, number> = { blocker: 0, warning: 1, info: 2 };
|
||||
return order[a.severity] - order[b.severity] || (a.collection || '').localeCompare(b.collection || '') || a.code.localeCompare(b.code);
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function printHuman(report: DryRunReport) {
|
||||
console.log('PocketBase dry-run migration report');
|
||||
console.log(`Export dir: ${report.exportDir}`);
|
||||
if (report.schemaPath) console.log(`Schema: ${report.schemaPath}`);
|
||||
console.log(
|
||||
`Summary: ${report.summary.records} records across ${report.summary.collections} collections; ` +
|
||||
`${report.summary.blockers} blocker(s), ${report.summary.warnings} warning(s)`,
|
||||
);
|
||||
console.log('');
|
||||
console.log('Business counts:');
|
||||
for (const [name, count] of Object.entries(report.businessCounts)) {
|
||||
console.log(` ${name.padEnd(24)} ${count}`);
|
||||
}
|
||||
console.log('');
|
||||
console.log('Collections:');
|
||||
for (const item of report.collections) {
|
||||
const marker = item.supported ? 'mapped' : 'unmapped';
|
||||
console.log(
|
||||
` ${item.collection.padEnd(24)} records=${String(item.recordCount).padStart(6)} ` +
|
||||
`fields=${String(item.fieldCount).padStart(3)} ${marker}`,
|
||||
);
|
||||
}
|
||||
console.log('');
|
||||
console.log('Issues:');
|
||||
if (report.issues.length === 0) {
|
||||
console.log(' none');
|
||||
} else {
|
||||
for (const item of report.issues.slice(0, 200)) {
|
||||
const location = [item.collection, item.fieldPath].filter(Boolean).join('.') || item.file || 'export';
|
||||
const count = item.count === undefined ? '' : ` (${item.count})`;
|
||||
const sample = item.sample ? ` sample=${item.sample}` : '';
|
||||
console.log(` [${item.severity.toUpperCase()}] ${item.code} ${location}: ${item.message}${count}${sample}`);
|
||||
}
|
||||
if (report.issues.length > 200) console.log(` ... ${report.issues.length - 200} more issue(s) omitted`);
|
||||
}
|
||||
}
|
||||
|
||||
function main() {
|
||||
const report = buildReport();
|
||||
if (jsonOutput) console.log(JSON.stringify(report, null, 2));
|
||||
else printHuman(report);
|
||||
|
||||
if (report.summary.blockers > 0 || (failOnWarnings && report.summary.warnings > 0)) {
|
||||
process.exitCode = 1;
|
||||
}
|
||||
}
|
||||
|
||||
main();
|
||||
122
scripts/pb-dry-run-report-test.js
Normal file
122
scripts/pb-dry-run-report-test.js
Normal file
@@ -0,0 +1,122 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
|
||||
const repoRoot = process.cwd();
|
||||
const dryRunScript = path.join(repoRoot, 'scripts', 'import-pocketbase', 'src', 'dry-run-report.ts');
|
||||
|
||||
function writeJson(filePath, value) {
|
||||
fs.writeFileSync(filePath, JSON.stringify(value, null, 2), 'utf8');
|
||||
}
|
||||
|
||||
function runDryRun(exportDir) {
|
||||
const result = spawnSync(process.execPath, ['--import', 'tsx', dryRunScript, '--json'], {
|
||||
cwd: repoRoot,
|
||||
encoding: 'utf8',
|
||||
env: {
|
||||
...process.env,
|
||||
PB_EXPORT_DIR: exportDir,
|
||||
},
|
||||
});
|
||||
|
||||
if (result.error) throw result.error;
|
||||
const stdout = result.stdout || '';
|
||||
const jsonStart = stdout.indexOf('{');
|
||||
const payload = JSON.parse(jsonStart >= 0 ? stdout.slice(jsonStart) : '{}');
|
||||
return { ...result, payload };
|
||||
}
|
||||
|
||||
function createSafeExport() {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'tiku-pb-dry-safe-'));
|
||||
writeJson(path.join(dir, 'users.json'), [
|
||||
{ id: 'u1', phone: '13800000000', role: 'student', nickname: '学生A' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'subjects.json'), [
|
||||
{ id: 's1', name: '英语' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'categories.json'), [
|
||||
{ id: 'c1', name: '阅读理解', subjectId: 's1' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'questions.json'), [
|
||||
{ id: 'q1', subjectId: 's1', categoryId: 'c1', type: 'choice', content: '题干', answer: 'A' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'orders.json'), [
|
||||
{ id: 'o1', userId: 'u1', amount: 99, status: 'paid' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'svip_plans.json'), [
|
||||
{ id: 'p1', name: '题库会员', price: 99 },
|
||||
]);
|
||||
writeJson(path.join(dir, 'codes.json'), [
|
||||
{ id: 'code1', code: 'SAFE-001', status: 'unused' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'vocabulary_units.json'), [
|
||||
{ id: 'vu1', name: 'Unit 1' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'vocabulary.json'), [
|
||||
{ id: 'vw1', unitId: 'vu1', word: 'abandon', meaning: '放弃' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'handbook_subjects.json'), [
|
||||
{ id: 'hs1', name: '政治' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'handbook_chapters.json'), [
|
||||
{ id: 'hc1', subjectId: 'hs1', title: '第一章' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'handbook_entries.json'), [
|
||||
{ id: 'he1', chapterId: 'hc1', title: '知识点', content: '内容' },
|
||||
]);
|
||||
return dir;
|
||||
}
|
||||
|
||||
function createUnsafeExport() {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'tiku-pb-dry-unsafe-'));
|
||||
writeJson(path.join(dir, 'users.json'), [
|
||||
{ id: 'u1', phone: '13800000000', password: 'plaintext-should-not-exist' },
|
||||
{ phone: '13900000000' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'questions.json'), [
|
||||
{ id: 'q1', subjectId: 'missing-subject', content: '题干' },
|
||||
{ id: 'q1', subjectId: 'missing-subject', content: '重复题' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'settings.json'), [
|
||||
{ id: 'settings1', appSecret: 'secret-in-public-export' },
|
||||
]);
|
||||
writeJson(path.join(dir, 'unknown_business.json'), [
|
||||
{ id: 'x1', name: '尚未映射集合' },
|
||||
]);
|
||||
return dir;
|
||||
}
|
||||
|
||||
const safeDir = createSafeExport();
|
||||
const safe = runDryRun(safeDir);
|
||||
assert.equal(safe.status, 0, `safe dry-run should pass: ${safe.stdout} ${safe.stderr}`);
|
||||
assert.equal(safe.payload.summary?.blockers, 0, 'safe dry-run should have no blockers');
|
||||
assert.ok(safe.payload.summary?.records >= 12, 'safe dry-run should count fixture records');
|
||||
assert.equal(safe.payload.businessCounts?.questions, 1, 'safe dry-run should report question count');
|
||||
|
||||
const unsafeDir = createUnsafeExport();
|
||||
const unsafe = runDryRun(unsafeDir);
|
||||
assert.notEqual(unsafe.status, 0, 'unsafe dry-run should fail');
|
||||
assert.ok(unsafe.payload.summary?.blockers > 0, 'unsafe dry-run should report blockers');
|
||||
assert.ok(
|
||||
unsafe.payload.issues?.some(item => item.code === 'record_id_missing' && item.collection === 'users'),
|
||||
'unsafe dry-run should detect missing user id',
|
||||
);
|
||||
assert.ok(
|
||||
unsafe.payload.issues?.some(item => item.code === 'record_id_duplicate' && item.collection === 'questions'),
|
||||
'unsafe dry-run should detect duplicate question id',
|
||||
);
|
||||
assert.ok(
|
||||
unsafe.payload.issues?.some(item => item.code === 'sensitive_field_in_export'),
|
||||
'unsafe dry-run should detect sensitive fields',
|
||||
);
|
||||
assert.ok(
|
||||
unsafe.payload.issues?.some(item => item.code === 'unsupported_collection'),
|
||||
'unsafe dry-run should warn about unsupported collections',
|
||||
);
|
||||
|
||||
fs.rmSync(safeDir, { recursive: true, force: true });
|
||||
fs.rmSync(unsafeDir, { recursive: true, force: true });
|
||||
|
||||
console.log('[PASS] PocketBase dry-run report');
|
||||
@@ -87,6 +87,109 @@ async function main() {
|
||||
try {
|
||||
await client.query('begin');
|
||||
|
||||
await client.query(
|
||||
`
|
||||
create temporary table smoke_seed_transient_questions (
|
||||
id uuid primary key
|
||||
) on commit drop
|
||||
`,
|
||||
);
|
||||
|
||||
await client.query(
|
||||
`
|
||||
insert into smoke_seed_transient_questions (id)
|
||||
select distinct q.id
|
||||
from public.questions q
|
||||
left join public.question_collection_items ci
|
||||
on ci.tenant_id = q.tenant_id
|
||||
and ci.question_id = q.id
|
||||
where q.tenant_id = $1
|
||||
and q.id <> all($3::uuid[])
|
||||
and (
|
||||
q.primary_collection_id = $2::uuid
|
||||
or ci.collection_id = $2::uuid
|
||||
or coalesce(q.legacy_id, '') like any($4::text[])
|
||||
)
|
||||
`,
|
||||
[
|
||||
tenantId,
|
||||
ids.questionCollection,
|
||||
[ids.question, ids.questionTwo, ids.questionThree],
|
||||
['integration-import-%', 'worker-import-question-%', 'public-sync-source-%', 'content-import:%'],
|
||||
],
|
||||
);
|
||||
|
||||
await client.query(
|
||||
`
|
||||
delete from public.question_collection_items
|
||||
where tenant_id = $1
|
||||
and (
|
||||
question_id in (select id from smoke_seed_transient_questions)
|
||||
or (
|
||||
collection_id = $2::uuid
|
||||
and question_id <> all($3::uuid[])
|
||||
)
|
||||
)
|
||||
`,
|
||||
[tenantId, ids.questionCollection, [ids.question, ids.questionTwo, ids.questionThree]],
|
||||
);
|
||||
|
||||
await client.query(
|
||||
`
|
||||
delete from public.question_versions
|
||||
where tenant_id = $1
|
||||
and question_id in (select id from smoke_seed_transient_questions)
|
||||
`,
|
||||
[tenantId],
|
||||
);
|
||||
|
||||
await client.query(
|
||||
`
|
||||
delete from public.questions
|
||||
where tenant_id = $1
|
||||
and id in (select id from smoke_seed_transient_questions)
|
||||
`,
|
||||
[tenantId],
|
||||
);
|
||||
|
||||
await client.query(
|
||||
`
|
||||
delete from public.content_export_jobs
|
||||
where tenant_id = $1
|
||||
and scope_id = $2::uuid
|
||||
`,
|
||||
[tenantId, ids.questionCollection],
|
||||
);
|
||||
|
||||
await client.query(
|
||||
`
|
||||
delete from public.content_import_jobs
|
||||
where tenant_id = $1
|
||||
and (
|
||||
source_name = any($2::text[])
|
||||
or source_name like 'worker-import-%'
|
||||
)
|
||||
`,
|
||||
[
|
||||
tenantId,
|
||||
[
|
||||
'large-but-allowed-question-import.json',
|
||||
'invalid-question-import.json',
|
||||
'valid-question-import.json',
|
||||
'questions.csv',
|
||||
'async-question-import.json',
|
||||
'invalid-vocabulary-import.json',
|
||||
'legacy-vocabulary-import.json',
|
||||
'vocabulary.csv',
|
||||
'handbook-nested-import.json',
|
||||
'scoreline-batch-import.json',
|
||||
'scoreline-mixed-items.json',
|
||||
'scoreline.xlsx',
|
||||
'video-batch-import.json',
|
||||
],
|
||||
],
|
||||
);
|
||||
|
||||
await client.query(
|
||||
`
|
||||
delete from public.report_status_events
|
||||
|
||||
Reference in New Issue
Block a user