diff --git a/README.md b/README.md index f3887285..a07810ac 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ - 租户内容能力:可配置题库入口、任意深度分类树、考试意向标记、题目集合、顺序/随机/全真模拟蓝图、题目录入/更新、视频绑定、分数线、单词、知识手册、资料资源台账、题目/单词/知识手册/分数线/视频 JSON/CSV/Excel 批量导入。 - 学生端能力:题库入口、分类树、题目集合、顺序/随机/模考 session 组卷快照、答题、错题本、收藏夹、背单词卡片学习/发音/收藏练习、个人中心、站内通知、勋章、考试倒计时、签到积分、积分活动任务、积分兑换、题目反馈、排行榜、分数线、AI 择校推荐、题目视频、订单详情/状态轮询、优惠券领取/抵扣、权益、激活码预检查/兑换、资料下载;签到、积分阈值、反馈解决和积分活动可返回自动获得勋章结果,反馈处理/奖励、勋章发放和积分兑换会写入用户站内通知。 - 平台后台能力:租户管理、租户详情、账务资料维护、平台员工创建/授权/启停、平台细粒度权限点、平台审计日志查询和 CSV/JSON 导出、平台审计告警规则/开放告警查询/确认/解决、平台审计告警外部通知渠道和发送事件、SaaS 套餐、订阅、订阅账单候选预览/dry-run/批量生成、自动计费 worker、账单、服务费收款、逾期标记、内部催缴台账、平台催缴外部通知渠道和发送事件、用量记录、公共题库授权。 -- 公共题库商业化能力:租户可采纳平台授权题库为本租户副本,并可手动或由 worker 自动同步平台新增/更新题目;同步会保护租户自改题目,返回冲突而不覆盖,后台可查询冲突明细。 +- 公共题库商业化能力:租户可采纳平台授权题库为本租户副本,并可手动或由 worker 自动同步平台新增/更新题目;同步会保护租户自改题目,返回冲突而不覆盖,后台可查询冲突明细;worker 失败会生成租户 `public_question_bank_sync_failed` 通知,恢复成功自动关闭失败通知,平台可用 `/api/platform-admin/question-bank-sync-status` 按 `platform:question_bank:ops` 查看跨租户同步运营摘要。 - 题库导出能力:租户内容编辑可按题目集合、内容入口或分类节点导出 JSON、`paper_json`、打印 payload、PDF、Word 和每日一练图片 ZIP 素材包,后端强制租户隔离、答案/解析开关、复合题子题脱敏、导出 job 和审计;PDF/Word/ZIP 由 exports worker 生成水印文件或运营素材并发布到 `content_assets`;`daily_practice` 支持每日一练九宫格 metadata、PDF/Word 版式、9 张 PNG/SVG 卡片和拼图包。 - 销售/代理/CRM 增长链路:邀请码、扫码/分享事件、首绑客资保护、销售统计、团队关系、CRM 配置、跟进分配策略和队列。 - `apps/worker` 后台任务进程:CRM webhook 队列消费、generic/钉钉/飞书/企微机器人发送、签名、失败重试和日志;commerce worker 可补偿查询微信/支付宝支付和退款状态;provider-bills worker 可下载微信/支付宝官方账单并导入资金对账;platform-billing worker 可自动为即将到期且未开票的 SaaS 订阅生成服务费账单并写审计;platform-dunning worker 可扫描逾期未结清服务费账单、标记 overdue、写内部催缴记录和审计;platform-dunning-notifications worker 可把内部催缴记录按平台渠道推送到 generic/钉钉/飞书/企微 webhook,并记录幂等发送事件;platform-audit-alerts worker 可把高风险平台审计动作转换为内部告警并递归脱敏告警 details;platform-audit-notifications worker 可把开放审计告警按平台渠道推送到 generic/钉钉/飞书/企微 webhook,并记录幂等发送事件;assets worker 可复检托管资源元数据、执行内置安全扫描并自动下架异常资源;imports worker 可执行大批量导入;public-banks worker 可自动同步公共题库采纳副本;exports worker 可渲染 PDF/Word 导出文件和每日一练 ZIP 图片素材包。 @@ -477,7 +477,7 @@ API 身份上下文: - 平台账号通过 `platform_users.platform_permissions` 控制细粒度能力,`{"*":true}` 表示超级管理员。 - 平台员工通过 `GET/PUT/PATCH /api/platform-admin/staff` 管理,必须绑定 Supabase Auth 用户 ID;禁用员工会使 `status='disabled'`,后续 Supabase JWT 映射和迁移期 session 都会被拒绝。 - 平台后台启动后可调用 `GET /api/platform-admin/permissions` 获取 `catalog/effective`,用于隐藏不可见菜单和按钮。 -- 后端接口继续按 `platform:staff:read/write/status`、`platform:tenant:read/write/status/billing_profile`、`platform:billing:read/write/payment/dunning/notification`、`platform:audit:read/export/alert/notification`、`platform:question_bank:read/grant` 等权限点强制校验。 +- 后端接口继续按 `platform:staff:read/write/status`、`platform:tenant:read/write/status/billing_profile`、`platform:billing:read/write/payment/dunning/notification`、`platform:audit:read/export/alert/notification`、`platform:question_bank:read/grant/ops` 等权限点强制校验。 - `x-platform-admin-key` 只允许本地兼容,生产必须关闭。 ## 重要安全约定 @@ -515,7 +515,7 @@ npm run audit:runtime git diff --check ``` -结果:通过。`npm run test:auth:remote-smoke` 覆盖远程 Auth/JWKS 验收脚本自身。`npm run test:rls` 覆盖 75 条运行时 RLS 断言,包含主租户、合作商租户、无租户 claim、平台管理员旁路和跨租户写入拒绝。`npm run test:api` 覆盖平台细粒度权限、资源访问事件、锁定 CDN 资源拒绝、provider-managed CDN 显式放行、学生短 TTL 下载/预览、访问记录查询、安全扫描门禁、官方账单下载任务权限和脱敏响应、异常订单运营台、人工调整凭证提交/复核/事件/报表、平台账单逾期 dry-run/催缴记录、平台审计告警查询/状态更新/越权拒绝/敏感 details 脱敏、平台审计告警通知渠道/事件查询和密钥不回显、平台催缴通知渠道/事件查询和密钥不回显、租户隔离,以及凭证审批不修改订单/支付/权益。`npm run test:worker:commerce` 覆盖支付/退款补偿、微信/支付宝官方账单下载、账单 hash 校验、导入 `provider_download` 对账批次和密钥不泄露。`npm run test:worker:platform-billing` 覆盖平台 SaaS 订阅自动计费、重复开票保护、账单明细和审计。`npm run test:worker:platform-dunning` 覆盖平台 SaaS 逾期账单标记、内部催缴记录、租户 `past_due` 状态和每日催缴幂等。`npm run test:worker:platform-dunning-notifications` 覆盖平台 SaaS 催缴外部通知入队、generic webhook 发送、幂等、防重复、联系方式掩码、签名密钥不泄露和请求 payload 脱敏。`npm run test:worker:platform-audit-alerts` 覆盖平台审计告警生成、规则匹配、幂等、防重复和告警 details 脱敏。`npm run test:worker:platform-audit-notifications` 覆盖平台审计告警外部通知入队、generic webhook 发送、幂等、防重复、签名密钥不泄露和请求 payload 脱敏。`npm run test:worker:assets` 覆盖托管资源复检、内置安全扫描、外部 HTTP scanner 通过/失败/不可用 fail-closed、扫描失败/跳过事件和异常资源自动下架。`npm run test:worker:exports` 覆盖导出 worker 生成可信资源并标记 `securityScanStatus=passed`。`npm run audit:runtime` 当前为 0 vulnerabilities;Excel 解析已从 `exceljs` 切换为 `read-excel-file`,避免生产运行时携带 `exceljs -> uuid` 的已知中危依赖。 +结果:通过。`npm run test:auth:remote-smoke` 覆盖远程 Auth/JWKS 验收脚本自身。`npm run test:rls` 覆盖 75 条运行时 RLS 断言,包含主租户、合作商租户、无租户 claim、平台管理员旁路和跨租户写入拒绝。`npm run test:api` 覆盖平台细粒度权限、公共题库跨租户同步运营状态、资源访问事件、锁定 CDN 资源拒绝、provider-managed CDN 显式放行、学生短 TTL 下载/预览、访问记录查询、安全扫描门禁、官方账单下载任务权限和脱敏响应、异常订单运营台、人工调整凭证提交/复核/事件/报表、平台账单逾期 dry-run/催缴记录、平台审计告警查询/状态更新/越权拒绝/敏感 details 脱敏、平台审计告警通知渠道/事件查询和密钥不回显、平台催缴通知渠道/事件查询和密钥不回显、租户隔离,以及凭证审批不修改订单/支付/权益。`npm run test:worker:commerce` 覆盖支付/退款补偿、微信/支付宝官方账单下载、账单 hash 校验、导入 `provider_download` 对账批次和密钥不泄露。`npm run test:worker:platform-billing` 覆盖平台 SaaS 订阅自动计费、重复开票保护、账单明细和审计。`npm run test:worker:platform-dunning` 覆盖平台 SaaS 逾期账单标记、内部催缴记录、租户 `past_due` 状态和每日催缴幂等。`npm run test:worker:platform-dunning-notifications` 覆盖平台 SaaS 催缴外部通知入队、generic webhook 发送、幂等、防重复、联系方式掩码、签名密钥不泄露和请求 payload 脱敏。`npm run test:worker:platform-audit-alerts` 覆盖平台审计告警生成、规则匹配、幂等、防重复和告警 details 脱敏。`npm run test:worker:platform-audit-notifications` 覆盖平台审计告警外部通知入队、generic webhook 发送、幂等、防重复、签名密钥不泄露和请求 payload 脱敏。`npm run test:worker:assets` 覆盖托管资源复检、内置安全扫描、外部 HTTP scanner 通过/失败/不可用 fail-closed、扫描失败/跳过事件和异常资源自动下架。`npm run test:worker:public-banks` 覆盖公共题库自动同步、失败通知和恢复自动关闭。`npm run test:worker:exports` 覆盖导出 worker 生成可信资源并标记 `securityScanStatus=passed`。`npm run audit:runtime` 当前为 0 vulnerabilities;Excel 解析已从 `exceljs` 切换为 `read-excel-file`,避免生产运行时携带 `exceljs -> uuid` 的已知中危依赖。 注意:`apps/taro` 是静态构建工程,线上发布 `apps/taro/dist/**`,不发布 `node_modules`。Taro 4.2.0 当前构建工具链仍会触发 `npm run audit:taro:toolchain` 的上游 high/critical 提示,不能用 `npm audit fix --force` 降级到 Taro 3 破坏构建;上线验收时以 `audit:runtime`、构建产物、前端密钥检查和静态服务器配置为准,并持续跟进 Taro 官方修复。 @@ -526,5 +526,7 @@ git diff --check 1. 真实云端 Auth/JWKS 回归、RLS 深测和生产环境配置验收。 2. 继续补 Taro 前端:学生端小程序公式真机验收、题图资源后台字段化、独立消息中心增强、背单词更细统计、小程序支付与分享,租户后台更细导入体验/数据范围 UI/主题素材库/财务复核细节,平台后台在线收款、审计报表增强、审计告警通知升级策略、催缴通知操作台细节和小程序兼容验证。 3. 对象存储真实 AV/内容安全扫描服务联调、CDN 防盗链、转码/CDN 级水印和生命周期策略。 -4. 题库导出模板精排、导出操作台、真实数据 dry-run、导入字段映射 UI 和复检结果操作台。 +4. 题库导出模板精排、导出操作台、使用 `F:\project\参考\旧题库数据库文件` 做真实数据 dry-run、导入字段映射 UI 和复检结果操作台。 5. 真实 OAuth/短信/支付生产账号联调、真实生产账单抽样验收、真实打款 provider、发票、公共题库版本通知/冲突处理操作台、积分活动风控和连续签到奖励深化,以及排行榜防刷/预聚合。 + +旧原生小程序前端位于 `F:\project\参考\旧题库小程序前端文件`,后续 Taro H5/小程序补体验时只作为页面状态、微信平台能力和交互参考,不继承旧 PocketBase 直连和旧鉴权逻辑。 diff --git a/apps/api/src/features/platform-admin/index.ts b/apps/api/src/features/platform-admin/index.ts index 50227c0f..6ce950e1 100644 --- a/apps/api/src/features/platform-admin/index.ts +++ b/apps/api/src/features/platform-admin/index.ts @@ -19,6 +19,7 @@ import { platformPermissionsRoute, platformPlansRoute, platformQuestionBanksRoute, + publicQuestionBankSyncStatusRoute, platformStaffRoute, processOverdueInvoicesRoute, questionBankGrantsRoute, @@ -47,6 +48,7 @@ export const platformAdminRoutes: RouteDefinition[] = [ ['GET', '/api/platform-admin/plans', platformPlansRoute], ['GET', '/api/platform-admin/question-banks', platformQuestionBanksRoute], ['GET', '/api/platform-admin/question-bank-grants', questionBankGrantsRoute], + ['GET', '/api/platform-admin/question-bank-sync-status', publicQuestionBankSyncStatusRoute], ['PUT', '/api/platform-admin/question-bank-grants', upsertQuestionBankGrantRoute], ['GET', '/api/platform-admin/tenants', tenantsRoute], ['POST', '/api/platform-admin/tenants', createTenantRoute], diff --git a/apps/api/src/features/platform-admin/routes.ts b/apps/api/src/features/platform-admin/routes.ts index ac23a958..38a60f88 100644 --- a/apps/api/src/features/platform-admin/routes.ts +++ b/apps/api/src/features/platform-admin/routes.ts @@ -86,6 +86,7 @@ const PLATFORM_PERMISSION_CATALOG = [ { key: 'platform:audit:notification', group: 'audit', label: '维护审计告警通知' }, { key: 'platform:question_bank:read', group: 'question_bank', label: '查看公共题库' }, { key: 'platform:question_bank:grant', group: 'question_bank', label: '授权公共题库' }, + { key: 'platform:question_bank:ops', group: 'question_bank', label: '查看公共题库同步运营' }, ] as const; function csvEscape(value: unknown) { @@ -1014,6 +1015,108 @@ export async function questionBankGrantsRoute(ctx: RequestContext) { return { items }; } +export async function publicQuestionBankSyncStatusRoute(ctx: RequestContext) { + await requirePlatformAdmin(ctx, 'platform:question_bank:ops'); + + const tenantId = listQuery(ctx, 'tenantId'); + const sourceQuestionBankId = listQuery(ctx, 'sourceQuestionBankId'); + const syncStatus = listQuery(ctx, 'syncStatus'); + const status = listQuery(ctx, 'status'); + const onlyOpenIssues = listQuery(ctx, 'onlyOpenIssues') === 'true'; + const limit = intParam(ctx, 'limit', 100, 500); + if (tenantId && !UUID_RE.test(tenantId)) throw new HttpError(400, 'tenantId is invalid', 'INVALID_UUID'); + if (sourceQuestionBankId && !UUID_RE.test(sourceQuestionBankId)) throw new HttpError(400, 'sourceQuestionBankId is invalid', 'INVALID_UUID'); + if (syncStatus && !['pending', 'synced', 'failed'].includes(syncStatus)) { + throw new HttpError(400, 'syncStatus is invalid', 'INVALID_SYNC_STATUS'); + } + if (status && !['active', 'sync_pending', 'suspended', 'archived'].includes(status)) { + throw new HttpError(400, 'status is invalid', 'INVALID_STATUS'); + } + + const items = await query( + ` + select a.id, + a.tenant_id as "tenantId", + t.slug::text as "tenantSlug", + t.name as "tenantName", + a.source_question_bank_id as "sourceQuestionBankId", + source_qb.name as "sourceQuestionBankName", + a.grant_id as "grantId", + a.target_question_bank_id as "targetQuestionBankId", + target_qb.name as "targetQuestionBankName", + a.target_entry_id as "targetEntryId", + ce.name as "targetEntryName", + a.target_collection_id as "targetCollectionId", + qc.name as "targetCollectionName", + a.status, + a.sync_status as "syncStatus", + a.copied_question_count as "copiedQuestionCount", + a.last_synced_at as "lastSyncedAt", + a.updated_at as "updatedAt", + coalesce((a.metadata #>> '{lastSync,conflictCount}')::integer, 0) as "conflictCount", + coalesce(a.metadata #> '{lastSync,counts}', '{}'::jsonb) as "lastSyncCounts", + jsonb_build_object( + 'lastSync', coalesce(a.metadata->'lastSync', '{}'::jsonb), + 'publicBankSyncWorker', coalesce(a.metadata->'publicBankSyncWorker', '{}'::jsonb) + ) as metadata, + coalesce(open_notifications.open_count, 0)::integer as "openNotificationCount", + coalesce(open_notifications.failure_count, 0)::integer as "openFailureNotificationCount", + coalesce(open_notifications.conflict_count, 0)::integer as "openConflictNotificationCount" + from public.tenant_question_bank_adoptions a + join public.tenants t on t.id = a.tenant_id + join public.question_banks source_qb on source_qb.id = a.source_question_bank_id + left join public.question_banks target_qb on target_qb.id = a.target_question_bank_id + left join public.content_entries ce on ce.id = a.target_entry_id and ce.tenant_id = a.tenant_id + left join public.question_collections qc on qc.id = a.target_collection_id and qc.tenant_id = a.tenant_id + left join lateral ( + select count(*) filter (where n.status in ('unread', 'read')) as open_count, + count(*) filter ( + where n.status in ('unread', 'read') + and n.notification_type = 'public_question_bank_sync_failed' + ) as failure_count, + count(*) filter ( + where n.status in ('unread', 'read') + and n.notification_type = 'public_question_bank_conflict' + ) as conflict_count + from public.tenant_content_notifications n + where n.tenant_id = a.tenant_id + and n.adoption_id = a.id + and n.notification_type in ('public_question_bank_sync_failed', 'public_question_bank_conflict') + ) open_notifications on true + where ($1::uuid is null or a.tenant_id = $1::uuid) + and ($2::uuid is null or a.source_question_bank_id = $2::uuid) + and ($3::text = '' or a.sync_status = $3) + and ($4::text = '' or a.status = $4) + and ( + $5::boolean = false + or a.sync_status = 'failed' + or coalesce(open_notifications.open_count, 0) > 0 + ) + order by + case when a.sync_status = 'failed' then 0 when a.sync_status = 'pending' then 1 else 2 end, + coalesce(a.last_synced_at, a.updated_at) desc + limit $6 + `, + [tenantId || null, sourceQuestionBankId || null, syncStatus, status, onlyOpenIssues, limit], + ); + + const summaryRows = await query<{ syncStatus: string; count: number }>( + ` + select a.sync_status as "syncStatus", count(*)::integer as count + from public.tenant_question_bank_adoptions a + where ($1::uuid is null or a.tenant_id = $1::uuid) + and ($2::uuid is null or a.source_question_bank_id = $2::uuid) + group by a.sync_status + `, + [tenantId || null, sourceQuestionBankId || null], + ); + + return { + items, + summary: Object.fromEntries(summaryRows.map(row => [row.syncStatus, Number(row.count || 0)])), + }; +} + export async function upsertQuestionBankGrantRoute(ctx: RequestContext) { await requirePlatformAdmin(ctx, 'platform:question_bank:grant'); diff --git a/apps/api/src/features/tenant-content/public-banks.ts b/apps/api/src/features/tenant-content/public-banks.ts index 9ce4befe..ff35cd53 100644 --- a/apps/api/src/features/tenant-content/public-banks.ts +++ b/apps/api/src/features/tenant-content/public-banks.ts @@ -766,6 +766,28 @@ async function markPublicBankConflictNotificationsResolved(client: pg.PoolClient ); } +async function markPublicBankFailureNotificationsResolved(client: pg.PoolClient, input: { + tenantId: string; + adoptionId: string; + actorUserId: string | null; +}) { + await client.query( + ` + update public.tenant_content_notifications + set status = 'resolved', + read_by = coalesce(read_by, $3), + read_at = coalesce(read_at, now()), + resolved_at = coalesce(resolved_at, now()), + updated_at = now() + where tenant_id = $1 + and adoption_id = $2 + and notification_type = 'public_question_bank_sync_failed' + and status <> 'dismissed' + `, + [input.tenantId, input.adoptionId, input.actorUserId], + ); +} + function objectValue(value: unknown): Record { return value && typeof value === 'object' && !Array.isArray(value) ? value as Record : {}; } @@ -1249,6 +1271,11 @@ export async function executePublicQuestionBankSync(input: PublicQuestionBankSyn actorUserId: auth.userId, }); } + await markPublicBankFailureNotificationsResolved(client, { + tenantId: auth.tenantId, + adoptionId: adoption.id, + actorUserId: auth.userId, + }); await client.query( ` diff --git a/apps/worker/src/jobs/public-banks.ts b/apps/worker/src/jobs/public-banks.ts index 61698b93..9f871266 100644 --- a/apps/worker/src/jobs/public-banks.ts +++ b/apps/worker/src/jobs/public-banks.ts @@ -39,6 +39,14 @@ function truncate(value: unknown, max = 1900) { return String(value ?? '').slice(0, max); } +function publicBankFailureDedupeKey(candidate: PublicBankSyncCandidate, code: string) { + return [ + 'public-bank-sync-failed', + candidate.id, + crypto.createHash('sha256').update(code).digest('hex').slice(0, 16), + ].join(':'); +} + async function claimPublicBankSyncCandidates(limit: number, claimId: string) { const client = await pool.connect(); try { @@ -126,38 +134,133 @@ async function markPublicBankSyncFailed(candidate: PublicBankSyncCandidate, erro workerId: config.publicBankSyncWorkerId, failedAt: nowIso(), }; - await pool.query( - ` - update public.tenant_question_bank_adoptions - set status = 'active', - sync_status = 'failed', - metadata = jsonb_set( - coalesce(metadata, '{}'::jsonb), - '{publicBankSyncWorker}', - coalesce(metadata->'publicBankSyncWorker', '{}'::jsonb) || $3::jsonb, - true - ), - updated_at = now() - where tenant_id = $1 and id = $2 - `, - [ - candidate.tenantId, - candidate.id, - JSON.stringify({ - lastStatus: 'failed', - lastError: details, - lastWorkerId: config.publicBankSyncWorkerId, - lastFinishedAt: nowIso(), - }), - ], - ); - await pool.query( - ` - insert into public.audit_logs (tenant_id, actor_user_id, action, target_type, target_id, details) - values ($1, null, 'content.public_question_bank.sync_worker_failed', 'tenant_question_bank_adoption', $2, $3::jsonb) - `, - [candidate.tenantId, candidate.id, JSON.stringify(details)], - ); + const client = await pool.connect(); + try { + await client.query('begin'); + await client.query( + ` + update public.tenant_question_bank_adoptions + set status = 'active', + sync_status = 'failed', + metadata = jsonb_set( + coalesce(metadata, '{}'::jsonb), + '{publicBankSyncWorker}', + coalesce(metadata->'publicBankSyncWorker', '{}'::jsonb) || $3::jsonb, + true + ), + updated_at = now() + where tenant_id = $1 and id = $2 + `, + [ + candidate.tenantId, + candidate.id, + JSON.stringify({ + lastStatus: 'failed', + lastError: details, + lastWorkerId: config.publicBankSyncWorkerId, + lastFinishedAt: nowIso(), + }), + ], + ); + + const adoptionResult = await client.query<{ + source_question_bank_id: string; + target_question_bank_id: string | null; + target_entry_id: string | null; + target_collection_id: string | null; + source_question_bank_name: string | null; + target_question_bank_name: string | null; + }>( + ` + select a.source_question_bank_id, + a.target_question_bank_id, + a.target_entry_id, + a.target_collection_id, + source_qb.name as source_question_bank_name, + target_qb.name as target_question_bank_name + from public.tenant_question_bank_adoptions a + join public.question_banks source_qb + on source_qb.id = a.source_question_bank_id + left join public.question_banks target_qb + on target_qb.id = a.target_question_bank_id + and target_qb.tenant_id = a.tenant_id + where a.tenant_id = $1 + and a.id = $2 + `, + [candidate.tenantId, candidate.id], + ); + const adoption = adoptionResult.rows[0] || null; + await client.query( + ` + insert into public.audit_logs (tenant_id, actor_user_id, action, target_type, target_id, details) + values ($1, null, 'content.public_question_bank.sync_worker_failed', 'tenant_question_bank_adoption', $2, $3::jsonb) + `, + [candidate.tenantId, candidate.id, JSON.stringify(details)], + ); + + if (adoption) { + await client.query( + ` + insert into public.tenant_content_notifications ( + tenant_id, notification_type, status, severity, + adoption_id, source_question_bank_id, + title, message, action_label, action_path, + dedupe_key, metadata, created_by + ) + values ( + $1, 'public_question_bank_sync_failed', 'unread', 'error', + $2, $3, + $4, $5, '查看同步状态', $6, + $7, $8::jsonb, null + ) + on conflict (tenant_id, notification_type, dedupe_key) + where dedupe_key is not null + do update set status = case + when public.tenant_content_notifications.status = 'dismissed' then public.tenant_content_notifications.status + else 'unread' + end, + severity = excluded.severity, + title = excluded.title, + message = excluded.message, + action_label = excluded.action_label, + action_path = excluded.action_path, + metadata = excluded.metadata, + resolved_at = null, + updated_at = now() + `, + [ + candidate.tenantId, + candidate.id, + adoption.source_question_bank_id, + `${adoption.source_question_bank_name || '公共题库'} 同步失败`, + `公共题库自动同步失败,错误码:${details.code}。请稍后重试或联系平台处理。`, + `/tenant-admin/content?adoptionId=${candidate.id}&panel=public-banks`, + publicBankFailureDedupeKey(candidate, details.code), + JSON.stringify({ + adoptionId: candidate.id, + sourceQuestionBankId: adoption.source_question_bank_id, + sourceQuestionBankName: adoption.source_question_bank_name, + targetQuestionBankId: adoption.target_question_bank_id, + targetQuestionBankName: adoption.target_question_bank_name, + targetEntryId: adoption.target_entry_id, + targetCollectionId: adoption.target_collection_id, + syncStatus: 'failed', + errorCode: details.code, + errorMessage: details.message, + triggeredBy: 'worker', + workerId: config.publicBankSyncWorkerId, + failedAt: details.failedAt, + }), + ], + ); + } + await client.query('commit'); + } catch (failure) { + await client.query('rollback'); + throw failure; + } finally { + client.release(); + } } export async function processPublicBankSyncBatch(limit = config.publicBankSyncBatchSize): Promise { diff --git a/docs/refactor/backend-capability-status.md b/docs/refactor/backend-capability-status.md index 3f492edf..fead0ae9 100644 --- a/docs/refactor/backend-capability-status.md +++ b/docs/refactor/backend-capability-status.md @@ -145,10 +145,10 @@ | 班级/学生/教师管理 | 可联调 | `/api/tenant-admin/classes`、`classes/members`、`students`、`teachers`,支持班级范围权限和审计 | | 学生批量运营 | 可联调 | `/api/tenant-admin/students/bulk-upsert`、`students/status`、`classes/members/bulk-assign`、`students/notes`、`students/followups`;支持逐行结果、限量、防跨租户和教师范围校验 | | 用户站内通知查看 | 可联调 | `GET /api/tenant-admin/user-notifications`;需要 `notifications:read` 权限,支持按用户、状态、类型查询租户内通知和状态汇总,租户后台只读不直接代学生改状态 | -| 平台租户/详情/账务资料/员工/审计/告警/套餐/订阅/账单/用量 | 可联调 | `/api/platform-admin/*`;已支持当前平台账号权限目录、平台员工列表、平台员工创建/编辑、平台员工禁用/恢复、租户列表、创建租户、租户详情、状态变更、账务资料维护、平台审计日志查询、CSV/JSON 审计导出、平台审计告警规则查询、告警列表、确认/解决/忽略、审计告警外部通知渠道和发送事件、SaaS 套餐、订阅、账单、订阅账单候选预览、dry-run、批量生成、自动计费 worker、重复开票保护、收款、逾期标记、内部催缴台账、催缴外部通知渠道和发送事件、用量;平台 API 已拆分 `platform:staff:read/write/status`、`platform:tenant:read/write/status/billing_profile`、`platform:billing:read/write/payment/dunning/notification`、`platform:audit:read/export/alert/notification`、`platform:question_bank:read/grant` 等权限点;审计导出、平台员工操作、告警响应和通知事件都会对 `details`/payload 中的 token/secret/password/key 等敏感字段递归脱敏;`apps/worker --job platform-audit-alerts` 会把租户状态变更、账务资料变更、批量开票、逾期处理、手工收款确认、审计导出等高风险平台审计动作生成内部告警;`apps/worker --job platform-audit-notifications` 会按 `platform_audit_notification_channels` 把开放告警推送到 generic/钉钉/飞书/企微 webhook,签名密钥放 `app_private.platform_secrets` 且 API 不回显原文;`apps/worker --job platform-dunning-notifications` 会按 `platform_dunning_notification_channels` 把内部催缴记录推送到 generic/钉钉/飞书/企微 webhook,发送成功会推进提醒状态,失败会退避重试,联系方式和请求 payload 会脱敏;创建租户、平台员工变更、状态变更、账务资料维护、订阅批量开票、自动开票、逾期催缴、手工收款确认、审计导出、告警状态更新、通知渠道变更和催缴通知渠道变更会写入审计 | +| 平台租户/详情/账务资料/员工/审计/告警/套餐/订阅/账单/用量 | 可联调 | `/api/platform-admin/*`;已支持当前平台账号权限目录、平台员工列表、平台员工创建/编辑、平台员工禁用/恢复、租户列表、创建租户、租户详情、状态变更、账务资料维护、平台审计日志查询、CSV/JSON 审计导出、平台审计告警规则查询、告警列表、确认/解决/忽略、审计告警外部通知渠道和发送事件、SaaS 套餐、订阅、账单、订阅账单候选预览、dry-run、批量生成、自动计费 worker、重复开票保护、收款、逾期标记、内部催缴台账、催缴外部通知渠道和发送事件、用量;平台 API 已拆分 `platform:staff:read/write/status`、`platform:tenant:read/write/status/billing_profile`、`platform:billing:read/write/payment/dunning/notification`、`platform:audit:read/export/alert/notification`、`platform:question_bank:read/grant/ops` 等权限点;审计导出、平台员工操作、告警响应和通知事件都会对 `details`/payload 中的 token/secret/password/key 等敏感字段递归脱敏;`apps/worker --job platform-audit-alerts` 会把租户状态变更、账务资料变更、批量开票、逾期处理、手工收款确认、审计导出等高风险平台审计动作生成内部告警;`apps/worker --job platform-audit-notifications` 会按 `platform_audit_notification_channels` 把开放告警推送到 generic/钉钉/飞书/企微 webhook,签名密钥放 `app_private.platform_secrets` 且 API 不回显原文;`apps/worker --job platform-dunning-notifications` 会按 `platform_dunning_notification_channels` 把内部催缴记录推送到 generic/钉钉/飞书/企微 webhook,发送成功会推进提醒状态,失败会退避重试,联系方式和请求 payload 会脱敏;创建租户、平台员工变更、状态变更、账务资料维护、订阅批量开票、自动开票、逾期催缴、手工收款确认、审计导出、告警状态更新、通知渠道变更和催缴通知渠道变更会写入审计 | | 数据看板聚合接口 | 可联调 | `GET /api/tenant-admin/dashboard`;支持 `7d/30d/90d`、地区筛选、学生/学习/内容/订单/激活码/反馈卡片、趋势、24h 活跃、题型分布、科目排行、地区统计、套餐销量和运营动态 | | 平台公共题库授权 | 可联调 | `/api/platform-admin/question-banks`、`question-bank-grants`;支持按 SaaS 套餐、指定租户或全部活跃租户披露平台公共题库,并可限制授权地区和科目。平台保存 grant 时会校验 `allowedRegionIds`、`allowedSubjectIds` 属于源平台题库租户,且已发布题目的科目必须被授权科目覆盖 | -| 租户采纳/同步公共题库 | 可联调 | `/api/tenant-content/public-question-banks`、`public-question-banks/adopt`、`public-question-banks/sync`、`public-question-banks/conflicts`、`public-question-banks/conflicts/resolve`、`public-question-banks/conflicts/resolve-batch`、`tenant-content/notifications`;租户只能看到自己 `question_bank_grants`、有效 `tenant_subscriptions`、`platform_saas_plans.feature_flags.publicQuestionBanks` 和订阅 `metadata.publicQuestionBankAccess` 同时允许的题库。基础版默认 `limited_regions` 且需要地区 allowlist,专业版默认 `national`;采纳、同步和冲突处理都会重新校验当前授权,越权 grant 返回 `QUESTION_BANK_GRANT_NOT_AVAILABLE`。采纳后生成租户自己的题库、入口、集合和题目快照,可直接进入练习;平台更新后可手动或由 worker 自动同步,新增/更新和冲突会生成租户内容通知;租户自改题目会标记冲突并跳过;后台可查询最近一次冲突明细,并可单条或批量选择“采纳平台版本”/“保留本地版本”,操作会写入逐条审计,冲突全部处理后相关通知自动 resolved | +| 租户采纳/同步公共题库 | 可联调 | `/api/tenant-content/public-question-banks`、`public-question-banks/adopt`、`public-question-banks/sync`、`public-question-banks/conflicts`、`public-question-banks/conflicts/resolve`、`public-question-banks/conflicts/resolve-batch`、`tenant-content/notifications`、`/api/platform-admin/question-bank-sync-status`;租户只能看到自己 `question_bank_grants`、有效 `tenant_subscriptions`、`platform_saas_plans.feature_flags.publicQuestionBanks` 和订阅 `metadata.publicQuestionBankAccess` 同时允许的题库。基础版默认 `limited_regions` 且需要地区 allowlist,专业版默认 `national`;采纳、同步和冲突处理都会重新校验当前授权,越权 grant 返回 `QUESTION_BANK_GRANT_NOT_AVAILABLE`。采纳后生成租户自己的题库、入口、集合和题目快照,可直接进入练习;平台更新后可手动或由 worker 自动同步,新增/更新、冲突和 worker 失败会生成租户内容通知;租户自改题目会标记冲突并跳过;后台可查询最近一次冲突明细,并可单条或批量选择“采纳平台版本”/“保留本地版本”,操作会写入逐条审计,冲突全部处理后相关通知自动 resolved,失败通知会在后续同步恢复成功后自动 resolved;平台运营接口需 `platform:question_bank:ops`,只返回跨租户同步摘要和通知数量,不返回题目正文/答案/解析 | | 题库导出 | 可联调 | `/api/tenant-content/exports/questions`、`/api/tenant-content/exports/jobs`;支持按题目集合、内容入口或分类节点导出 JSON/试卷 payload,也支持 `pdf/docx/daily_practice_zip` 异步导出;`exportType=daily_practice` 可生成每日一练九宫格运营素材 metadata、PDF/Word 基础版式和 ZIP 图片素材包;后端校验租户内容编辑权限、跨租户隔离、答案/解析开关、复合题子题脱敏、导出 job 和审计;`apps/worker --job exports` 负责 PDF/Word/ZIP 渲染、生成 `content_assets`、记录 hash/size/assetId,前端通过资源签名接口下载/预览 | ## 销售、代理、CRM @@ -173,7 +173,7 @@ | PocketBase schema/导出分析 | 可联调 | `scripts/import-pocketbase` 支持 schema summary/risk、`npm run pb:import:dry-run` 导出目录静态迁移报告 | | PocketBase JSON dry-run | 可联调 | 不写数据库,检查导出目录、JSON 形态、核心集合、旧 ID、敏感字段、schema relation、未映射集合和关键业务计数;`--profile=production` 会额外检查生产迁移必需集合和关键字段覆盖率,正式切换建议配合 `--fail-on-warnings` | | 题目 JSON preview/import | 可联调 | 后端负责规范化、issue、幂等、审计 | -| 公共题库采纳、手动同步和自动同步 | 可联调 | 平台授权后,租户可采纳公共题库并复制已发布题目快照;同步 API 和 `public-banks` worker 支持新增/更新题目、重新校验授权、跨租户拒绝、审计记录、租户内容通知和租户自改冲突保护;冲突处理 API 已支持单条/批量采纳平台版本和保留租户本地版本;已覆盖跨租户、重复采纳、采纳后组卷、同步新增题、通知隔离/已读/自动 resolved、冲突不覆盖、单条/批量冲突处理、worker 自动同步、starter 单地区不可见/不可采纳第二地区题库、pro 全国套餐可见第二地区题库等测试 | +| 公共题库采纳、手动同步和自动同步 | 可联调 | 平台授权后,租户可采纳公共题库并复制已发布题目快照;同步 API 和 `public-banks` worker 支持新增/更新题目、重新校验授权、跨租户拒绝、审计记录、租户内容通知和租户自改冲突保护;冲突处理 API 已支持单条/批量采纳平台版本和保留租户本地版本;worker 失败会写 `public_question_bank_sync_failed` 租户通知并保留稳定错误码,恢复成功会自动关闭失败通知;平台可通过 `/api/platform-admin/question-bank-sync-status` 查看跨租户同步运营状态;已覆盖跨租户、重复采纳、采纳后组卷、同步新增题、通知隔离/已读/自动 resolved、冲突不覆盖、单条/批量冲突处理、worker 自动同步/失败通知/恢复关闭、starter 单地区不可见/不可采纳第二地区题库、pro 全国套餐可见第二地区题库等测试 | | 单词 JSON preview/import | 可联调 | 兼容旧模板 | | 知识手册 JSON preview/import | 可联调 | 支持书籍/章节/小节/知识点归一化 | | 分数线 JSON preview/import | 可联调 | 支持 `fields/schools/majors/records` 分桶或 `items` 列表,后端校验租户地区和院校/专业引用 | @@ -181,7 +181,7 @@ | Excel/CSV 导入 | 可联调 | 题目、单词、知识手册、分数线、视频已支持 CSV 和 `.xlsx` 解析,解析后复用 `content_import_jobs/items/issues` 管线并保留 `parser_metadata`;模板下载、字段映射 API、字段映射覆盖白名单、导入任务详情、异步 worker 状态、导入后复检已接入;Taro 租户内容页已接上传/粘贴 preview/import、字段别名编辑、异步任务轮询和复检详情第一版 | | 大批量异步导入 | 可联调 | `executionMode=async` 会将 preview job 置为 `pending`;`apps/worker --job imports` 抢占 queued job,复用 API 导入 executor,支持重试、清锁和审计 | | 题库导出任务 | 可联调 | `content_export_jobs` 记录导出范围、格式、题量、输出 hash、选项、执行人、异步状态、重试次数和 `asset_id`;JSON 类导出返回 inline base64,`pdf/docx/daily_practice_zip` 由 exports worker 渲染为资源台账文件 | -| 公共题库自动同步增强 | 部分覆盖 | `apps/worker --job public-banks` 已可抢占待同步采纳记录、自动同步平台新增/更新题目、记录失败和审计;同步新增/更新和冲突会写入 `tenant_content_notifications`,租户后台已有通知列表、已读/忽略、单条和批量冲突处理第一版;后续需接入生产定时调度、失败告警和更完整运营消息 | +| 公共题库自动同步增强 | 部分覆盖 | `apps/worker --job public-banks` 已可抢占待同步采纳记录、自动同步平台新增/更新题目、记录失败和审计;同步新增/更新、冲突和 worker 失败会写入 `tenant_content_notifications`,租户后台已有通知列表、已读/忽略、失败查看、单条和批量冲突处理第一版;平台侧已有 `/api/platform-admin/question-bank-sync-status` 运营摘要接口。后续需接入生产定时调度、外部告警升级和更完整运营消息 | ## 当前验证 diff --git a/docs/refactor/multitenant-auth-security-contract.md b/docs/refactor/multitenant-auth-security-contract.md index 1f98a600..d3bfec37 100644 --- a/docs/refactor/multitenant-auth-security-contract.md +++ b/docs/refactor/multitenant-auth-security-contract.md @@ -179,6 +179,7 @@ provider event id 幂等 | `platform:audit:notification` | 配置/查看审计告警外部通知 | | `platform:question_bank:read` | 查看平台公共题库 | | `platform:question_bank:grant` | 授权平台公共题库 | +| `platform:question_bank:ops` | 查看跨租户公共题库采纳同步运营状态 | 前端平台后台启动后可调用: diff --git a/docs/refactor/next-development-todo.md b/docs/refactor/next-development-todo.md index e5ea8762..74a4e9a1 100644 --- a/docs/refactor/next-development-todo.md +++ b/docs/refactor/next-development-todo.md @@ -189,6 +189,7 @@ - 已新建 `apps/taro`,不要继续在旧 React Web 上堆兼容。 - H5 和小程序共用同一套业务 API client。 +- 旧原生小程序前端位于 `F:\project\参考\旧题库小程序前端文件`,后续 Taro 小程序/H5 交互补齐时只作为交互、页面状态和微信平台能力参考,不继承旧直连 PocketBase 或旧鉴权逻辑。 - 租户通过域名、小程序配置或启动参数解析。 - 页面主题、品牌、功能开关都从后端租户配置读取;学生端和后台只消费 `/api/tenant/resolve` 的已发布 `branding.theme/publicAssets`,租户后台草稿只通过 `/api/tenant-admin/theme` 查看。 - 当前已完成 H5 学生端、租户后台、平台后台三套构建入口和统一 API client;学生端、租户后台、平台后台都有第一批真实 API 页面;学生端已补地区选择、刷题答题卡、后端权威断点续练、本地进度恢复、模拟倒计时、主观题后端自评、阅读理解/案例分析多小题、错题/收藏复习、题目反馈、视频解析、练习/模考报告、个人中心学习报告可视化、会员收银台、订单详情、售后入口、站内消息中心第一版、积分任务/兑换/积分明细第一版、题干/选项/解析/知识手册 RichContent 安全渲染、H5 KaTeX 公式渲染、私有资源 ID 题图短签名渲染、逐题复盘、背单词卡片学习/发音/收藏练习第一版;平台后台已接入创建租户、租户详情、状态变更、账务资料维护、平台员工列表/创建/编辑/禁用恢复、平台审计查询/CSV 导出、开放审计告警展示/确认/解决、审计告警外部通知渠道/事件状态摘要、催缴外部通知渠道/事件摘要、订阅、订阅账单候选/dry-run/批量生成、自动计费生成结果查看、收款、用量和公共题库授权第一版写操作;租户后台已接权限驱动工作台、学生运营操作台、主题模板预览/发布、角色模板、成员绑定、CRM/分佣操作台、积分任务/兑换操作台和用户通知查看第一版;下一步补小程序公式真机验收、题图资源字段化、独立消息中心增强、状态管理、更细数据范围 UI、学生批量运营增强和小程序兼容验证。 @@ -246,6 +247,6 @@ 2. 继续补 Taro 学生端旧体验:地区选择、刷题答题卡、后端权威断点续练、本地进度恢复、模拟倒计时、主观题后端自评、阅读理解/案例分析多小题、视频播放、反馈、模考报告、逐题复盘、错题/收藏专题、个人中心学习报告、收银台、订单详情、售后入口、站内消息筛选/已读/归档、积分任务/兑换/积分明细、题干/解析/知识手册 RichContent 安全渲染、H5 KaTeX 公式渲染、私有资源 ID 题图短签名、背单词卡片学习/发音/收藏练习、资料短签名水印预览/下载确认已接第一版;继续补小程序公式真机验收、题图资源字段化、独立消息中心增强、背单词更细统计、小程序支付容器、分享场景和状态管理。 3. 补平台后台增强:租户基础资料编辑增强、平台审计告警升级策略、平台催缴通知配置操作台细节和平台在线收款。 4. 云服务器部署 Supabase/PostgreSQL 和 API,配置对象存储生产环境变量,跑 `check:refactor` 的远程等价测试。 -5. 导出现有 PocketBase 数据,按 `docs/refactor/pocketbase-real-data-migration-runbook.md` 做 production dry-run、导入演练、校验和抽样验收。 +5. 使用 `F:\project\参考\旧题库数据库文件` 中的真实 PocketBase 数据,按 `docs/refactor/pocketbase-real-data-migration-runbook.md` 做 production dry-run、导入演练、校验和抽样验收;导入过程中发现的字段污染、跨集合引用断裂、敏感字段和旧权限问题都要沉淀到 importer mapper 或修复脚本,不手工临时修库。 6. 并行补真实登录、真实生产账单格式验收、异常订单运营台、对象存储真实 AV/内容安全服务联调、转码/CDN 级水印/生命周期、题库导出模板精排/操作台、公共题库生产定时调度和失败告警。 7. 前后端联调通过后,再做支付、权限、数据导入、资料下载、视频播放的商用验收。 diff --git a/docs/refactor/pocketbase-real-data-migration-runbook.md b/docs/refactor/pocketbase-real-data-migration-runbook.md index 89425424..d96e91a2 100644 --- a/docs/refactor/pocketbase-real-data-migration-runbook.md +++ b/docs/refactor/pocketbase-real-data-migration-runbook.md @@ -47,6 +47,17 @@ npm run supabase:reset ## 导出目录 +当前工作区已有旧 PocketBase SQLite 数据参考: + +```text +F:\project\参考\旧题库数据库文件 + data.db + auxiliary.db + storage/ +``` + +`data.db` 和 `auxiliary.db` 必须按只读源处理,不能直接在旧库上执行修复 SQL。后续应由本地转换脚本把 PocketBase SQLite 集合导出为规范 JSON,再进入下面的 dry-run/import 管线;`storage/` 中的附件、题图、PDF、视频封面等资源要同步生成资源迁移清单,最终进入 `content_assets` 和对象存储,不允许把旧本地路径或长效 URL 直接写给前端。 + 把 PocketBase 导出的集合 JSON 放到仓库根目录: ```text @@ -82,6 +93,21 @@ $env:PB_EXPORT_DIR="F:\migration\pb_export_20260630" npm run pb:import:dry-run ``` +如果输入源仍是 SQLite,而不是 JSON 目录,下一阶段需要先补 `scripts/import-pocketbase` 的 SQLite 只读导出命令,建议输出到仓库外或 `.gitignore` 覆盖的 `pb_export/`: + +```powershell +$env:PB_SQLITE_DIR="F:\project\参考\旧题库数据库文件" +$env:PB_EXPORT_DIR="F:\project\pb_export" +# 规划命令:npm run pb:export:sqlite +``` + +导出脚本要求: + +- 使用只读 SQLite 连接,禁止修改 `data.db`、`auxiliary.db` 和 `storage/`。 +- 每个 PocketBase collection 输出一个 JSON 文件,并保留旧 `id`、`created`、`updated`、relation 字段和文件字段。 +- 对 `users`、订单、支付、openid/unionid、手机号等敏感字段默认脱敏写 dry-run 报告;正式导入只在后端 mapper 中进入受控身份/订单/审计表。 +- 资源文件字段只输出相对路径、hash、size、mime 和旧 collection/record/file 三元组,后续由对象存储迁移步骤转为 `content_assets`。 + ## 阶段 1:静态 Dry-Run 先运行不写数据库的静态报告。默认是 `development` profile,适合开发环境快速发现 JSON 形态、关系和敏感字段问题: diff --git a/docs/refactor/project-structure.md b/docs/refactor/project-structure.md index 0a6ef9de..92507987 100644 --- a/docs/refactor/project-structure.md +++ b/docs/refactor/project-structure.md @@ -8,7 +8,7 @@ - Git 跟踪的新项目代码,是商用 SaaS 重构后的后端、数据库迁移、导入工具、测试脚本和重构文档。 - 旧 PocketBase/React 题库项目只作为功能对照、数据迁移参考和前端样式参考,不作为当前 Gitea 仓库的源码主体。 -- 旧项目参考文件集中放在 `参考/旧题库项目/`,该目录被 `.gitignore` 忽略,不会进入提交。 +- 旧项目参考文件集中放在 `参考/旧题库项目/`、`参考/旧题库小程序前端文件/` 和 `参考/旧题库数据库文件/`,这些目录被 `.gitignore` 忽略,不会进入提交。 - 旧前端构建产物、临时打包结果和与新 Supabase 技术栈无关的材料,统一放入 `参考/` 或被 `.gitignore` 忽略,根目录只保留新 monorepo 的工程入口。 ## 新重构项目目录 @@ -100,6 +100,18 @@ F:\project\参考\旧题库项目 F:\project\参考\旧构建产物 ``` +旧原生小程序前端参考位于: + +```text +F:\project\参考\旧题库小程序前端文件 +``` + +旧 PocketBase SQLite 数据和 storage 参考位于: + +```text +F:\project\参考\旧题库数据库文件 +``` + 其中主要内容: ```text @@ -126,6 +138,8 @@ F:\project\参考\旧构建产物 dist-public/ 旧学生端构建输出 ``` +旧小程序前端只用于后续 Taro H5/小程序补齐微信登录、分享、支付容器、页面状态和刷题交互细节;不得继续沿用旧 PocketBase 直连、旧鉴权或旧本地权限判断。旧数据库文件目前包含 `data.db`、`auxiliary.db`、备份 SQLite 文件和 `storage/`,只用于本地只读导出、`scripts/import-pocketbase` dry-run、mapper 修复和抽样验收;不得提交到 Gitea,也不得在文档或日志中输出真实手机号、openid、支付单号等敏感数据。 + ## 后续提交规范 提交前建议先看: diff --git a/docs/refactor/taro-frontend-integration.md b/docs/refactor/taro-frontend-integration.md index df835add..0fb7c548 100644 --- a/docs/refactor/taro-frontend-integration.md +++ b/docs/refactor/taro-frontend-integration.md @@ -1724,6 +1724,16 @@ GET /api/tenant-content/notifications POST /api/tenant-content/notifications/status ``` +平台公共题库运营台使用: + +```text +GET /api/platform-admin/question-bank-sync-status +GET /api/platform-admin/question-bank-sync-status?tenantId=&syncStatus=failed&onlyOpenIssues=true +GET /api/platform-admin/question-bank-sync-status?sourceQuestionBankId=&limit=100 +``` + +该接口需要 `platform:question_bank:ops` 权限。它用于平台超级管理员查看各租户采纳公共题库后的同步状态、失败/冲突通知数量和 worker 最近执行摘要;不返回题目正文、答案、解析或对象存储签名。 + 采纳请求: ```json @@ -1795,11 +1805,12 @@ POST /api/tenant-content/notifications/status - 单条冲突处理调用 `POST /api/tenant-content/public-question-banks/conflicts/resolve`,body 为 `{ "adoptionId": "...", "sourceQuestionId": "...", "resolution": "accept_platform | keep_local" }`。`accept_platform` 会把租户副本写成平台当前版本并生成新题目版本;`keep_local` 会记录本地保留决策,同一平台 hash 和本地 hash 后续同步不再反复提示。两种操作都会写审计日志。 - 批量冲突处理调用 `POST /api/tenant-content/public-question-banks/conflicts/resolve-batch`,body 为 `{ "adoptionId": "...", "sourceQuestionIds": ["..."], "resolution": "accept_platform | keep_local", "limit": 50 }`。后端最多处理 100 条,仍会重新校验租户授权、锁定采纳记录和目标题,逐条写审计;前端只提交当前冲突列表中明确展示给操作者的 source id。 - 同步产生新增/更新时,后端会写入 `public_question_bank_synced` 通知;同步产生冲突时,会写入 `public_question_bank_conflict` 通知。通知只包含同步摘要、题库 ID、题目 hash 和操作入口,不保存题目答案或解析。 -- 租户后台可调用 `GET /api/tenant-content/notifications?notificationType=public_question_bank_conflict&status=unread&limit=20` 展示待处理同步消息;也可带 `adoptionId` 查看某个采纳记录的通知。 +- worker 自动同步因授权失效、目标题库缺失等原因失败时,后端会写入 `public_question_bank_sync_failed` 通知,`severity=error`,metadata 只包含 `errorCode/errorMessage/workerId/failedAt` 等脱敏运维摘要。后续同步恢复成功后,后端会自动把未 dismissed 的失败通知标记为 `resolved`。 +- 租户后台可调用 `GET /api/tenant-content/notifications?notificationType=public_question_bank_conflict&status=unread&limit=20` 展示待处理同步消息;也可用 `public_question_bank_sync_failed` 展示自动同步失败消息,并带 `adoptionId` 查看某个采纳记录的通知。 - 通知状态更新调用 `POST /api/tenant-content/notifications/status`,body 为 `{ "notificationIds": ["..."], "status": "read | dismissed | resolved" }`。冲突被单条或批量全部处理后,后端会自动把相关冲突通知标记为 `resolved`。 - `QUESTION_BANK_GRANT_NOT_AVAILABLE`:说明 SaaS 套餐/授权已失效,提示联系平台或升级套餐。 - `QUESTION_BANK_ADOPTION_NOT_FOUND`:说明不是当前租户的采纳记录或记录已归档,前端不要跨租户重试。 -- 当前租户后台可以提供手动“同步平台更新”按钮,并展示 worker 自动同步后的通知、冲突查询结果、单条处理和批量处理按钮。后续继续补更完整运营消息、失败告警和生产定时调度。 +- 当前租户后台可以提供手动“同步平台更新”按钮,并展示 worker 自动同步后的通知、失败消息、冲突查询结果、单条处理和批量处理按钮。平台后台可用 `question-bank-sync-status` 做跨租户运营看板,优先展示失败、pending 和开放冲突。 ## 登录对接 diff --git a/scripts/api-integration-test.js b/scripts/api-integration-test.js index 56aee53c..1e1e5c98 100644 --- a/scripts/api-integration-test.js +++ b/scripts/api-integration-test.js @@ -1184,6 +1184,14 @@ async function testPlatformAdminPermissions() { expectStatus: 403, }); assert.equal(paymentDenied.code, 'PLATFORM_PERMISSION_REQUIRED', 'manual service-fee payment must require billing payment permission'); + + const publicBankOpsDenied = await request('/api/platform-admin/question-bank-sync-status', { + tenantId: false, + userId: false, + headers: restrictedHeaders, + expectStatus: 403, + }); + assert.equal(publicBankOpsDenied.code, 'PLATFORM_PERMISSION_REQUIRED', 'public bank sync ops should require dedicated platform question bank ops permission'); } async function testPlatformStaffManagement() { @@ -6841,6 +6849,22 @@ async function testPublicQuestionBankAdoption() { 'public bank sync should insert newly published source questions', ); + const platformSyncStatus = await request('/api/platform-admin/question-bank-sync-status', { + userId: false, + headers: { 'x-platform-admin-key': 'local-platform-admin-key' }, + query: { tenantId: PARTNER_TENANT_ID, sourceQuestionBankId: ids.questionBank }, + }); + const platformSyncStatusItem = platformSyncStatus.items?.find(item => item.id === adopted.item.id); + assert.equal(platformSyncStatusItem?.syncStatus, 'synced', 'platform admin should inspect public bank sync status across tenants'); + assert.equal(platformSyncStatus.summary?.synced >= 1, true, 'platform sync status should include summary counts'); + assert.ok(!JSON.stringify(platformSyncStatusItem).includes('新增公共题应同步到已采纳租户'), 'platform sync status must not leak source question explanations'); + + const studentPlatformSyncStatusDenied = await request('/api/platform-admin/question-bank-sync-status', { + query: { tenantId: PARTNER_TENANT_ID }, + expectStatus: 403, + }); + assert.equal(studentPlatformSyncStatusDenied.code, 'PLATFORM_ADMIN_REQUIRED', 'public bank sync ops endpoint must require platform admin access'); + const syncNotifications = await request('/api/tenant-content/notifications', { tenantId: PARTNER_TENANT_ID, userId: PARTNER_TENANT_ADMIN_USER_ID, @@ -6945,6 +6969,15 @@ async function testPublicQuestionBankAdoption() { 'public bank sync should identify the source question that conflicts with tenant edits', ); + const failedPlatformSyncStatus = await request('/api/platform-admin/question-bank-sync-status', { + userId: false, + headers: { 'x-platform-admin-key': 'local-platform-admin-key' }, + query: { tenantId: PARTNER_TENANT_ID, syncStatus: 'failed', onlyOpenIssues: 'true' }, + }); + const failedPlatformSyncStatusItem = failedPlatformSyncStatus.items?.find(item => item.id === adopted.item.id); + assert.equal(failedPlatformSyncStatusItem?.syncStatus, 'failed', 'platform sync ops should expose failed public bank adoptions'); + assert.equal(failedPlatformSyncStatusItem?.openConflictNotificationCount >= 1, true, 'platform sync ops should expose open conflict notification count'); + const conflictNotifications = await request('/api/tenant-content/notifications', { tenantId: PARTNER_TENANT_ID, userId: PARTNER_TENANT_ADMIN_USER_ID, diff --git a/scripts/public-bank-worker-integration-test.js b/scripts/public-bank-worker-integration-test.js index 9f2b4e0e..5d2a9eca 100644 --- a/scripts/public-bank-worker-integration-test.js +++ b/scripts/public-bank-worker-integration-test.js @@ -230,8 +230,111 @@ async function main() { const secondOutput = await runWorkerOnce(); assert.match(secondOutput, /processed=0/, 'worker should skip already synced public bank when source has not changed'); + await pool.query( + 'update public.question_bank_grants set status = $2, updated_at = now() where id = $1', + [ids.grant, 'disabled'], + ); + await pool.query( + ` + update public.tenant_question_bank_adoptions + set status = 'active', + sync_status = 'pending', + metadata = metadata || '{"source":"public_bank_worker_failure_test"}'::jsonb, + updated_at = now() + where tenant_id = $1 and id = $2 + `, + [PARTNER_TENANT_ID, ids.adoption], + ); + + const failedOutput = await runWorkerOnce(); + assert.match(failedOutput, /processed=1/, 'worker should claim forced pending adoption'); + assert.match(failedOutput, /failed=1/, 'worker should count unavailable grant as failed'); + + const failedAdoption = await pool.query( + ` + select status, sync_status, metadata + from public.tenant_question_bank_adoptions + where tenant_id = $1 and id = $2 + `, + [PARTNER_TENANT_ID, ids.adoption], + ); + assert.equal(failedAdoption.rows[0]?.status, 'active', 'failed worker sync should return adoption to active'); + assert.equal(failedAdoption.rows[0]?.sync_status, 'failed', 'failed worker sync should mark adoption failed'); + assert.equal( + failedAdoption.rows[0]?.metadata?.publicBankSyncWorker?.lastError?.code, + 'QUESTION_BANK_GRANT_NOT_AVAILABLE', + 'failed worker sync should persist a stable error code', + ); + + const failureNotifications = await pool.query( + ` + select notification_type, status, severity, message, metadata + from public.tenant_content_notifications + where tenant_id = $1 and adoption_id = $2 and notification_type = 'public_question_bank_sync_failed' + order by created_at desc + limit 5 + `, + [PARTNER_TENANT_ID, ids.adoption], + ); + const failureNotification = failureNotifications.rows[0]; + assert.equal(failureNotification?.status, 'unread', 'failed worker sync should create an unread tenant notification'); + assert.equal(failureNotification?.severity, 'error', 'failed worker sync notification should use error severity'); + assert.equal( + failureNotification?.metadata?.errorCode, + 'QUESTION_BANK_GRANT_NOT_AVAILABLE', + 'failed worker sync notification should expose stable error code metadata', + ); + assert.ok( + !JSON.stringify(failureNotification).includes('integration-second-region-v1'), + 'failed worker sync notification should not leak source question details', + ); + + await pool.query( + 'update public.question_bank_grants set status = $2, updated_at = now() where id = $1', + [ids.grant, 'active'], + ); + await pool.query( + ` + update public.tenant_question_bank_adoptions + set status = 'active', sync_status = 'pending', updated_at = now() + where tenant_id = $1 and id = $2 + `, + [PARTNER_TENANT_ID, ids.adoption], + ); + + const recoveryOutput = await runWorkerOnce(); + assert.match(recoveryOutput, /processed=1/, 'worker should retry adoption after grant recovers'); + + const recoveredAdoption = await pool.query( + ` + select sync_status + from public.tenant_question_bank_adoptions + where tenant_id = $1 and id = $2 + `, + [PARTNER_TENANT_ID, ids.adoption], + ); + assert.equal(recoveredAdoption.rows[0]?.sync_status, 'synced', 'worker should recover failed sync after grant becomes available'); + + const resolvedFailureNotifications = await pool.query( + ` + select status, resolved_at + from public.tenant_content_notifications + where tenant_id = $1 and adoption_id = $2 and notification_type = 'public_question_bank_sync_failed' + order by updated_at desc + limit 1 + `, + [PARTNER_TENANT_ID, ids.adoption], + ); + assert.equal( + resolvedFailureNotifications.rows[0]?.status, + 'resolved', + 'successful recovery should resolve previous public bank sync failure notifications', + ); + assert.ok(resolvedFailureNotifications.rows[0]?.resolved_at, 'resolved failure notification should record resolved_at'); + console.log('Public question bank sync worker integration test complete.'); } finally { + await pool.query('update public.question_bank_grants set status = $2, updated_at = now() where id = $1', [ids.grant, 'active']).catch(() => {}); await cleanup(pool).catch(() => {}); await pool.end(); } diff --git a/supabase/migrations/202606300008_public_bank_sync_failure_notifications.sql b/supabase/migrations/202606300008_public_bank_sync_failure_notifications.sql new file mode 100644 index 00000000..088b48d0 --- /dev/null +++ b/supabase/migrations/202606300008_public_bank_sync_failure_notifications.sql @@ -0,0 +1,14 @@ +alter table public.tenant_content_notifications + drop constraint if exists tenant_content_notifications_notification_type_check; + +alter table public.tenant_content_notifications + add constraint tenant_content_notifications_notification_type_check + check (notification_type in ( + 'public_question_bank_synced', + 'public_question_bank_conflict', + 'public_question_bank_sync_failed' + )); + +create index if not exists idx_tenant_question_bank_adoptions_sync_ops + on public.tenant_question_bank_adoptions(sync_status, updated_at desc) + where sync_status in ('pending', 'failed');