feat: add provider bill download reconciliation jobs

This commit is contained in:
Codex
2026-06-29 21:59:42 +08:00
parent aeca84b260
commit f3577e257c
21 changed files with 1691 additions and 158 deletions

View File

@@ -19,17 +19,17 @@
- 公共题库商业化能力:租户可采纳平台授权题库为本租户副本,并可手动或由 worker 自动同步平台新增/更新题目;同步会保护租户自改题目,返回冲突而不覆盖,后台可查询冲突明细。
- 题库导出能力:租户内容编辑可按题目集合、内容入口或分类节点导出 JSON、`paper_json`、打印 payload、PDF、Word 和每日一练图片 ZIP 素材包,后端强制租户隔离、答案/解析开关、复合题子题脱敏、导出 job 和审计PDF/Word/ZIP 由 exports worker 生成水印文件或运营素材并发布到 `content_assets``daily_practice` 支持每日一练九宫格 metadata、PDF/Word 版式、9 张 PNG/SVG 卡片和拼图包。
- 销售/代理/CRM 增长链路:邀请码、扫码/分享事件、首绑客资保护、销售统计、团队关系、CRM 配置、跟进分配策略和队列。
- `apps/worker` 后台任务进程CRM webhook 队列消费、generic/钉钉/飞书/企微机器人发送、签名、失败重试和日志commerce worker 可补偿查询微信/支付宝支付和退款状态assets worker 可复检托管资源元数据、执行内置安全扫描并自动下架异常资源imports worker 可执行大批量导入public-banks worker 可自动同步公共题库采纳副本exports worker 可渲染 PDF/Word 导出文件和每日一练 ZIP 图片素材包。
- `apps/worker` 后台任务进程CRM webhook 队列消费、generic/钉钉/飞书/企微机器人发送、签名、失败重试和日志commerce worker 可补偿查询微信/支付宝支付和退款状态;provider-bills worker 可下载微信/支付宝官方账单并导入资金对账;assets worker 可复检托管资源元数据、执行内置安全扫描并自动下架异常资源imports worker 可执行大批量导入public-banks worker 可自动同步公共题库采纳副本exports worker 可渲染 PDF/Word 导出文件和每日一练 ZIP 图片素材包。
- 销售/代理分佣结算基础闭环:租户默认比例、成员比例、激活码批次比例、订单/激活码归因、结算单生成、审核、线下打款状态、CSV/JSON 导出、打款凭证登记/复核和权限隔离。
- 订单售后基础闭环:退款请求、审核、处理状态流、微信/支付宝发起退款、微信/支付宝退款查询确认、微信/支付宝退款通知 webhook、退款金额累计、部分/全额退款订单状态、全额退款权益撤销、退款事件和审计日志。
- 资金对账和差错工单闭环:租户财务/运营可通过 `/api/commerce/reconciliation/*` 导入或预览支付/退款账单行,后端按租户隔离比对本地订单、支付、退款记录,识别已匹配、金额不一致、状态不一致、供应商有本地无、本地有供应商无、重复行和无效行,并写入对账批次、明细和审计日志;异常明细可创建差错工单,支持分配、开始处理、升级、解决、忽略、重开和事件留痕。工单只做财务审核闭环,不直接修改订单、支付、退款或权益。
- 资金对账和差错工单闭环:租户财务/运营可通过 `/api/commerce/reconciliation/*` 导入或预览支付/退款账单行,也可创建微信/支付宝官方账单下载任务;后端按租户隔离比对本地订单、支付、退款记录,识别已匹配、金额不一致、状态不一致、供应商有本地无、本地有供应商无、重复行和无效行,并写入对账批次、明细和审计日志;异常明细可创建差错工单,支持分配、开始处理、升级、解决、忽略、重开和事件留痕。工单只做财务审核闭环,不直接修改订单、支付、退款或权益。
- PocketBase schema/数据导入器雏形和导入后校验脚本。
- 本地 Supabase reset、烟测 seed、API 集成测试、完整重构检查命令。
还没有达到生产交付的部分:
- Supabase Auth/JWT、租户角色模板、班级/教师/学生范围权限已可联调;生产前还要做真实云端 Auth/JWKS 回归和 RLS 深测。
- 阿里云/腾讯云短信、微信小程序登录、微信网页登录、QQ 登录、手机号绑定/换绑、微信支付、支付宝主链路、微信/支付宝发起退款/查询确认/退款通知、支付/退款补偿 worker 已完成本地适配;资金对账已支持手工/API 账单导入比对和差错工单处理,微信/支付宝官方账单自动下载、异常订单运营台和真实生产账号联调还没接完。
- 阿里云/腾讯云短信、微信小程序登录、微信网页登录、QQ 登录、手机号绑定/换绑、微信支付、支付宝主链路、微信/支付宝发起退款/查询确认/退款通知、支付/退款补偿 worker 已完成本地适配;资金对账已支持手工/API 账单导入、微信/支付宝官方账单下载任务、provider-bills worker 自动导入比对和差错工单处理;异常订单运营台、人工调整凭证复核报表和真实生产账号联调还没接完。
- OSS/COS/Supabase Storage 上传下载签名 provider 已接入上传后校验、PDF/图片预览、资源访问事件、动态水印上下文、锁定资源 CDN 边界、资源复检 worker、内置 `metadata_rules` 安全扫描和外部 HTTP 杀毒/内容安全 scanner 接入层已完成。生产还要配置真实扫描服务 endpoint/token并继续补转码/CDN 级水印、CDN 刷新和对象生命周期策略。
- Excel/CSV 导入解析已完成并复用 `content_import_jobs/items/issues` 管线;大批量异步导入 worker 基础已接入,支持 queued job 消费、重试和审计;导入后复检、模板下载和字段映射 API 已完成,前端 UI 待接。
- 题库导出已完成服务端结构化 payload、PDF/Word 二进制 worker、每日一练基础导出和每日一练 ZIP 图片素材包;后续还要补更精细试卷模板、多模板排版和导出操作台体验。
@@ -61,7 +61,7 @@
```text
apps/api/ Node.js 业务 API
apps/taro/ Taro 4 React 跨端前端H5 三入口,后续扩展小程序
apps/worker/ 后台异步任务CRM webhook、支付/退款补偿、资源复检、导入执行、公共题库同步、题库导出渲染等
apps/worker/ 后台异步任务CRM webhook、支付/退款补偿、官方账单下载、资源复检、导入执行、公共题库同步、题库导出渲染等
packages/config/ 共享配置
packages/db/ PostgreSQL 连接池和查询封装
packages/domain/ 领域常量和共享类型
@@ -165,6 +165,12 @@ npm --workspace @tiku-saas/worker run crm:once
npm --workspace @tiku-saas/worker run commerce:once
```
单次运行微信/支付宝官方账单下载 worker
```bash
npm --workspace @tiku-saas/worker run provider-bills:once
```
单次运行内容资源复检 worker
```bash
@@ -310,7 +316,7 @@ API 身份上下文:
- 题库入口和分类使用 `content_entries/content_nodes`;题目列表和练习规则使用 `question_collections/practice_blueprints`,前端不要再把旧树字段当成唯一业务结构。
- 批量导入必须先写 `content_import_jobs/items/issues`,保留原始 payload、规范化 payload、逐行问题和审计记录。题目、单词、知识手册、分数线和视频 JSON/CSV/Excel 导入已走这套后台校验管线;大批量任务可提交 `executionMode=async`,由 imports worker 消费,前端只轮询 job 状态和展示 issues。
- 题库导出必须由后端按权限生成,不允许前端直接读取数据库拼导出文件;不开启答案/解析时顶层题目和复合题子题都必须脱敏PDF/Word/每日一练 ZIP 只通过 exports worker 写入 `content_assets` 后再签名下载/预览。
- 支付 webhook 必须先设计幂等键和验签流程,再进入生产使用;生产环境还应定时运行 commerce worker 兜底供应商漏通知和处理中退款,并定期通过资金对账接口导入供应商账单核对本地订单。对账差错工单只允许记录财务处理结论和凭证,不允许前端或工单接口直接篡改订单、支付、退款或权益状态。
- 支付 webhook 必须先设计幂等键和验签流程,再进入生产使用;生产环境还应定时运行 commerce worker 兜底供应商漏通知和处理中退款,并定时运行 provider-bills worker 下载官方账单核对本地订单。官方账单下载任务只保存下载域名、hash 和对账批次 ID不向前端暴露下载 URL 或商户密钥。对账差错工单只允许记录财务处理结论和凭证,不允许前端或工单接口直接篡改订单、支付、退款或权益状态。
## 最近一次验证
@@ -320,6 +326,7 @@ API 身份上下文:
npx supabase db reset
npm run check:api
npm run check:worker
npm run test:worker:commerce
npm run test:worker:assets
npm run test:worker:exports
npm run test:api
@@ -328,7 +335,7 @@ npm run audit:runtime
git diff --check
```
结果:通过。`npm run test:api` 覆盖资源访问事件、锁定 CDN 资源拒绝、provider-managed CDN 显式放行、学生短 TTL 下载/预览、访问记录查询安全扫描门禁。`npm run test:worker:assets` 覆盖托管资源复检、内置安全扫描、外部 HTTP scanner 通过/失败/不可用 fail-closed、扫描失败/跳过事件和异常资源自动下架。`npm run test:worker:exports` 覆盖导出 worker 生成可信资源并标记 `securityScanStatus=passed``npm run audit:runtime` 无 high/critical 漏洞;当前运行时依赖树仍有 `exceljs -> uuid` 的 moderate 级提示,修复需要破坏性降级 `exceljs`,后续应在导入 Excel 回归充分后单独处理。
结果:通过。`npm run test:api` 覆盖资源访问事件、锁定 CDN 资源拒绝、provider-managed CDN 显式放行、学生短 TTL 下载/预览、访问记录查询安全扫描门禁、官方账单下载任务权限和脱敏响应。`npm run test:worker:commerce` 覆盖支付/退款补偿、微信/支付宝官方账单下载、账单 hash 校验、导入 `provider_download` 对账批次和密钥不泄露`npm run test:worker:assets` 覆盖托管资源复检、内置安全扫描、外部 HTTP scanner 通过/失败/不可用 fail-closed、扫描失败/跳过事件和异常资源自动下架。`npm run test:worker:exports` 覆盖导出 worker 生成可信资源并标记 `securityScanStatus=passed``npm run audit:runtime` 无 high/critical 漏洞;当前运行时依赖树仍有 `exceljs -> uuid` 的 moderate 级提示,修复需要破坏性降级 `exceljs`,后续应在导入 Excel 回归充分后单独处理。
注意:`apps/taro` 是静态构建工程,线上发布 `apps/taro/dist/**`,不发布 `node_modules`。Taro 4.2.0 当前构建工具链仍会触发 `npm run audit:taro:toolchain` 的上游 high/critical 提示,不能用 `npm audit fix --force` 降级到 Taro 3 破坏构建;上线验收时以 `audit:runtime`、构建产物、前端密钥检查和静态服务器配置为准,并持续跟进 Taro 官方修复。
@@ -340,4 +347,4 @@ git diff --check
2. 继续补 Taro 前端:学生端视频/反馈/模考报告/订单收银台,租户后台写入表单/导入操作台/公共题库同步/角色模板 UI平台后台租户详情/审计/自动计费增强,小程序兼容验证。
3. 对象存储真实 AV/内容安全扫描服务联调、CDN 防盗链、转码/CDN 级水印和生命周期策略。
4. 题库导出模板精排、导出操作台、真实数据 dry-run、导入字段映射 UI 和复检结果操作台。
5. 真实 OAuth/短信/支付生产账号联调、微信/支付宝官方账单自动下载、异常订单运营台、公共题库版本通知/冲突处理操作台、积分活动深化,以及排行榜防刷/预聚合。
5. 真实 OAuth/短信/支付生产账号联调、异常订单运营台、人工调整凭证复核、公共题库版本通知/冲突处理操作台、积分活动深化,以及排行榜防刷/预聚合。

View File

@@ -21,11 +21,13 @@ import {
createReconciliationIssueRoute,
importReconciliationRoute,
previewReconciliationRoute,
providerBillDownloadJobsRoute,
reconciliationAnomaliesRoute,
reconciliationBatchesRoute,
reconciliationIssueEventsRoute,
reconciliationIssuesRoute,
reconciliationItemsRoute,
requestProviderBillDownloadRoute,
updateReconciliationIssueStatusRoute,
} from './reconciliation.js';
@@ -52,6 +54,8 @@ export const commerceRoutes: RouteDefinition[] = [
['GET', '/api/commerce/reconciliation/batches', reconciliationBatchesRoute],
['GET', '/api/commerce/reconciliation/items', reconciliationItemsRoute],
['GET', '/api/commerce/reconciliation/anomalies', reconciliationAnomaliesRoute],
['POST', '/api/commerce/reconciliation/provider-bills/request', requestProviderBillDownloadRoute],
['GET', '/api/commerce/reconciliation/provider-bills/jobs', providerBillDownloadJobsRoute],
['POST', '/api/commerce/reconciliation/issues/create', createReconciliationIssueRoute],
['GET', '/api/commerce/reconciliation/issues', reconciliationIssuesRoute],
['POST', '/api/commerce/reconciliation/issues/status', updateReconciliationIssueStatusRoute],

View File

@@ -6,9 +6,9 @@ import { intParam, optionalInteger, optionalString, readJsonBody, requiredString
import { query, transaction } from '../../core/db.js';
import { requireTenantAdmin, requireTenantPermission, type TenantAdminAuth } from '../tenant-admin/auth.js';
type ReconciliationProvider = 'wechat_pay' | 'alipay' | 'manual';
type BillType = 'payment' | 'refund' | 'combined';
type ReconciliationSource = 'manual_upload' | 'provider_download' | 'api' | 'worker';
export type ReconciliationProvider = 'wechat_pay' | 'alipay' | 'manual';
export type BillType = 'payment' | 'refund' | 'combined';
export type ReconciliationSource = 'manual_upload' | 'provider_download' | 'api' | 'worker';
type TransactionType = 'payment' | 'refund';
type MatchStatus =
| 'matched'
@@ -62,6 +62,7 @@ const PROVIDER_SUCCESS_STATUSES = new Set([
]);
const LOCAL_PAID_PAYMENT_STATUSES = new Set(['paid', 'partially_refunded', 'refunded']);
const LOCAL_SUCCEEDED_REFUND_STATUSES = new Set(['succeeded']);
const MAX_PROVIDER_DOWNLOAD_ROWS = 50_000;
interface RawBillRow {
[key: string]: unknown;
@@ -164,7 +165,19 @@ interface ReconciliationBuildResult {
items: ReconciliationItemDraft[];
}
function normalizeProvider(value: string): ReconciliationProvider {
export interface ReconciliationBatchImportInput {
tenantId: string;
actorUserId: string | null;
provider: ReconciliationProvider;
billDate: string;
billType: BillType;
source: ReconciliationSource;
sourceName: string | null;
rows: unknown[];
metadata?: Record<string, unknown>;
}
export function normalizeReconciliationProvider(value: string): ReconciliationProvider {
const normalized = value.trim().toLowerCase().replace(/[-\s]/g, '_');
if (['wechat', 'wechatpay', 'wxpay', 'wx_pay', 'wechat_pay'].includes(normalized)) return 'wechat_pay';
if (['alipay', 'ali_pay'].includes(normalized)) return 'alipay';
@@ -178,7 +191,7 @@ function providerAliases(provider: ReconciliationProvider) {
return ['manual'];
}
function normalizeBillType(value: unknown): BillType {
export function normalizeReconciliationBillType(value: unknown): BillType {
if (typeof value !== 'string' || !value.trim()) return 'combined';
const normalized = value.trim().toLowerCase();
if (BILL_TYPES.has(normalized as BillType)) return normalized as BillType;
@@ -192,7 +205,7 @@ function normalizeSource(value: unknown): ReconciliationSource {
throw new HttpError(400, 'source is invalid', 'RECONCILIATION_SOURCE_INVALID');
}
function normalizeBillDate(value: string) {
export function normalizeReconciliationBillDate(value: string) {
const trimmed = value.trim();
if (!/^\d{4}-\d{2}-\d{2}$/.test(trimmed)) {
throw new HttpError(400, 'billDate must be YYYY-MM-DD', 'RECONCILIATION_BILL_DATE_INVALID');
@@ -823,9 +836,9 @@ async function buildReconciliation(
}
function readReconciliationInput(body: Record<string, unknown>) {
const provider = normalizeProvider(requiredString(body, 'provider'));
const billDate = normalizeBillDate(requiredString(body, 'billDate'));
const billType = normalizeBillType(body.billType ?? body.bill_type);
const provider = normalizeReconciliationProvider(requiredString(body, 'provider'));
const billDate = normalizeReconciliationBillDate(requiredString(body, 'billDate'));
const billType = normalizeReconciliationBillType(body.billType ?? body.bill_type);
const source = normalizeSource(body.source);
const sourceName = optionalString(body, 'sourceName') || optionalString(body, 'source_name') || null;
const rawRows = body.rows;
@@ -839,6 +852,160 @@ function readReconciliationInput(body: Record<string, unknown>) {
return { provider, billDate, billType, source, sourceName, rows };
}
function normalizeProviderDownloadedRows(provider: ReconciliationProvider, billType: BillType, rows: unknown[]) {
if (rows.length > MAX_PROVIDER_DOWNLOAD_ROWS) {
throw new HttpError(413, `Too many provider bill rows. Max ${MAX_PROVIDER_DOWNLOAD_ROWS}.`, 'RECONCILIATION_PROVIDER_ROWS_TOO_MANY');
}
return normalizeRows(provider, billType, rows);
}
export async function importReconciliationBatch(
client: pg.PoolClient,
input: ReconciliationBatchImportInput,
) {
const provider = input.provider;
const billDate = normalizeReconciliationBillDate(input.billDate);
const billType = normalizeReconciliationBillType(input.billType);
const source = normalizeSource(input.source);
const sourceName = input.sourceName;
const rows = normalizeProviderDownloadedRows(provider, billType, input.rows);
const built = await buildReconciliation(client, {
tenantId: input.tenantId,
provider,
billDate,
billType,
source,
sourceName,
rows,
});
const insertedBatch = await client.query<Record<string, unknown>>(
`
insert into public.commerce_reconciliation_batches (
tenant_id, provider, bill_date, bill_type, source, source_name, source_hash,
status, total_count, matched_count, mismatch_count, missing_local_count,
missing_provider_count, duplicate_count, ignored_count, amount_cents,
refund_amount_cents, fee_cents, created_by, completed_at, metadata
)
values (
$1, $2, $3::date, $4, $5, $6, $7,
$8, $9, $10, $11, $12,
$13, $14, $15, $16,
$17, $18, $19::uuid, now(), $20::jsonb
)
returning id, provider, bill_date as "billDate", bill_type as "billType",
source, source_name as "sourceName", source_hash as "sourceHash",
status, total_count as "totalCount", matched_count as "matchedCount",
mismatch_count as "mismatchCount", missing_local_count as "missingLocalCount",
missing_provider_count as "missingProviderCount", duplicate_count as "duplicateCount",
ignored_count as "ignoredCount", amount_cents as "amountCents",
refund_amount_cents as "refundAmountCents", fee_cents as "feeCents",
created_by as "createdBy", completed_at as "completedAt", error,
metadata, created_at as "createdAt", updated_at as "updatedAt"
`,
[
input.tenantId,
built.batch.provider,
built.batch.billDate,
built.batch.billType,
built.batch.source,
built.batch.sourceName,
built.batch.sourceHash,
built.batch.status,
built.batch.totalCount,
built.batch.matchedCount,
built.batch.mismatchCount,
built.batch.missingLocalCount,
built.batch.missingProviderCount,
built.batch.duplicateCount,
built.batch.ignoredCount,
built.batch.amountCents,
built.batch.refundAmountCents,
built.batch.feeCents,
input.actorUserId,
JSON.stringify(input.metadata || {}),
],
);
const batch = insertedBatch.rows[0];
const batchId = String(batch.id);
for (const item of built.items) {
await client.query(
`
insert into public.commerce_reconciliation_items (
tenant_id, batch_id, row_no, provider, transaction_type,
provider_trade_no, provider_refund_no, order_no, refund_no,
amount_cents, refund_amount_cents, fee_cents, paid_at, refunded_at,
provider_status, local_status, order_id, payment_id, refund_request_id,
match_status, severity, issue_code, details
)
values (
$1, $2, $3, $4, $5,
$6, $7, $8, $9,
$10, $11, $12, $13::timestamptz, $14::timestamptz,
$15, $16, $17::uuid, $18::uuid, $19::uuid,
$20, $21, $22, $23::jsonb
)
`,
[
input.tenantId,
batchId,
item.rowNo,
item.provider,
item.transactionType,
item.providerTradeNo,
item.providerRefundNo,
item.orderNo,
item.refundNo,
item.amountCents,
item.refundAmountCents,
item.feeCents,
item.paidAt,
item.refundedAt,
item.providerStatus,
item.localStatus,
item.orderId,
item.paymentId,
item.refundRequestId,
item.matchStatus,
item.severity,
item.issueCode,
JSON.stringify(item.details),
],
);
}
await client.query(
`
insert into public.audit_logs (tenant_id, actor_user_id, action, target_type, target_id, details)
values ($1, $2::uuid, 'commerce.reconciliation.imported', 'commerce_reconciliation_batch', $3, $4::jsonb)
`,
[
input.tenantId,
input.actorUserId,
batchId,
JSON.stringify({
provider: built.batch.provider,
billDate: built.batch.billDate,
billType: built.batch.billType,
source: built.batch.source,
sourceName: built.batch.sourceName,
sourceHash: built.batch.sourceHash,
summary: {
totalCount: built.batch.totalCount,
matchedCount: built.batch.matchedCount,
mismatchCount: built.batch.mismatchCount,
missingLocalCount: built.batch.missingLocalCount,
missingProviderCount: built.batch.missingProviderCount,
duplicateCount: built.batch.duplicateCount,
ignoredCount: built.batch.ignoredCount,
},
}),
],
);
return { batch, built };
}
function batchPayload(row: Record<string, unknown>) {
return {
id: row.id,
@@ -1065,6 +1232,158 @@ async function authorizeWrite(ctx: RequestContext) {
return auth;
}
async function authorizeDownload(ctx: RequestContext) {
const auth = await requireTenantAdmin(ctx);
requireTenantPermission(auth, 'tenant:reconciliation:download');
return auth;
}
function providerBillJobPayload(row: Record<string, unknown>) {
return {
id: row.id,
provider: row.provider,
billDate: row.billDate,
billType: row.billType,
status: row.status,
sourceName: row.sourceName,
sourceHash: row.sourceHash,
rowCount: row.rowCount,
downloadHashType: row.downloadHashType,
downloadHashValue: row.downloadHashValue,
downloadUrlHost: row.downloadUrlHost,
reconciliationBatchId: row.reconciliationBatchId,
requestedBy: row.requestedBy,
claimedBy: row.claimedBy,
claimedAt: row.claimedAt,
completedAt: row.completedAt,
failedAt: row.failedAt,
errorCode: row.errorCode,
errorMessage: row.errorMessage,
metadata: row.metadata || {},
createdAt: row.createdAt,
updatedAt: row.updatedAt,
};
}
export async function requestProviderBillDownloadRoute(ctx: RequestContext) {
const auth = await authorizeDownload(ctx);
const body = await readJsonBody(ctx);
const provider = normalizeReconciliationProvider(requiredString(body, 'provider'));
if (provider === 'manual') {
throw new HttpError(400, 'Manual provider does not support official bill download', 'PROVIDER_BILL_DOWNLOAD_UNSUPPORTED');
}
const billDate = normalizeReconciliationBillDate(requiredString(body, 'billDate'));
const billType = normalizeReconciliationBillType(body.billType ?? body.bill_type);
const metadata = objectValue(body.metadata);
const sourceName = `provider-bill:${provider}:${billDate}:${billType}`;
const result = await transaction(async client => {
const inserted = await client.query<Record<string, unknown>>(
`
insert into public.commerce_bill_download_jobs (
tenant_id, provider, bill_date, bill_type, status,
source_name, requested_by, metadata
)
values ($1, $2, $3::date, $4, 'queued', $5, $6::uuid, $7::jsonb)
on conflict (tenant_id, provider, bill_date, bill_type)
where status in ('queued', 'running', 'completed')
do update set
metadata = public.commerce_bill_download_jobs.metadata || excluded.metadata,
updated_at = now()
returning id, provider, bill_date as "billDate", bill_type as "billType",
status, source_name as "sourceName", source_hash as "sourceHash",
row_count as "rowCount", download_hash_type as "downloadHashType",
download_hash_value as "downloadHashValue", download_url_host as "downloadUrlHost",
reconciliation_batch_id as "reconciliationBatchId",
requested_by as "requestedBy", claimed_by as "claimedBy",
claimed_at as "claimedAt", completed_at as "completedAt",
failed_at as "failedAt", error_code as "errorCode",
error_message as "errorMessage", metadata,
created_at as "createdAt", updated_at as "updatedAt",
(xmax = 0) as "inserted"
`,
[
auth.tenantId,
provider,
billDate,
billType,
sourceName,
auth.userId,
JSON.stringify({
...metadata,
requestedFrom: 'api',
}),
],
);
const job = inserted.rows[0];
await recordReconciliationAudit(
client,
auth,
job.inserted ? 'commerce.provider_bill_download.requested' : 'commerce.provider_bill_download.request_reused',
String(job.id),
{
provider,
billDate,
billType,
sourceName,
status: job.status,
},
);
return job;
});
return {
item: providerBillJobPayload(result),
idempotent: result.inserted === false,
};
}
export async function providerBillDownloadJobsRoute(ctx: RequestContext) {
const auth = await authorizeRead(ctx);
const limit = intParam(ctx, 'limit', 50, 200);
const provider = stringParam(ctx, 'provider');
const status = stringParam(ctx, 'status');
const billDate = stringParam(ctx, 'billDate');
const params: unknown[] = [auth.tenantId, limit];
const where = ['tenant_id = $1'];
if (provider) {
const normalized = normalizeReconciliationProvider(provider);
if (normalized === 'manual') throw new HttpError(400, 'provider is invalid for bill download', 'PROVIDER_BILL_DOWNLOAD_UNSUPPORTED');
params.push(normalized);
where.push(`provider = $${params.length}`);
}
if (status) {
params.push(status);
where.push(`status = $${params.length}`);
}
if (billDate) {
params.push(normalizeReconciliationBillDate(billDate));
where.push(`bill_date = $${params.length}::date`);
}
const items = await query<Record<string, unknown>>(
`
select id, provider, bill_date as "billDate", bill_type as "billType",
status, source_name as "sourceName", source_hash as "sourceHash",
row_count as "rowCount", download_hash_type as "downloadHashType",
download_hash_value as "downloadHashValue", download_url_host as "downloadUrlHost",
reconciliation_batch_id as "reconciliationBatchId",
requested_by as "requestedBy", claimed_by as "claimedBy",
claimed_at as "claimedAt", completed_at as "completedAt",
failed_at as "failedAt", error_code as "errorCode",
error_message as "errorMessage", metadata,
created_at as "createdAt", updated_at as "updatedAt"
from public.commerce_bill_download_jobs
where ${where.join(' and ')}
order by created_at desc
limit $2
`,
params,
);
return { items: items.map(providerBillJobPayload) };
}
export async function previewReconciliationRoute(ctx: RequestContext) {
const auth = await authorizeRead(ctx);
const body = await readJsonBody(ctx, { maxBytes: config.maxImportJsonBodyBytes });
@@ -1090,122 +1409,21 @@ export async function importReconciliationRoute(ctx: RequestContext) {
const metadata = objectValue(body.metadata);
const result = await transaction(async client => {
const built = await buildReconciliation(client, { tenantId: auth.tenantId, ...input });
const insertedBatch = await client.query<Record<string, unknown>>(
`
insert into public.commerce_reconciliation_batches (
tenant_id, provider, bill_date, bill_type, source, source_name, source_hash,
status, total_count, matched_count, mismatch_count, missing_local_count,
missing_provider_count, duplicate_count, ignored_count, amount_cents,
refund_amount_cents, fee_cents, created_by, completed_at, metadata
)
values (
$1, $2, $3::date, $4, $5, $6, $7,
$8, $9, $10, $11, $12,
$13, $14, $15, $16,
$17, $18, $19, now(), $20::jsonb
)
returning id, provider, bill_date as "billDate", bill_type as "billType",
source, source_name as "sourceName", source_hash as "sourceHash",
status, total_count as "totalCount", matched_count as "matchedCount",
mismatch_count as "mismatchCount", missing_local_count as "missingLocalCount",
missing_provider_count as "missingProviderCount", duplicate_count as "duplicateCount",
ignored_count as "ignoredCount", amount_cents as "amountCents",
refund_amount_cents as "refundAmountCents", fee_cents as "feeCents",
created_by as "createdBy", completed_at as "completedAt", error,
metadata, created_at as "createdAt", updated_at as "updatedAt"
`,
[
auth.tenantId,
built.batch.provider,
built.batch.billDate,
built.batch.billType,
built.batch.source,
built.batch.sourceName,
built.batch.sourceHash,
built.batch.status,
built.batch.totalCount,
built.batch.matchedCount,
built.batch.mismatchCount,
built.batch.missingLocalCount,
built.batch.missingProviderCount,
built.batch.duplicateCount,
built.batch.ignoredCount,
built.batch.amountCents,
built.batch.refundAmountCents,
built.batch.feeCents,
auth.userId,
JSON.stringify(metadata),
],
);
const batch = insertedBatch.rows[0];
const batchId = String(batch.id);
for (const item of built.items) {
await client.query(
`
insert into public.commerce_reconciliation_items (
tenant_id, batch_id, row_no, provider, transaction_type,
provider_trade_no, provider_refund_no, order_no, refund_no,
amount_cents, refund_amount_cents, fee_cents, paid_at, refunded_at,
provider_status, local_status, order_id, payment_id, refund_request_id,
match_status, severity, issue_code, details
)
values (
$1, $2, $3, $4, $5,
$6, $7, $8, $9,
$10, $11, $12, $13::timestamptz, $14::timestamptz,
$15, $16, $17::uuid, $18::uuid, $19::uuid,
$20, $21, $22, $23::jsonb
)
`,
[
auth.tenantId,
batchId,
item.rowNo,
item.provider,
item.transactionType,
item.providerTradeNo,
item.providerRefundNo,
item.orderNo,
item.refundNo,
item.amountCents,
item.refundAmountCents,
item.feeCents,
item.paidAt,
item.refundedAt,
item.providerStatus,
item.localStatus,
item.orderId,
item.paymentId,
item.refundRequestId,
item.matchStatus,
item.severity,
item.issueCode,
JSON.stringify(item.details),
],
);
}
await recordReconciliationAudit(client, auth, 'commerce.reconciliation.imported', batchId, {
provider: built.batch.provider,
billDate: built.batch.billDate,
billType: built.batch.billType,
source: built.batch.source,
sourceName: built.batch.sourceName,
sourceHash: built.batch.sourceHash,
summary: {
totalCount: built.batch.totalCount,
matchedCount: built.batch.matchedCount,
mismatchCount: built.batch.mismatchCount,
missingLocalCount: built.batch.missingLocalCount,
missingProviderCount: built.batch.missingProviderCount,
duplicateCount: built.batch.duplicateCount,
ignoredCount: built.batch.ignoredCount,
},
const imported = await importReconciliationBatch(client, {
tenantId: auth.tenantId,
actorUserId: auth.userId,
provider: input.provider,
billDate: input.billDate,
billType: input.billType,
source: input.source,
sourceName: input.sourceName,
rows: body.rows as unknown[],
metadata,
});
return { batch, previewItems: built.items.slice(0, optionalInteger(body, 'previewLimit', 50)) };
return {
batch: imported.batch,
previewItems: imported.built.items.slice(0, optionalInteger(body, 'previewLimit', 50)),
};
});
return {
@@ -1224,7 +1442,7 @@ export async function reconciliationBatchesRoute(ctx: RequestContext) {
const params: unknown[] = [auth.tenantId, limit];
const where = ['tenant_id = $1'];
if (provider) {
params.push(normalizeProvider(provider));
params.push(normalizeReconciliationProvider(provider));
where.push(`provider = $${params.length}`);
}
if (status) {
@@ -1232,7 +1450,7 @@ export async function reconciliationBatchesRoute(ctx: RequestContext) {
where.push(`status = $${params.length}`);
}
if (billDate) {
params.push(normalizeBillDate(billDate));
params.push(normalizeReconciliationBillDate(billDate));
where.push(`bill_date = $${params.length}::date`);
}
@@ -1309,7 +1527,7 @@ export async function reconciliationAnomaliesRoute(ctx: RequestContext) {
const auth = await authorizeRead(ctx);
const limit = intParam(ctx, 'limit', 50, 200);
const providerParam = stringParam(ctx, 'provider');
const provider = providerParam ? normalizeProvider(providerParam) : '';
const provider = providerParam ? normalizeReconciliationProvider(providerParam) : '';
const issueParams: unknown[] = [auth.tenantId, limit];
const issueWhere = [`i.tenant_id = $1`, `i.match_status not in ('matched', 'ignored')`];

View File

@@ -92,6 +92,7 @@ export function tenantPermissionCatalog() {
{ key: 'tenant:payment:write', label: '商户配置管理' },
{ key: 'tenant:reconciliation:read', label: '资金对账查看' },
{ key: 'tenant:reconciliation:write', label: '资金对账导入' },
{ key: 'tenant:reconciliation:download', label: '官方账单下载' },
{ key: 'tenant:refund:read', label: '退款查看' },
{ key: 'tenant:refund:write', label: '退款申请/处理' },
{ key: 'tenant:refund:review', label: '退款审核' },

View File

@@ -10,6 +10,7 @@
"check": "tsc -p tsconfig.json --noEmit",
"crm:once": "tsx src/index.ts --once --job crm",
"commerce:once": "tsx src/index.ts --once --job commerce",
"provider-bills:once": "tsx src/index.ts --once --job provider-bills",
"assets:once": "tsx src/index.ts --once --job assets",
"imports:once": "tsx src/index.ts --once --job imports",
"public-banks:once": "tsx src/index.ts --once --job public-banks",
@@ -20,6 +21,7 @@
"@supabase/storage-js": "^2.108.2",
"ali-oss": "^6.23.0",
"docx": "^9.7.1",
"iconv-lite": "^0.6.3",
"jszip": "^3.10.1",
"pdfkit": "^0.19.1",
"pg": "^8.16.3"

View File

@@ -13,6 +13,9 @@ export interface WorkerConfig {
commerceBatchSize: number;
commerceMinAgeSeconds: number;
commerceRequestTimeoutMs: number;
providerBillBatchSize: number;
providerBillWorkerId: string;
providerBillClaimStaleSeconds: number;
assetBatchSize: number;
assetMinAgeSeconds: number;
assetRecheckIntervalSeconds: number;
@@ -69,6 +72,9 @@ export const config: WorkerConfig = {
commerceBatchSize: envNumber('WORKER_COMMERCE_BATCH_SIZE', 20),
commerceMinAgeSeconds: envNumber('WORKER_COMMERCE_MIN_AGE_SECONDS', 300),
commerceRequestTimeoutMs: envNumber('WORKER_COMMERCE_REQUEST_TIMEOUT_MS', 10_000),
providerBillBatchSize: envNumber('WORKER_PROVIDER_BILL_BATCH_SIZE', 5),
providerBillWorkerId: envString('WORKER_PROVIDER_BILL_ID', `provider-bills-${process.pid}`),
providerBillClaimStaleSeconds: envNumber('WORKER_PROVIDER_BILL_CLAIM_STALE_SECONDS', 15 * 60),
assetBatchSize: envNumber('WORKER_ASSET_BATCH_SIZE', 50),
assetMinAgeSeconds: envNumber('WORKER_ASSET_MIN_AGE_SECONDS', 300),
assetRecheckIntervalSeconds: envNumber('WORKER_ASSET_RECHECK_INTERVAL_SECONDS', 60 * 60 * 24),

View File

@@ -31,6 +31,17 @@ async function runOnce() {
);
return;
}
if (job === 'provider-bills') {
const { closePool: closeApiPool } = await import('../../api/src/core/db.js');
const { processProviderBillBatch } = await import('./jobs/provider-bills.js');
extraClosers.add(closeApiPool);
const result = await processProviderBillBatch();
console.log(
`[worker] provider-bills batch processed=${result.processed}`
+ ` completed=${result.completed} failed=${result.failed} skipped=${result.skipped}`,
);
return;
}
if (job === 'assets') {
const result = await processAssetBatch();
console.log(

View File

@@ -0,0 +1,784 @@
import crypto from 'node:crypto';
import JSZip from 'jszip';
import iconv from 'iconv-lite';
import type pg from 'pg';
import { pool } from '../db.js';
import { config } from '../config.js';
import {
importReconciliationBatch,
normalizeReconciliationBillDate,
normalizeReconciliationBillType,
normalizeReconciliationProvider,
type BillType,
type ReconciliationProvider,
} from '../../../api/src/features/commerce/reconciliation.js';
type ProviderBillProvider = Exclude<ReconciliationProvider, 'manual'>;
interface SecretRow {
secretValue: string | null;
secretJson: Record<string, unknown> | null;
}
interface ProviderConfig {
tenantId: string;
provider: ProviderBillProvider;
configPublic: Record<string, unknown>;
secret: SecretRow | null;
}
interface ProviderBillJob {
id: string;
tenantId: string;
provider: ProviderBillProvider;
billDate: string;
billType: BillType;
sourceName: string;
requestedBy: string | null;
metadata: Record<string, unknown>;
}
interface DownloadedBill {
rows: Record<string, unknown>[];
sourceHash: string;
downloadHashType: string | null;
downloadHashValue: string | null;
downloadUrlHost: string | null;
metadata: Record<string, unknown>;
}
interface ProviderBillWorkerResult {
processed: number;
completed: number;
failed: number;
skipped: number;
}
class ProviderBillWorkerError extends Error {
readonly code: string;
constructor(message: string, code = 'PROVIDER_BILL_WORKER_ERROR') {
super(message);
this.code = code;
}
}
function objectValue(value: unknown): Record<string, unknown> {
return value && typeof value === 'object' && !Array.isArray(value) ? value as Record<string, unknown> : {};
}
function safeString(value: unknown) {
return typeof value === 'string' && value.trim() ? value.trim() : '';
}
function optionalPublicString(providerConfig: ProviderConfig, keys: string[]) {
for (const key of keys) {
const value = providerConfig.configPublic[key];
if (typeof value === 'string' && value.trim()) return value.trim();
}
return '';
}
function requirePublicString(providerConfig: ProviderConfig, keys: string[]) {
const value = optionalPublicString(providerConfig, keys);
if (value) return value;
throw new ProviderBillWorkerError(`${providerConfig.provider} public config is missing ${keys[0]}`, 'PAYMENT_PUBLIC_CONFIG_REQUIRED');
}
function secretString(providerConfig: ProviderConfig, keys: string[]) {
const secret = providerConfig.secret;
if (!secret) return '';
if (typeof secret.secretValue === 'string' && secret.secretValue.trim()) return secret.secretValue.trim();
const json = objectValue(secret.secretJson);
for (const key of keys) {
const value = json[key];
if (typeof value === 'string' && value.trim()) return value.trim();
}
return '';
}
function requireSecretString(providerConfig: ProviderConfig, keys: string[]) {
const value = secretString(providerConfig, keys);
if (value) return value;
throw new ProviderBillWorkerError(`${providerConfig.provider} secret is missing ${keys[0]}`, 'PAYMENT_SECRET_REQUIRED');
}
function normalizePem(value: string, label: 'PRIVATE KEY' | 'PUBLIC KEY') {
if (value.includes('-----BEGIN')) return value;
const wrapped = value.match(/.{1,64}/g)?.join('\n') || value;
return `-----BEGIN ${label}-----\n${wrapped}\n-----END ${label}-----`;
}
function rsaSignSha256(privateKey: string, message: string) {
return crypto.createSign('RSA-SHA256').update(message, 'utf8').sign(privateKey, 'base64');
}
function randomNonce(size = 16) {
return crypto.randomBytes(size).toString('base64url');
}
function canonicalForm(params: Record<string, string>) {
return Object.keys(params)
.filter(key => params[key] !== undefined && params[key] !== null && params[key] !== '')
.sort()
.map(key => `${key}=${params[key]}`)
.join('&');
}
function encodedForm(params: Record<string, string>) {
return Object.keys(params)
.filter(key => params[key] !== undefined && params[key] !== null && params[key] !== '')
.sort()
.map(key => `${encodeURIComponent(key)}=${encodeURIComponent(params[key])}`)
.join('&');
}
function parseSecretRef(value: unknown) {
if (typeof value !== 'string' || !value.trim()) return null;
const parts = value.trim().split(':');
if (parts.length !== 3 || parts[0] !== 'app_private.tenant_secrets') return null;
if (parts[1] !== 'payment' || !parts[2]) return null;
return { scope: parts[1], key: parts[2] };
}
function normalizeProvider(value: string | null | undefined): ProviderBillProvider | null {
const normalized = normalizeReconciliationProvider(String(value || ''));
return normalized === 'manual' ? null : normalized;
}
function providerAliases(provider: ProviderBillProvider) {
return provider === 'wechat_pay'
? ['wechat_pay', 'wechat-pay', 'wechatpay', 'wxpay']
: ['alipay', 'ali_pay'];
}
function isLocalDevHost(hostname: string) {
return ['127.0.0.1', 'localhost', '::1'].includes(hostname);
}
function isAllowedHost(hostname: string, allowedHosts: string[]) {
return allowedHosts.some(host => hostname === host || hostname.endsWith(`.${host}`));
}
function providerEndpointForKeys(
providerConfig: ProviderConfig,
keys: string[],
fallback: string,
allowedHosts: string[],
code = 'PAYMENT_ENDPOINT_NOT_ALLOWED',
) {
const raw = optionalPublicString(providerConfig, keys) || fallback;
let endpoint: URL;
try {
endpoint = new URL(raw);
} catch {
throw new ProviderBillWorkerError(`${providerConfig.provider} endpoint is invalid`, 'PAYMENT_ENDPOINT_INVALID');
}
const localDev = process.env.NODE_ENV !== 'production' && isLocalDevHost(endpoint.hostname);
if (!localDev && endpoint.protocol !== 'https:') {
throw new ProviderBillWorkerError(`${providerConfig.provider} endpoint must use HTTPS`, code);
}
if (!localDev && !isAllowedHost(endpoint.hostname, allowedHosts)) {
throw new ProviderBillWorkerError(`${providerConfig.provider} endpoint host is not allowed`, code);
}
endpoint.username = '';
endpoint.password = '';
return endpoint.toString();
}
function assertDownloadUrlAllowed(url: string, provider: ProviderBillProvider) {
let parsed: URL;
try {
parsed = new URL(url);
} catch {
throw new ProviderBillWorkerError('Provider bill download URL is invalid', 'PROVIDER_BILL_DOWNLOAD_URL_INVALID');
}
const localDev = process.env.NODE_ENV !== 'production' && isLocalDevHost(parsed.hostname);
if (localDev) return parsed;
if (parsed.protocol !== 'https:') {
throw new ProviderBillWorkerError('Provider bill download URL must use HTTPS', 'PROVIDER_BILL_DOWNLOAD_URL_NOT_ALLOWED');
}
const hosts = provider === 'wechat_pay'
? ['api.mch.weixin.qq.com', 'download.mch.weixin.qq.com']
: ['openapi.alipay.com', 'dwbillcenter.alipay.com', 'download.alipay.com'];
if (!isAllowedHost(parsed.hostname, hosts)) {
throw new ProviderBillWorkerError('Provider bill download URL host is not allowed', 'PROVIDER_BILL_DOWNLOAD_URL_NOT_ALLOWED');
}
return parsed;
}
async function loadSecret(client: pg.PoolClient, tenantId: string, configPublic: Record<string, unknown>, provider: ProviderBillProvider) {
const secretRef = parseSecretRef(configPublic.secretRef);
const secretKey = secretRef?.key || provider;
const result = await client.query<SecretRow>(
`
select secret_value as "secretValue", secret_json as "secretJson"
from app_private.tenant_secrets
where tenant_id = $1
and secret_scope = 'payment'
and secret_key = $2
limit 1
`,
[tenantId, secretKey],
);
return result.rows[0] || null;
}
async function loadPaymentProviderConfig(client: pg.PoolClient, tenantId: string, provider: ProviderBillProvider): Promise<ProviderConfig> {
const aliases = providerAliases(provider);
const accountResult = await client.query<{
provider: string;
configPublic: Record<string, unknown> | null;
}>(
`
select provider, config_public as "configPublic"
from public.tenant_payment_accounts
where tenant_id = $1
and provider = any($2::text[])
and status = 'active'
order by array_position($2::text[], provider)
limit 1
`,
[tenantId, aliases],
);
const account = accountResult.rows[0];
if (!account) throw new ProviderBillWorkerError(`${provider} provider is not configured`, 'PAYMENT_PROVIDER_NOT_CONFIGURED');
const normalized = normalizeProvider(account.provider);
if (!normalized) throw new ProviderBillWorkerError(`${account.provider} provider is unsupported`, 'PAYMENT_PROVIDER_NOT_SUPPORTED');
const configPublic = objectValue(account.configPublic);
const secret = await loadSecret(client, tenantId, configPublic, normalized);
return { tenantId, provider: normalized, configPublic, secret };
}
async function fetchJson(url: string, init: RequestInit, timeoutMs: number) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
const response = await fetch(url, { ...init, signal: controller.signal });
const raw = objectValue(await response.json().catch(() => ({})));
return { response, raw };
} finally {
clearTimeout(timeout);
}
}
async function fetchBuffer(url: string, init: RequestInit, timeoutMs: number) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
const response = await fetch(url, { ...init, signal: controller.signal });
const buffer = Buffer.from(await response.arrayBuffer());
return { response, buffer };
} finally {
clearTimeout(timeout);
}
}
function wechatPrivateKey(providerConfig: ProviderConfig) {
return normalizePem(requireSecretString(providerConfig, ['privateKey', 'merchantPrivateKey']), 'PRIVATE KEY');
}
function signedWechatGetHeaders(providerConfig: ProviderConfig, url: URL) {
const mchId = requirePublicString(providerConfig, ['merchantId', 'mchId']);
const merchantSerialNo = requirePublicString(providerConfig, ['merchantSerialNo']);
const timestamp = Math.floor(Date.now() / 1000).toString();
const nonce = randomNonce();
const message = ['GET', `${url.pathname}${url.search}`, timestamp, nonce, ''].join('\n') + '\n';
const signature = rsaSignSha256(wechatPrivateKey(providerConfig), message);
return {
accept: 'application/json',
authorization: `WECHATPAY2-SHA256-RSA2048 mchid="${mchId}",nonce_str="${nonce}",signature="${signature}",timestamp="${timestamp}",serial_no="${merchantSerialNo}"`,
};
}
function wechatBillEndpoint(providerConfig: ProviderConfig, billType: BillType) {
if (billType === 'refund') {
return providerEndpointForKeys(
providerConfig,
['wechatRefundBillEndpoint', 'refundBillEndpoint', 'tradeBillEndpoint'],
'https://api.mch.weixin.qq.com/v3/bill/tradebill',
['api.mch.weixin.qq.com'],
);
}
return providerEndpointForKeys(
providerConfig,
['wechatTradeBillEndpoint', 'tradeBillEndpoint'],
'https://api.mch.weixin.qq.com/v3/bill/tradebill',
['api.mch.weixin.qq.com'],
);
}
function mapWechatBillType(billType: BillType) {
if (billType === 'refund') return 'REFUND';
return 'ALL';
}
async function downloadWechatBill(providerConfig: ProviderConfig, job: ProviderBillJob): Promise<DownloadedBill> {
const endpoint = wechatBillEndpoint(providerConfig, job.billType);
const url = new URL(endpoint);
url.searchParams.set('bill_date', job.billDate);
url.searchParams.set('bill_type', mapWechatBillType(job.billType));
const { response, raw } = await fetchJson(url.toString(), {
method: 'GET',
headers: signedWechatGetHeaders(providerConfig, url),
}, config.commerceRequestTimeoutMs);
if (!response.ok) throw new ProviderBillWorkerError('WeChat Pay bill URL request failed', 'PROVIDER_BILL_URL_REQUEST_FAILED');
const downloadUrl = safeString(raw.download_url);
if (!downloadUrl) throw new ProviderBillWorkerError('WeChat Pay bill response missing download_url', 'PROVIDER_BILL_RESPONSE_INVALID');
const hashType = safeString(raw.hash_type) || null;
const hashValue = safeString(raw.hash_value) || null;
const parsedDownloadUrl = assertDownloadUrlAllowed(downloadUrl, 'wechat_pay');
const { response: downloadResponse, buffer } = await fetchBuffer(downloadUrl, {
method: 'GET',
headers: signedWechatGetHeaders(providerConfig, parsedDownloadUrl),
}, config.commerceRequestTimeoutMs);
if (!downloadResponse.ok) throw new ProviderBillWorkerError('WeChat Pay bill download failed', 'PROVIDER_BILL_DOWNLOAD_FAILED');
verifyProviderHash(buffer, hashType, hashValue);
const rows = await parseBillRows(buffer, downloadResponse.headers.get('content-type') || '', parsedDownloadUrl.pathname, 'wechat_pay');
return {
rows,
sourceHash: crypto.createHash('sha256').update(buffer).digest('hex'),
downloadHashType: hashType,
downloadHashValue: hashValue,
downloadUrlHost: parsedDownloadUrl.hostname,
metadata: {
providerResponse: {
hashType,
hasHashValue: Boolean(hashValue),
},
downloadedBytes: buffer.length,
},
};
}
function alipayBillType(billType: BillType) {
if (billType === 'refund') return 'trade';
return 'trade';
}
async function downloadAlipayBill(providerConfig: ProviderConfig, job: ProviderBillJob): Promise<DownloadedBill> {
const appId = requirePublicString(providerConfig, ['appId']);
const gateway = providerEndpointForKeys(
providerConfig,
['billEndpoint', 'endpoint'],
'https://openapi.alipay.com/gateway.do',
['openapi.alipay.com'],
);
const privateKey = normalizePem(requireSecretString(providerConfig, ['privateKey', 'appPrivateKey']), 'PRIVATE KEY');
const params: Record<string, string> = {
app_id: appId,
method: 'alipay.data.dataservice.bill.downloadurl.query',
charset: 'utf-8',
sign_type: 'RSA2',
timestamp: new Date().toISOString().replace('T', ' ').slice(0, 19),
version: '1.0',
biz_content: JSON.stringify({
bill_type: alipayBillType(job.billType),
bill_date: job.billDate,
}),
};
params.sign = rsaSignSha256(privateKey, canonicalForm(params));
const { response, raw } = await fetchJson(gateway, {
method: 'POST',
headers: {
accept: 'application/json',
'content-type': 'application/x-www-form-urlencoded;charset=utf-8',
},
body: encodedForm(params),
}, config.commerceRequestTimeoutMs);
if (!response.ok) throw new ProviderBillWorkerError('Alipay bill URL request failed', 'PROVIDER_BILL_URL_REQUEST_FAILED');
const responseBody = objectValue(raw.alipay_data_dataservice_bill_downloadurl_query_response);
if (safeString(responseBody.code) !== '10000') {
throw new ProviderBillWorkerError('Alipay bill URL request was rejected', 'PROVIDER_BILL_URL_REQUEST_REJECTED');
}
const downloadUrl = safeString(responseBody.bill_download_url);
if (!downloadUrl) throw new ProviderBillWorkerError('Alipay bill response missing bill_download_url', 'PROVIDER_BILL_RESPONSE_INVALID');
const parsedDownloadUrl = assertDownloadUrlAllowed(downloadUrl, 'alipay');
const { response: downloadResponse, buffer } = await fetchBuffer(downloadUrl, {
method: 'GET',
headers: { accept: '*/*' },
}, config.commerceRequestTimeoutMs);
if (!downloadResponse.ok) throw new ProviderBillWorkerError('Alipay bill download failed', 'PROVIDER_BILL_DOWNLOAD_FAILED');
const rows = await parseBillRows(buffer, downloadResponse.headers.get('content-type') || '', parsedDownloadUrl.pathname, 'alipay');
return {
rows,
sourceHash: crypto.createHash('sha256').update(buffer).digest('hex'),
downloadHashType: null,
downloadHashValue: null,
downloadUrlHost: parsedDownloadUrl.hostname,
metadata: {
providerResponse: {
code: responseBody.code,
},
downloadedBytes: buffer.length,
},
};
}
function verifyProviderHash(buffer: Buffer, hashType: string | null, hashValue: string | null) {
if (!hashType && !hashValue) return;
if (!hashType || !hashValue) {
throw new ProviderBillWorkerError('Provider bill hash metadata is incomplete', 'PROVIDER_BILL_HASH_INVALID');
}
const normalized = hashType.trim().toUpperCase();
const algorithm = normalized === 'SHA1' ? 'sha1' : normalized === 'SHA256' ? 'sha256' : '';
if (!algorithm) {
throw new ProviderBillWorkerError(`Provider bill hash type ${hashType} is unsupported`, 'PROVIDER_BILL_HASH_UNSUPPORTED');
}
const actual = crypto.createHash(algorithm).update(buffer).digest('hex');
if (actual.toLowerCase() !== hashValue.toLowerCase()) {
throw new ProviderBillWorkerError('Provider bill hash mismatch', 'PROVIDER_BILL_HASH_MISMATCH');
}
}
function decodeText(buffer: Buffer) {
if (buffer.length >= 3 && buffer[0] === 0xef && buffer[1] === 0xbb && buffer[2] === 0xbf) {
return buffer.subarray(3).toString('utf8');
}
const utf8 = buffer.toString('utf8');
if (utf8.includes('<27>')) {
return iconv.decode(buffer, 'gb18030');
}
return utf8;
}
async function parseBillRows(buffer: Buffer, contentType: string, pathname: string, provider: ProviderBillProvider) {
const lowerPath = pathname.toLowerCase();
const lowerType = contentType.toLowerCase();
if (lowerPath.endsWith('.zip') || lowerType.includes('zip')) {
const zip = await JSZip.loadAsync(buffer);
const rows: Record<string, unknown>[] = [];
for (const entry of Object.values(zip.files)) {
if (entry.dir) continue;
const name = entry.name.toLowerCase();
if (!name.endsWith('.csv') && !name.endsWith('.txt') && !name.endsWith('.json')) continue;
const entryBuffer = Buffer.from(await entry.async('uint8array'));
rows.push(...parseBillRowsFromText(decodeText(entryBuffer), provider));
}
return rows;
}
return parseBillRowsFromText(decodeText(buffer), provider);
}
function parseBillRowsFromText(text: string, provider: ProviderBillProvider) {
const trimmed = text.trim();
if (!trimmed) return [];
if (trimmed.startsWith('{') || trimmed.startsWith('[')) {
const parsed = JSON.parse(trimmed) as unknown;
const parsedObject = objectValue(parsed);
const rows: unknown[] = Array.isArray(parsed)
? parsed
: Array.isArray(parsedObject.rows)
? parsedObject.rows
: [];
return rows.map(row => objectValue(row));
}
return parseDelimitedBillRows(text, provider);
}
function parseDelimitedBillRows(text: string, provider: ProviderBillProvider) {
const lines = text
.split(/\r?\n/)
.map(line => line.trim())
.filter(line => line && !line.startsWith('#') && !line.startsWith('`----------'));
const headerIndex = lines.findIndex(line => {
const normalized = line.replace(/`/g, '');
return normalized.includes(',') && (
normalized.includes('商户订单号')
|| normalized.includes('out_trade_no')
|| normalized.includes('providerTradeNo')
|| normalized.includes('交易号')
|| normalized.includes('微信支付订单号')
);
});
if (headerIndex < 0) {
throw new ProviderBillWorkerError('Provider bill header not found', 'PROVIDER_BILL_PARSE_FAILED');
}
const delimiter = lines[headerIndex].includes('\t') ? '\t' : ',';
const headers = splitDelimitedLine(lines[headerIndex], delimiter).map(cleanCell);
const rows: Record<string, unknown>[] = [];
for (const line of lines.slice(headerIndex + 1)) {
const values = splitDelimitedLine(line, delimiter).map(cleanCell);
if (values.length < 2) continue;
const raw: Record<string, unknown> = {};
headers.forEach((header, index) => {
if (header) raw[header] = values[index] || '';
});
const normalized = provider === 'wechat_pay' ? normalizeWechatBillRow(raw) : normalizeAlipayBillRow(raw);
if (Object.keys(normalized).length > 0) rows.push(normalized);
}
return rows;
}
function splitDelimitedLine(line: string, delimiter: string) {
if (delimiter === '\t') return line.split('\t');
const cells: string[] = [];
let current = '';
let quoted = false;
for (let index = 0; index < line.length; index += 1) {
const char = line[index];
if (char === '"') {
if (quoted && line[index + 1] === '"') {
current += '"';
index += 1;
} else {
quoted = !quoted;
}
} else if (char === ',' && !quoted) {
cells.push(current);
current = '';
} else {
current += char;
}
}
cells.push(current);
return cells;
}
function cleanCell(value: string) {
return value.trim().replace(/^`/, '').replace(/^"|"$/g, '').trim();
}
function firstValue(row: Record<string, unknown>, keys: string[]) {
for (const key of keys) {
const value = row[key];
if (typeof value === 'string' && value.trim()) return value.trim();
if (typeof value === 'number' && Number.isFinite(value)) return String(value);
}
return '';
}
function normalizeAmount(value: string) {
if (!value) return '';
const cleaned = value.replace(/,/g, '').replace(/[¥¥]/g, '').trim();
const parsed = Number(cleaned);
return Number.isFinite(parsed) ? cleaned : '';
}
function rowTypeFromText(value: string) {
const normalized = value.toLowerCase();
if (normalized.includes('refund') || value.includes('退款')) return 'refund';
return 'payment';
}
function normalizeWechatBillRow(row: Record<string, unknown>) {
const transactionType = rowTypeFromText(firstValue(row, ['账单类型', '交易类型', 'transaction_type', 'type']));
const amount = normalizeAmount(firstValue(row, ['订单金额(元)', '应结订单金额(元)', '总金额', 'amount', 'total_amount']));
const refundAmount = normalizeAmount(firstValue(row, ['申请退款金额(元)', '退款金额(元)', 'refund_amount', 'refund_fee']));
return {
transactionType,
providerTradeNo: firstValue(row, ['微信支付订单号', 'transaction_id', 'providerTradeNo']),
providerRefundNo: firstValue(row, ['微信退款单号', 'refund_id', 'providerRefundNo']),
orderNo: firstValue(row, ['商户订单号', 'out_trade_no', 'orderNo']),
refundNo: firstValue(row, ['商户退款单号', 'out_refund_no', 'refundNo']),
amountYuan: transactionType === 'payment' ? amount : '',
refundAmount: transactionType === 'refund' ? (refundAmount || amount) : '',
feeYuan: normalizeAmount(firstValue(row, ['手续费(元)', 'service_fee', 'fee'])),
paidAt: firstValue(row, ['支付完成时间', '交易时间', 'success_time', 'paidAt']),
refundedAt: firstValue(row, ['退款成功时间', 'refund_success_time', 'refundedAt']),
providerStatus: firstValue(row, ['交易状态', '退款状态', 'trade_state', 'status']) || 'SUCCESS',
rawProviderRow: row,
};
}
function normalizeAlipayBillRow(row: Record<string, unknown>) {
const transactionType = rowTypeFromText(firstValue(row, ['业务类型', '交易类型', 'transaction_type', 'type']));
const amount = normalizeAmount(firstValue(row, ['订单金额(元)', '订单金额(元)', '收入(+元)', '收入(+元)', 'amount', 'total_amount']));
const refundAmount = normalizeAmount(firstValue(row, ['退款金额(元)', '退款金额(元)', '支出(-元)', '支出(-元)', 'refund_amount']));
return {
transactionType,
providerTradeNo: firstValue(row, ['支付宝交易号', '交易号', 'trade_no', 'providerTradeNo']),
providerRefundNo: firstValue(row, ['支付宝退款单号', 'refund_id', 'providerRefundNo']),
orderNo: firstValue(row, ['商户订单号', 'out_trade_no', 'orderNo']),
refundNo: firstValue(row, ['商户退款单号', 'out_request_no', 'refundNo']),
amountYuan: transactionType === 'payment' ? amount : '',
refundAmount: transactionType === 'refund' ? (refundAmount || amount) : '',
feeYuan: normalizeAmount(firstValue(row, ['服务费(元)', '服务费(元)', 'fee', 'service_fee'])),
paidAt: firstValue(row, ['交易创建时间', '付款时间', 'gmt_payment', 'paidAt']),
refundedAt: firstValue(row, ['退款时间', 'gmt_refund', 'refundedAt']),
providerStatus: firstValue(row, ['交易状态', 'trade_status', 'status']) || 'TRADE_SUCCESS',
rawProviderRow: row,
};
}
async function claimProviderBillJobs(client: pg.PoolClient, limit = config.providerBillBatchSize) {
const result = await client.query<ProviderBillJob>(
`
update public.commerce_bill_download_jobs j
set status = 'running',
claimed_by = $2,
claimed_at = now(),
updated_at = now(),
error_code = null,
error_message = null
where j.id in (
select id
from public.commerce_bill_download_jobs
where status = 'queued'
or (
status = 'running'
and claimed_at < now() - ($3::int * interval '1 second')
)
order by created_at asc
limit $1
for update skip locked
)
returning id, tenant_id as "tenantId", provider,
bill_date::text as "billDate", bill_type as "billType",
source_name as "sourceName", requested_by as "requestedBy",
metadata
`,
[limit, config.providerBillWorkerId, config.providerBillClaimStaleSeconds],
);
return result.rows.map(row => ({
...row,
provider: normalizeProvider(row.provider) || 'wechat_pay',
billDate: normalizeReconciliationBillDate(row.billDate),
billType: normalizeReconciliationBillType(row.billType),
metadata: objectValue(row.metadata),
}));
}
async function processProviderBillJob(job: ProviderBillJob) {
const client = await pool.connect();
try {
const providerConfig = await loadPaymentProviderConfig(client, job.tenantId, job.provider);
const downloaded = job.provider === 'wechat_pay'
? await downloadWechatBill(providerConfig, job)
: await downloadAlipayBill(providerConfig, job);
await client.query('begin');
const imported = await importReconciliationBatch(client, {
tenantId: job.tenantId,
actorUserId: job.requestedBy,
provider: job.provider,
billDate: job.billDate,
billType: job.billType,
source: 'provider_download',
sourceName: `${job.sourceName}:${job.id}`,
rows: downloaded.rows,
metadata: {
...job.metadata,
providerBillJobId: job.id,
providerBillSourceHash: downloaded.sourceHash,
providerBill: downloaded.metadata,
},
});
await client.query(
`
update public.commerce_bill_download_jobs
set status = 'completed',
source_hash = $3,
row_count = $4,
download_hash_type = $5,
download_hash_value = $6,
download_url_host = $7,
reconciliation_batch_id = $8::uuid,
completed_at = now(),
failed_at = null,
error_code = null,
error_message = null,
metadata = metadata || $9::jsonb,
updated_at = now()
where tenant_id = $1 and id = $2
`,
[
job.tenantId,
job.id,
downloaded.sourceHash,
downloaded.rows.length,
downloaded.downloadHashType,
downloaded.downloadHashValue,
downloaded.downloadUrlHost,
imported.batch.id,
JSON.stringify({
completedBy: config.providerBillWorkerId,
completedAt: new Date().toISOString(),
reconciliationBatchId: imported.batch.id,
reconciliationStatus: imported.batch.status,
reconciliationSummary: {
totalCount: imported.batch.totalCount,
matchedCount: imported.batch.matchedCount,
mismatchCount: imported.batch.mismatchCount,
missingLocalCount: imported.batch.missingLocalCount,
missingProviderCount: imported.batch.missingProviderCount,
duplicateCount: imported.batch.duplicateCount,
},
}),
],
);
await client.query('commit');
return 'completed';
} catch (error) {
await client.query('rollback').catch(() => {});
const failure = publicFailure(error);
await client.query(
`
update public.commerce_bill_download_jobs
set status = 'failed',
failed_at = now(),
error_code = $3,
error_message = $4,
metadata = metadata || $5::jsonb,
updated_at = now()
where tenant_id = $1 and id = $2
`,
[
job.tenantId,
job.id,
failure.code,
failure.message,
JSON.stringify({
failedBy: config.providerBillWorkerId,
failedAt: new Date().toISOString(),
}),
],
).catch(() => {});
return 'failed';
} finally {
client.release();
}
}
function publicFailure(error: unknown) {
const code = error instanceof ProviderBillWorkerError ? error.code : 'PROVIDER_BILL_WORKER_ERROR';
const message = error instanceof Error ? error.message : 'Provider bill worker failed';
return {
code,
message: message
.replace(/-----BEGIN[\s\S]+?-----END [A-Z ]+-----/g, '[redacted-pem]')
.slice(0, 500),
};
}
export async function processProviderBillBatch(limit = config.providerBillBatchSize): Promise<ProviderBillWorkerResult> {
const client = await pool.connect();
let jobs: ProviderBillJob[] = [];
try {
await client.query('begin');
jobs = await claimProviderBillJobs(client, limit);
await client.query('commit');
} catch (error) {
await client.query('rollback');
throw error;
} finally {
client.release();
}
const result: ProviderBillWorkerResult = {
processed: jobs.length,
completed: 0,
failed: 0,
skipped: 0,
};
for (const job of jobs) {
const status = await processProviderBillJob(job);
if (status === 'completed') result.completed += 1;
else if (status === 'failed') result.failed += 1;
else result.skipped += 1;
}
return result;
}

View File

@@ -281,6 +281,8 @@ body: { "code": "<qq oauth code>", "redirectUri": "https://h5.example.com/auth/q
- `GET /api/commerce/reconciliation/batches`:查询对账批次。
- `GET /api/commerce/reconciliation/items`:查询逐行结果。
- `GET /api/commerce/reconciliation/anomalies`:查询金额不一致、状态不一致、供应商有本地无、本地有供应商无、重复行等异常。
- `POST /api/commerce/reconciliation/provider-bills/request`:创建微信/支付宝官方账单下载任务body 为 `{ provider, billDate, billType }`
- `GET /api/commerce/reconciliation/provider-bills/jobs`查看官方账单下载任务状态、下载域名、hash、行数和生成的对账批次 ID。
- `POST /api/commerce/reconciliation/issues/create`:从异常明细创建差错工单,重复创建同一未关闭明细会返回现有工单。
- `GET /api/commerce/reconciliation/issues`:按状态、严重级别、负责人、批次、订单号筛选差错工单。
- `POST /api/commerce/reconciliation/issues/status`:执行 `start/assign/resolve/ignore/escalate/reopen` 状态流转。
@@ -290,8 +292,11 @@ body: { "code": "<qq oauth code>", "redirectUri": "https://h5.example.com/auth/q
- `tenant:reconciliation:read`:查看/预览对账。
- `tenant:reconciliation:write`:导入对账批次、创建/处理差错工单。
- `tenant:reconciliation:download`:创建微信/支付宝官方账单下载任务。
对账和差错工单只生成差异台账、处理记录和审计,不自动修改订单、支付、退款和权益。`resolve/ignore` 只是财务审核结论,例如 `manual_adjustment``provider_confirmed``false_positive`,最终落账仍必须走退款状态机、支付补偿、手工支付确认或后续专门的人工调整命令。真实生产中,微信/支付宝官方账单下载 adapter 应复用同一套 `commerce_reconciliation_batches/items`,下载后的 CSV/JSON 先规范化为 `rows`,再调用同一套匹配逻辑。官方账单自动下载、人工调整凭证附件、财务复核报表和异常订单运营台仍需要继续补
官方账单下载由 `apps/worker --job provider-bills` 执行。API 只创建 `commerce_bill_download_jobs`,不会在前端返回供应商 `download_url`、商户私钥、微信 API v3 key 或支付宝应用私钥。worker 使用租户 `tenant_payment_accounts.config_public.secretRef` 找到 `app_private.tenant_secrets(secret_scope='payment')`,在后端签名申请下载 URL校验微信返回的 `hash_type/hash_value`,解析 JSON/CSV/ZIP 账单后复用 `importReconciliationBatch` 写入 `commerce_reconciliation_batches/items``source='provider_download'`
对账和差错工单只生成差异台账、处理记录和审计,不自动修改订单、支付、退款和权益。`resolve/ignore` 只是财务审核结论,例如 `manual_adjustment``provider_confirmed``false_positive`,最终落账仍必须走退款状态机、支付补偿、手工支付确认或后续专门的人工调整命令。人工调整凭证附件、财务复核报表、异常订单运营台和真实生产账单格式抽样验收仍需要继续补。
B 端合作商年费、服务费、服务器资源费不走学生端 `orders`,而是走平台账务:

View File

@@ -121,7 +121,8 @@
| 支付/退款补偿 worker | 可联调 | `apps/worker --job commerce` 查询微信/支付宝订单和处理中退款,补偿漏通知支付、补发权益、确认退款、全额退款撤销权益;`npm run test:worker:commerce` 覆盖幂等和密钥不泄露 |
| 资金流水对账 | 可联调 | `commerce_reconciliation_batches/items` + `/api/commerce/reconciliation/preview/import/batches/items/anomalies`;租户后台需 `tenant:reconciliation:read/write`,支持支付/退款账单行手工或 API 导入、来源 hash、批次统计、逐行匹配、金额/状态差异、本地缺失、供应商缺失、重复行、无效行和审计;对账只生成差异,不自动改订单/权益 |
| 对账差错工单 | 可联调 | `commerce_reconciliation_issues/events` + `/api/commerce/reconciliation/issues*`;异常明细可创建工单,支持分配、开始处理、升级、解决、忽略、重开、事件轨迹和审计;处理结论只作为财务审核记录,不直接修改订单、支付、退款或权益 |
| 官方账单下载和异常运营台 | 待补齐 | 后续补微信/支付宝账单自动下载、人工调整凭证附件/复核、异常订单运营台和财务报表 |
| 官方账单下载 | 可联调 | `commerce_bill_download_jobs` + `/api/commerce/reconciliation/provider-bills/request/jobs` + `apps/worker --job provider-bills`;租户后台需 `tenant:reconciliation:download` 创建任务worker 后端使用租户商户密钥申请微信/支付宝官方账单下载 URL、校验 hash、解析 JSON/CSV/ZIP 账单并复用同一套 `provider_download` 对账导入响应只暴露任务状态、下载域名、hash 和对账批次 ID不暴露下载 URL 或密钥 |
| 异常订单运营台和财务报表 | 待补齐 | 后续补人工调整凭证附件/复核、异常订单运营台、财务复核报表和真实生产账单格式抽样验收 |
## 租户后台与平台后台

View File

@@ -29,7 +29,7 @@
| 分数线 | 可联调 | 院校、专业、动态字段、记录、年份、趋势、后台维护、JSON/CSV/Excel 导入 | 复杂筛选、AI 择校上下文 |
| 视频解析 | 可联调 | 单题视频、批量查询、后台视频绑定、JSON/CSV/Excel 导入、会员播放权限、播放次数扣减、签名 URL、播放日志和动态水印上下文 | 深度防盗链、转码级水印、播放统计 |
| 资料下载 | 部分完成 | 资源台账、SVIP 权限校验、`local_dev`/阿里云 OSS/腾讯 COS/Supabase Storage 上传下载签名、上传确认、PDF/图片预览签名、访问审计、动态水印上下文、assets worker 复检、内置安全扫描、外部 HTTP scanner 接入层、题库导出 PDF/Word/每日一练 ZIP 可生成可信 `content_assets` 并走签名下载/预览 | CDN 防盗链、真实 AV/内容安全服务联调、转码级水印、资料前端操作体验 |
| 会员与订单 | 可联调 | 下单、订单详情/状态轮询、优惠券领取/抵扣、零元订单自动开通、手工确认权限保护、激活码预检查/兑换、微信支付、支付宝、微信/支付宝发起退款、微信/支付宝退款查询确认、微信/支付宝退款通知 webhook、支付/退款补偿 worker、权益发放、资金对账批次/明细/异常查询 API、对账差错工单和事件轨迹 | 微信/支付宝官方账单自动下载、财务复核报表异常订单运营台 |
| 会员与订单 | 可联调 | 下单、订单详情/状态轮询、优惠券领取/抵扣、零元订单自动开通、手工确认权限保护、激活码预检查/兑换、微信支付、支付宝、微信/支付宝发起退款、微信/支付宝退款查询确认、微信/支付宝退款通知 webhook、支付/退款补偿 worker、权益发放、资金对账批次/明细/异常查询 API、微信/支付宝官方账单下载 worker、对账差错工单和事件轨迹 | 财务复核报表异常订单运营台和真实生产账单格式验收 |
| 登录认证 | 可联调 | 短信 mock、阿里云/腾讯云短信 adapter、迁移期 session、Supabase Auth JWT、微信小程序登录、微信网页登录、QQ 登录、手机号绑定/换绑、OAuth 配置表 | 真实生产账号和回调域名联调 |
| 销售/代理/CRM | 可联调 | 邀请码、首绑保护、团队关系、销售统计、CRM 配置/队列、钉钉/飞书/企微 worker、分佣规则、成员分佣比例、订单/激活码归因、结算生成、审核和打款状态Taro 营销中心已接第一版操作台 | 小程序码真实生成、CRM 分配策略、结算导出、真实打款、凭证、财务复核和销售转化看板 |
| 内容导入 | 可联调 | 题目、单词、知识手册、分数线、视频 JSON/CSV/Excel preview/import、issue、job/detail、审计、幂等、`executionMode=async`、imports worker、导入后复检、模板下载、字段映射 API、字段映射覆盖白名单校验、PocketBase JSON dry-run 报告Taro 租户内容页已接上传/粘贴预览、模板文件下载、字段别名编辑、同步/异步执行、异步轮询和复检详情第一版 | 真实数据 dry-run 执行验收、抽样校验和导入性能压测 |
@@ -83,7 +83,7 @@
### P1商用收费和运营能力
- 资金对账已支持手工/API 账单导入比对、异常查询和差错工单处理;继续补微信/支付宝官方账单下载、财务复核报表和异常订单运营台。
- 资金对账已支持手工/API 账单导入比对、微信/支付宝官方账单下载任务、异常查询和差错工单处理;继续补真实生产账单格式验收、财务复核报表和异常订单运营台。
- XPay 或其它实际支付网关 adapter。
- 阿里云/腾讯云短信、微信小程序登录、微信网页登录、QQ 登录真实账号联调。
- 公共题库/地区题库自动同步 worker 已具备单批执行能力,租户后台已有同步通知、单条/批量冲突采纳平台或保留本地操作;继续补生产定时调度、失败告警,以及租户按 SaaS 套餐购买地区、科目和题库范围的更细计费策略。

View File

@@ -269,7 +269,7 @@ GET /api/tenant-admin/audit-logs
1. 完善内容导入和文件上传:字段映射 UI、真实数据 dry-run、PDF 预览渲染、防盗链、真实 AV/内容安全服务联调和转码/CDN 级视频水印。
2. 完成真实短信 provider 联调:阿里云/腾讯云,密钥放 `app_private.tenant_secrets` 或生产 Vault。
3. 完成真实 OAuth provider 联调微信网页、微信小程序、QQ确认回调域名、开放平台账号和旧 PocketBase 身份映射策略。
4.微信/支付宝官方账单自动下载、异常订单运营台和优惠券核销报表;支付/退款补偿、退款查询确认、退款通知、资金对账导入比对和差错工单主链路已完成。
4. 补异常订单运营台、真实生产账单格式验收和优惠券核销报表;支付/退款补偿、退款查询确认、退款通知、微信/支付宝官方账单下载、资金对账导入比对和差错工单主链路已完成。
5. 扩展 `apps/worker`日报统计、CRM 死信告警、公共题库同步失败告警和更完整冲突处理运营台;公共题库同步 worker 和同步通知已具备基础闭环。
6. 开始 Taro scaffold`supabaseApi` 抽到跨端包或适配层。

View File

@@ -30,7 +30,7 @@
| CRM 系统 | 可联调 | CRM 配置、密钥私密存储、客资入队、队列查询、generic/钉钉/飞书/企微 worker、签名、重试和日志 | 定向/轮询分配、富卡片模板、失败告警、死信运营台 |
| 数据看板 | 可联调 | 租户 dashboard 聚合接口收益、注册、学习、内容、激活码、反馈、趋势、24h 活跃、套餐销量和运营动态 | 预聚合 worker、缓存、慢 SQL 监控和销售转化看板 |
| 登录认证 | 可联调 | 短信 mock、阿里云/腾讯云短信 adapter、迁移期 session、Supabase Auth JWT、微信小程序登录、微信网页登录、QQ 登录、手机号绑定/换绑、OAuth 配置表 | 真实生产账号和回调域名联调 |
| 支付 | 可联调 | 订单、支付记录、手动确认权限保护、权益发放、租户商户配置、微信支付 JSAPI、支付宝 WAP/H5、webhook 幂等、退款状态机、退款通知、补偿 worker、资金对账导入比对、异常查询、差错工单和事件轨迹 | 官方账单自动下载、异常订单运营台、服务商/平台代收模式 |
| 支付 | 可联调 | 订单、支付记录、手动确认权限保护、权益发放、租户商户配置、微信支付 JSAPI、支付宝 WAP/H5、webhook 幂等、退款状态机、退款通知、补偿 worker、微信/支付宝官方账单下载 worker、资金对账导入比对、异常查询、差错工单和事件轨迹 | 异常订单运营台、真实生产账单格式验收、服务商/平台代收模式 |
| AI 择校推荐 | 部分完成 | SVIP 门禁、学生输入 schema、地区/分数线上下文、`local_rules` 稳定 JSON 报告、报告台账/列表/详情、Taro 学生端基础页 | 真实 AI provider、prompt 版本管理后台、PDF 报告生成、人工复核和运营配置 |
| Taro 跨端 | 未开始 | 旧 Web 新 API 适配开始 | `apps/taro`、共享 API client、H5/小程序统一构建 |

View File

@@ -27,7 +27,7 @@
| 刷题题库 | 已建题库、题目、题目版本、内容入口、任意深度分类树、考试意向标记、题目集合、练习蓝图、导入任务台账、导出任务台账、公共题库授权/采纳表、租户内容通知表 | 已支持核心映射JSON/CSV/Excel 导入可落到新入口/节点/集合,阅读理解/案例分析子题沿用 `subQuestions/sub_questions` | 题目列表、内容入口、分类树、集合题目、顺序/随机/全真模拟 session、答题提交、复合题 `subAnswers` 判分和报告明细、租户后台题目录入/更新、JSON/CSV/Excel 预览/导入、JSON/试卷 payload 导出、PDF/Word 异步导出 worker、每日一练九宫格 metadata、PDF/Word 运营版式、ZIP 图片素材包、异步导入 worker、平台公共题库授权、租户采纳快照、手动同步、自动同步 worker、同步通知、冲突查询和单条/批量冲突处理已实现 | 核心 API 集成测试含导航、组卷、复合题后台录入/练习/判分/报告、导入、导出权限/脱敏、每日一练导出 metadata、异步 PDF/Word/每日一练 ZIP job 创建、exports worker、公共题库授权、采纳后组卷、同步新增题、通知隔离/已读/自动 resolved、租户自改冲突保护、单条/批量冲突处理和 worker 自动同步断言 | 新题库导航和组卷基础闭环可跑,阅读理解/案例分析多小题第一版可联调,公共题库采纳/手动/自动同步、同步通知、冲突查询/处理、导入后复检、模板下载、字段映射 API、JSON/PDF/Word/每日一练 ZIP 基础导出可联调;公共题库生产调度/失败告警、更精细导出模板和更完整运营消息仍需补齐 |
| 错题本 | 已建 `wrong_questions` | 已支持旧错题归一化 | 错题列表、答题自动入错题、移出错题已实现 | 仅烟测 | 基础功能已实现,复习计划和统计未完成 |
| 收藏夹 | 已建 `favorite_questions` | 已支持旧收藏归一化 | 收藏/取消收藏、收藏列表已实现 | 仅烟测 | 基础功能已实现 |
| 用户订阅/题库会员/SVIP | 已建 `orders``payments``entitlements``svip_plans`、激活码 | 已映射旧 SVIP/会员权益 | 下单、订单详情/状态轮询、手工支付确认权限保护、微信/支付宝支付、微信/支付宝发起退款、微信/支付宝退款查询确认、微信/支付宝退款通知 webhook、激活码预检查/兑换、优惠券抵扣、零元订单自动开通、权益查询、支付/退款补偿、资金对账和差错工单已实现 | API 集成测试 | 商城主链路可联调,官方账单自动下载、异常订单运营台和生产账号联调待补 |
| 用户订阅/题库会员/SVIP | 已建 `orders``payments``entitlements``svip_plans`、激活码 | 已映射旧 SVIP/会员权益 | 下单、订单详情/状态轮询、手工支付确认权限保护、微信/支付宝支付、微信/支付宝发起退款、微信/支付宝退款查询确认、微信/支付宝退款通知 webhook、激活码预检查/兑换、优惠券抵扣、零元订单自动开通、权益查询、支付/退款补偿、微信/支付宝官方账单下载、资金对账和差错工单已实现 | API 集成测试、commerce worker 集成测试 | 商城主链路可联调,异常订单运营台、真实生产账单格式验收和生产账号联调待补 |
| 背单词 | 已建单词单元、单词、进度、收藏表,并可绑定 `content_entries/content_nodes` | 已支持内容和部分用户状态映射 | 单元/单词只读、进度、收藏、统计、每日复习计划、租户后台单词维护 API、旧模板/新模板 JSON 预览导入、排行榜已实现 | 核心 API 集成测试含导入和排行榜断言 | 学生端学习状态、后台维护、批量 JSON 导入和基础排行榜已实现,更细复习参数和后台统计待完善 |
| 知识手册 | 已建手册科目、章节、条目,并可绑定 `content_entries/content_nodes` | 已支持内容导入 | 只读 API、租户后台手册科目/章节/条目维护 API、嵌套 JSON 预览导入已实现 | 核心 API 集成测试含导入断言 | 学生端阅读、后台维护和批量 JSON 导入基础可用,富文本资源/版本管理待补 |
| 分数线 | 已建院校、专业、字段、记录表 | 已支持导入映射 | 字段、院校、专业、记录、趋势、年份、租户后台维护 API、JSON 预览导入已实现 | 核心 API 集成测试含导入断言 | 查询、后台维护和批量 JSON 导入基础闭环已实现,复杂动态筛选和 AI 择校上下文待补 |
@@ -300,4 +300,4 @@ platform-admin:
3. 补学习统计增强:排行榜防刷/预聚合、断点续练、专项练习策略和更细题型分析。
4. 补视频商用控制:深度防盗链、转码级水印和播放统计。
5. 补 AI 择校推荐报告、排行榜防刷/预聚合、勋章自动发放。
6. 接真实短信/OAuth 生产账号、微信/支付宝官方账单下载和异常订单运营台,并继续推进 Taro scaffold。
6. 接真实短信/OAuth 生产账号、真实生产账单格式验收和异常订单运营台,并继续推进 Taro scaffold。

View File

@@ -30,7 +30,7 @@
| 背单词 | `VocabularyPage.tsx``VocabularyQuiz.tsx` | 部分覆盖 | 单词列表、进度、收藏、统计、每日计划和后端复习调度已覆盖;后续补收藏练习体验、发音/音频策略、排行榜和更精细的间隔算法参数 |
| 知识手册 | `Handbook*.tsx` | 已覆盖 | 前端需做好 Markdown/公式/图片渲染和搜索体验 |
| 分数线 | `ScorelinePage.tsx` | 已覆盖 | 动态字段/趋势、后台维护和 JSON 批量导入已有;后续补复杂筛选优化和 AI 择校数据上下文 |
| 商城/SVIP | `Store.tsx``SvipModal.tsx` | 部分覆盖 | 套餐、订单、订单详情/状态轮询、权益、激活码预检查/兑换、优惠券领取/下单抵扣、微信支付/支付宝 provider 主链路、内部退款状态机、微信/支付宝发起退款、退款查询确认、退款通知 webhook、支付/退款补偿 worker、全额退款权益撤销、资金对账手工/API 导入比对、异常查询、差错工单和事件轨迹已有;缺微信/支付宝官方账单自动下载、异常订单运营台和前端收银台/售后体验 |
| 商城/SVIP | `Store.tsx``SvipModal.tsx` | 部分覆盖 | 套餐、订单、订单详情/状态轮询、权益、激活码预检查/兑换、优惠券领取/下单抵扣、微信支付/支付宝 provider 主链路、内部退款状态机、微信/支付宝发起退款、退款查询确认、退款通知 webhook、支付/退款补偿 worker、全额退款权益撤销、资金对账手工/API 导入比对、微信/支付宝官方账单下载 worker、异常查询、差错工单和事件轨迹已有缺异常订单运营台、人工调整凭证复核报表和前端收银台/售后体验 |
| 个人中心 | `Profile.tsx` | 部分覆盖 | 基本资料、手机号绑定/换绑、权益、订单统计、练习历史、学习统计、签到积分、考试倒计时、趋势和勋章展示 API 已有;缺学习报告可视化 |
| 资料下载 | `QuestionExporterPublishModal.tsx` 等 | 部分覆盖 | 资源台账、上传确认、签名下载、PDF/图片预览、动态水印上下文、worker 复检、内置安全扫描和外部 HTTP scanner 接入层已有;缺前端水印渲染、深度防盗链、真实 AV/内容安全服务联调和生命周期策略 |
| AI 择校推荐 | 业务规划新增 | 部分覆盖 | 已有 SVIP 门禁、学生输入 schema、地区/分数线上下文、稳定 JSON 输出、报告台账、审计和 Taro 学生端基础页;真实 AI provider、prompt 版本管理后台、报告 PDF 渲染和更细推荐算法待补 |

View File

@@ -28,6 +28,7 @@
- 租户后台数据看板已完成首版聚合 API`GET /api/tenant-admin/dashboard`,支持租户/地区维度的收益、注册、学习、内容、激活码、反馈、趋势、24h 活跃、套餐销量和运营动态,前端可直接联调。
- 支付/退款补偿 worker 已完成:`apps/worker --job commerce` 可查询微信/支付宝支付和处理中退款,补偿漏通知订单,支付成功幂等开通权益,退款成功幂等更新退款/订单/支付并在全额退款时撤销订单权益。
- 资金对账和差错工单闭环已完成:`commerce_reconciliation_batches/items``/api/commerce/reconciliation/*` 支持手工/API 导入供应商账单行、预览差异、生成批次统计、查询异常、租户隔离、权限点 `tenant:reconciliation:read/write` 和审计日志;`commerce_reconciliation_issues/events` 支持异常明细创建工单、分配、开始处理、升级、解决、忽略、重开和事件留痕,且不直接修改订单/支付/退款/权益。
- 微信/支付宝官方账单下载地基已完成:`commerce_bill_download_jobs``POST /api/commerce/reconciliation/provider-bills/request``GET /api/commerce/reconciliation/provider-bills/jobs``apps/worker --job provider-bills` 已接入worker 负责后端签名申请下载 URL、hash 校验、JSON/CSV/ZIP 账单解析、复用 `provider_download` 对账导入、任务状态回写和密钥脱敏。
- 内容资源复检与安全扫描 worker 已完成:`apps/worker --job assets` 可复检 `content_assets` 中的托管对象元数据,并执行内置 `metadata_rules` 和可选外部 HTTP scanner正常资源写回复检/扫描证据,异常资源自动置为 `failed/skipped + draft``security_scan_status=failed`,外部 scanner 不可用默认 fail-closed并写入审计、扫描事件和安全标记。
- 题库导出 worker 已完成:`apps/worker --job exports` 可抢占 `pdf/docx/daily_practice_zip` 导出任务,渲染 PDF/Word、水印或每日一练图片素材包写入对象存储或本地开发存储创建 `content_assets` 并回填 `assetId/hash/size``daily_practice` 已支持每日一练九宫格 metadata、PDF/Word 基础版式、9 张 PNG/SVG 卡片和拼图 ZIP。
- 租户后台媒体运营报表已完成:`/api/tenant-content/media-analytics/summary``asset-events``video-events` 可按 7/30/90 天、资源、视频、用户和水印 traceId 查询资料访问、视频播放、拒绝访问、Top 资源/视频和日趋势;仅开放给 owner/admin/operator 或 `content:analytics:read` 权限角色,前端不会拿到签名 URL 或播放 token。
@@ -75,7 +76,7 @@
- 已完成微信支付 JSAPI、支付宝 WAP/H5 的创建支付参数和 webhook 幂等开通权益。
- 已完成内部退款状态机、退款申请/审核/处理接口、微信/支付宝发起退款、微信/支付宝退款查询确认、微信/支付宝退款通知 webhook、部分/全额退款状态、全额退款权益撤销和审计事件。
- 已完成支付/退款补偿 worker可兜底供应商漏通知、处理中退款和重复执行幂等。
- 已完成资金对账手工/API 导入比对、批次/明细/异常查询、差错工单状态流和审计;继续补微信/支付宝官方账单自动下载、人工调整凭证附件、财务复核报表和异常订单运营台。
- 已完成资金对账手工/API 导入比对、微信/支付宝官方账单下载任务、批次/明细/异常查询、差错工单状态流和审计;继续补真实生产账单格式抽样验收、人工调整凭证附件、财务复核报表和异常订单运营台。
- 租户自有商户收款和平台代收/服务商模式。
2. 国内登录和短信
@@ -114,7 +115,7 @@
8. 订单和营销体验
- 已完成订单详情、订单状态轮询、激活码预检查、优惠券前台领取、下单抵扣计算和内部退款状态机。
- 已完成支付/退款补偿 worker、资金对账导入比对和差错工单继续补异常订单运营台、优惠券核销报表和复杂活动规则。
- 已完成支付/退款补偿 worker、官方账单下载 worker、资金对账导入比对和差错工单;继续补异常订单运营台、优惠券核销报表和复杂活动规则。
9. 积分和反馈增强
- 已完成每日签到、积分流水、反馈提交、租户后台处理、奖励积分幂等。
@@ -228,5 +229,5 @@
3. 补平台后台增强:租户详情/编辑、平台审计报表、自动计费、账单批量操作和更细平台权限点。
4. 云服务器部署 Supabase/PostgreSQL 和 API配置对象存储生产环境变量`check:refactor` 的远程等价测试。
5. 导出现有 PocketBase 数据,做完整 dry-run 迁移。
6. 并行补真实登录、微信/支付宝官方账单自动下载、异常订单运营台、对象存储真实 AV/内容安全服务联调、转码/CDN 级水印/生命周期、题库导出模板精排/操作台、公共题库生产定时调度和失败告警。
6. 并行补真实登录、真实生产账单格式验收、异常订单运营台、对象存储真实 AV/内容安全服务联调、转码/CDN 级水印/生命周期、题库导出模板精排/操作台、公共题库生产定时调度和失败告警。
7. 前后端联调通过后,再做支付、权限、数据导入、资料下载、视频播放的商用验收。

View File

@@ -1853,7 +1853,7 @@ approved/processing -> failed
- 退款通知地址由支付账户或 `submit_provider_refund.providerNotifyUrl` 配置,后端公开接收路径为 `POST /api/commerce/refunds/notify/wechat_pay?tenantId=<tenantId>``POST /api/commerce/refunds/notify/alipay?tenantId=<tenantId>`。这是支付平台回调地址Taro 前端不要主动调用。
- 退款通知只会推进已经审核/处理中的退款申请;未审核的 `requested` 退款不能被外部通知直接落账。
- 已经 `succeeded` 的退款不能再次查询或再次标记成功,避免订单退款金额重复累加。前端应按接口返回状态展示,不要假设点击后立即到账。
- 自动补偿 worker 已接入:支付漏通知和处理中退款会由后端定时查询供应商并幂等落账。资金对账已支持租户后台手工/API 导入供应商账单、查询差异和差错工单处理;官方账单自动下载和异常订单运营台后续继续补。生产联调时仍需保留人工确认/失败登记入口。
- 自动补偿 worker 已接入:支付漏通知和处理中退款会由后端定时查询供应商并幂等落账。资金对账已支持租户后台手工/API 导入供应商账单、微信/支付宝官方账单下载任务、查询差异和差错工单处理;异常订单运营台和人工调整凭证复核后续继续补。生产联调时仍需保留人工确认/失败登记入口。
### 租户后台资金对账
@@ -1955,6 +1955,66 @@ GET /api/commerce/reconciliation/issues/events?issueId=<issueId>
权限tenant:reconciliation:read
```
官方账单下载:
```text
POST /api/commerce/reconciliation/provider-bills/request
权限tenant:reconciliation:download
body: {
"provider": "wechat_pay | alipay",
"billDate": "2026-06-29",
"billType": "payment | refund | combined",
"metadata": {
"remark": "财务手动申请"
}
}
```
返回:
```json
{
"item": {
"id": "<jobId>",
"provider": "wechat_pay",
"billDate": "2026-06-29",
"billType": "payment",
"status": "queued",
"sourceName": "provider-bill:wechat_pay:2026-06-29:payment",
"sourceHash": null,
"rowCount": 0,
"downloadUrlHost": null,
"reconciliationBatchId": null
},
"idempotent": false
}
```
轮询任务:
```text
GET /api/commerce/reconciliation/provider-bills/jobs?provider=wechat_pay&billDate=2026-06-29
权限tenant:reconciliation:read
```
任务状态:
```text
queued 已排队,等待 provider-bills worker
running worker 正在申请和下载官方账单
completed 已下载、校验、导入对账批次
failed 下载、hash 校验、解析或导入失败
cancelled 已取消
```
前端处理规则:
- 前端只创建任务和轮询状态,不直接请求微信/支付宝账单 URL。
- 后端响应只会返回 `downloadUrlHost`,不会返回完整下载 URL、商户私钥、API v3 key、支付宝应用私钥。
- `completed` 后用 `reconciliationBatchId` 跳转到对账批次明细。
- `failed` 时展示 `errorCode/errorMessage`,让财务重新发起或联系技术处理。
- 官方账单下载由服务器定时运行 `apps/worker --job provider-bills`,前端不要自行触发供应商接口。
工单状态:
```text
@@ -1995,7 +2055,7 @@ ignored 无效行或不符合本次 billType
- `sourceHash` 可作为同一文件内容的识别线索,但当前接口不会阻止重复导入;前端应展示最近同名/同 hash 批次提醒。
- 金额统一是分,前端不要传元。
- 对账差异和差错工单只是运营判断依据,`resolve/ignore` 不会落账。最终订单修正必须走退款、补偿、人工确认或后续专门的人工调整接口。
- 当前后端支持 JSON 行导入;CSV/Excel 可以先由前端或后续后端 parser 转成上述 `rows`。微信/支付宝官方账单自动下载仍是后续后端任务
- 当前后端支持 JSON 行手工导入;官方账单下载 worker 支持微信/支付宝账单 JSON/CSV/ZIP 解析,并复用同一套对账导入逻辑。真实生产接入时仍要用真实账单文件抽样验收字段映射
### 激活码预检查与兑换

1
package-lock.json generated
View File

@@ -88,6 +88,7 @@
"@supabase/storage-js": "^2.108.2",
"ali-oss": "^6.23.0",
"docx": "^9.7.1",
"iconv-lite": "^0.6.3",
"jszip": "^3.10.1",
"pdfkit": "^0.19.1",
"pg": "^8.16.3"

View File

@@ -2484,6 +2484,63 @@ async function testCommerce() {
});
assert.equal(crossTenantReconciliationDenied.code, 'TENANT_ADMIN_REQUIRED', 'reconciliation items must be tenant isolated');
const studentProviderBillRequestDenied = await request('/api/commerce/reconciliation/provider-bills/request', {
method: 'POST',
body: {
provider: 'wechat_pay',
billDate,
billType: 'payment',
},
expectStatus: 403,
});
assert.equal(studentProviderBillRequestDenied.code, 'TENANT_ADMIN_REQUIRED', 'students must not request official provider bill downloads');
const providerBillJob = await request('/api/commerce/reconciliation/provider-bills/request', {
userId: TENANT_ADMIN_USER_ID,
method: 'POST',
body: {
provider: 'wechat_pay',
billDate,
billType: 'payment',
metadata: { source: 'api-integration-test' },
},
});
assert.ok(providerBillJob.item?.id, 'tenant admin should request official provider bill download job');
assert.equal(providerBillJob.item?.status, 'queued', 'new provider bill job should be queued');
assert.equal(providerBillJob.item?.provider, 'wechat_pay', 'provider bill job should keep provider');
assert.ok(!JSON.stringify(providerBillJob).includes(paymentFixture.wechatApiV3Key), 'provider bill job response must not leak payment secrets');
assert.ok(!JSON.stringify(providerBillJob).includes('PRIVATE KEY'), 'provider bill job response must not leak private keys');
const providerBillJobAgain = await request('/api/commerce/reconciliation/provider-bills/request', {
userId: TENANT_ADMIN_USER_ID,
method: 'POST',
body: {
provider: 'wechat_pay',
billDate,
billType: 'payment',
},
});
assert.equal(providerBillJobAgain.item?.id, providerBillJob.item.id, 'provider bill job request should be idempotent by provider/date/type');
assert.equal(providerBillJobAgain.idempotent, true, 'provider bill duplicate request should return idempotent=true');
const providerBillJobs = await request('/api/commerce/reconciliation/provider-bills/jobs', {
userId: TENANT_ADMIN_USER_ID,
query: { provider: 'wechat_pay', billDate },
});
assert.ok(
providerBillJobs.items?.some(item => item.id === providerBillJob.item.id),
'provider bill jobs endpoint should list tenant jobs',
);
assert.ok(!JSON.stringify(providerBillJobs).includes(paymentFixture.wechatApiV3Key), 'provider bill jobs list must not leak payment secrets');
const crossTenantProviderBillJobsDenied = await request('/api/commerce/reconciliation/provider-bills/jobs', {
tenantId: PARTNER_TENANT_ID,
userId: TENANT_ADMIN_USER_ID,
query: { provider: 'wechat_pay', billDate },
expectStatus: 403,
});
assert.equal(crossTenantProviderBillJobsDenied.code, 'TENANT_ADMIN_REQUIRED', 'provider bill jobs must be tenant isolated');
const fakeWechatPay = await startFakeWechatPayServer();
const wechatAccount = await request('/api/tenant-admin/payment-accounts', {
userId: TENANT_ADMIN_USER_ID,

View File

@@ -17,18 +17,29 @@ const ids = {
refundPayment: '10000000-0000-0000-0000-000000000704',
refundRequest: '10000000-0000-0000-0000-000000000705',
refundEntitlement: '10000000-0000-0000-0000-000000000706',
providerBillJobWechat: '10000000-0000-0000-0000-000000000707',
providerBillJobAlipay: '10000000-0000-0000-0000-000000000708',
providerBillOrderWechat: '10000000-0000-0000-0000-000000000709',
providerBillPaymentWechat: '10000000-0000-0000-0000-000000000710',
providerBillOrderAlipay: '10000000-0000-0000-0000-000000000711',
providerBillPaymentAlipay: '10000000-0000-0000-0000-000000000712',
};
const orderNos = {
payment: 'CW-WX-PAY-001',
refund: 'CW-WX-REFUND-001',
providerBillWechat: 'CW-WX-BILL-001',
providerBillAlipay: 'CW-ALI-BILL-001',
};
const refundNo = 'RF-CW-WX-001';
const providerBillDate = '2026-06-29';
const paymentFixture = (() => {
const wechatMerchant = crypto.generateKeyPairSync('rsa', { modulusLength: 2048 });
const alipayApp = crypto.generateKeyPairSync('rsa', { modulusLength: 2048 });
return {
wechatMerchantPrivateKey: wechatMerchant.privateKey.export({ type: 'pkcs8', format: 'pem' }).toString(),
alipayAppPrivateKey: alipayApp.privateKey.export({ type: 'pkcs8', format: 'pem' }).toString(),
wechatApiV3Key: '12345678901234567890123456789012',
};
})();
@@ -48,6 +59,25 @@ async function startFakeWechatPayServer() {
const port = await getFreePort();
const baseUrl = `http://127.0.0.1:${port}`;
const requests = [];
const billRows = [
{
transactionType: 'payment',
orderNo: orderNos.providerBillWechat,
providerTradeNo: `wx-bill-trade-${orderNos.providerBillWechat}`,
amountCents: 990,
providerStatus: 'SUCCESS',
paidAt: '2026-06-29T10:00:00+08:00',
},
{
transactionType: 'payment',
orderNo: 'CW-WX-PROVIDER-ONLY',
providerTradeNo: 'wx-provider-only-trade',
amountCents: 1888,
providerStatus: 'SUCCESS',
},
];
const billBody = Buffer.from(JSON.stringify({ rows: billRows }), 'utf8');
const billHash = crypto.createHash('sha256').update(billBody).digest('hex');
const server = http.createServer((req, res) => {
const url = new URL(req.url || '/', baseUrl);
let raw = '';
@@ -85,6 +115,22 @@ async function startFakeWechatPayServer() {
return;
}
if (req.method === 'GET' && url.pathname === '/v3/bill/tradebill') {
res.writeHead(200, { 'content-type': 'application/json' });
res.end(JSON.stringify({
hash_type: 'SHA256',
hash_value: billHash,
download_url: `${baseUrl}/download/wechat-tradebill.json`,
}));
return;
}
if (req.method === 'GET' && url.pathname === '/download/wechat-tradebill.json') {
res.writeHead(200, { 'content-type': 'application/json' });
res.end(billBody);
return;
}
res.writeHead(404, { 'content-type': 'application/json' });
res.end(JSON.stringify({ code: 'NOT_FOUND' }));
});
@@ -96,6 +142,61 @@ async function startFakeWechatPayServer() {
return {
paymentQueryEndpoint: `${baseUrl}/v3/pay/transactions/out-trade-no`,
refundEndpoint: `${baseUrl}/v3/refund/domestic/refunds`,
tradeBillEndpoint: `${baseUrl}/v3/bill/tradebill`,
requests,
close: () => new Promise(resolve => server.close(resolve)),
};
}
async function startFakeAlipayServer() {
const port = await getFreePort();
const baseUrl = `http://127.0.0.1:${port}`;
const requests = [];
const billRows = [
{
transactionType: 'payment',
orderNo: orderNos.providerBillAlipay,
providerTradeNo: `ali-bill-trade-${orderNos.providerBillAlipay}`,
amountCents: 1288,
providerStatus: 'TRADE_SUCCESS',
paidAt: '2026-06-29 11:00:00',
},
];
const billBody = Buffer.from(JSON.stringify({ rows: billRows }), 'utf8');
const server = http.createServer((req, res) => {
const url = new URL(req.url || '/', baseUrl);
let raw = '';
req.on('data', chunk => {
raw += chunk.toString();
});
req.on('end', () => {
requests.push({ method: req.method, pathname: url.pathname, search: url.search, body: raw });
if (req.method === 'POST' && url.pathname === '/gateway.do') {
res.writeHead(200, { 'content-type': 'application/json' });
res.end(JSON.stringify({
alipay_data_dataservice_bill_downloadurl_query_response: {
code: '10000',
msg: 'Success',
bill_download_url: `${baseUrl}/download/alipay-tradebill.json`,
},
}));
return;
}
if (req.method === 'GET' && url.pathname === '/download/alipay-tradebill.json') {
res.writeHead(200, { 'content-type': 'application/json' });
res.end(billBody);
return;
}
res.writeHead(404, { 'content-type': 'application/json' });
res.end(JSON.stringify({ code: 'NOT_FOUND' }));
});
});
await new Promise((resolve, reject) => {
server.once('error', reject);
server.listen(port, '127.0.0.1', resolve);
});
return {
endpoint: `${baseUrl}/gateway.do`,
requests,
close: () => new Promise(resolve => server.close(resolve)),
};
@@ -129,7 +230,55 @@ async function runWorkerOnce() {
return output;
}
async function runProviderBillsWorkerOnce() {
const child = spawn(process.execPath, ['apps/worker/dist/apps/worker/src/index.js', '--once', '--job', 'provider-bills'], {
cwd: process.cwd(),
env: {
...process.env,
DATABASE_URL: databaseUrl,
WORKER_PROVIDER_BILL_BATCH_SIZE: '10',
WORKER_PROVIDER_BILL_ID: 'provider-bills-integration-test',
WORKER_COMMERCE_REQUEST_TIMEOUT_MS: '5000',
},
stdio: ['ignore', 'pipe', 'pipe'],
windowsHide: true,
});
let output = '';
child.stdout.on('data', chunk => {
output += chunk.toString();
});
child.stderr.on('data', chunk => {
output += chunk.toString();
});
const code = await new Promise(resolve => child.on('exit', resolve));
assert.equal(code, 0, `provider bill worker should exit 0\n${output}`);
assert.match(output, /provider-bills batch processed=\d+/, 'worker output should include provider bill summary');
assert.ok(!output.includes(paymentFixture.wechatApiV3Key), 'provider bill worker output must not leak WeChat API v3 key');
assert.ok(!output.includes('PRIVATE KEY'), 'provider bill worker output must not leak private keys');
return output;
}
async function cleanup(pool) {
await pool.query(
`
delete from public.commerce_bill_download_jobs
where tenant_id = $1
and (
id in ($2::uuid, $3::uuid)
or source_name like 'provider-bill:%:2026-06-29:%'
)
`,
[tenantId, ids.providerBillJobWechat, ids.providerBillJobAlipay],
);
await pool.query(
`
delete from public.commerce_reconciliation_batches
where tenant_id = $1
and source = 'provider_download'
and source_name like 'provider-bill:%:2026-06-29:%'
`,
[tenantId],
);
await pool.query(
`
delete from public.payment_events
@@ -169,20 +318,20 @@ async function cleanup(pool) {
await pool.query(
`
delete from public.payments
where tenant_id = $1 and id in ($2::uuid, $3::uuid)
where tenant_id = $1 and id in ($2::uuid, $3::uuid, $4::uuid, $5::uuid)
`,
[tenantId, ids.payment, ids.refundPayment],
[tenantId, ids.payment, ids.refundPayment, ids.providerBillPaymentWechat, ids.providerBillPaymentAlipay],
);
await pool.query(
`
delete from public.orders
where tenant_id = $1 and id in ($2::uuid, $3::uuid)
where tenant_id = $1 and id in ($2::uuid, $3::uuid, $4::uuid, $5::uuid)
`,
[tenantId, ids.paymentOrder, ids.refundOrder],
[tenantId, ids.paymentOrder, ids.refundOrder, ids.providerBillOrderWechat, ids.providerBillOrderAlipay],
);
}
async function seed(pool, fakeWechat) {
async function seed(pool, fakeWechat, fakeAlipay) {
await pool.query(
`
insert into app_private.tenant_secrets (
@@ -201,6 +350,23 @@ async function seed(pool, fakeWechat) {
})],
);
await pool.query(
`
insert into app_private.tenant_secrets (
tenant_id, secret_scope, secret_key, secret_json, provider, last_rotated_at
)
values ($1, 'payment', 'alipay', $2::jsonb, 'alipay', now())
on conflict (tenant_id, secret_scope, secret_key)
do update set secret_json = excluded.secret_json,
provider = excluded.provider,
last_rotated_at = now(),
updated_at = now()
`,
[tenantId, JSON.stringify({
privateKey: paymentFixture.alipayAppPrivateKey,
})],
);
await pool.query(
`
insert into public.tenant_payment_accounts (
@@ -221,10 +387,32 @@ async function seed(pool, fakeWechat) {
paymentQueryEndpoint: fakeWechat.paymentQueryEndpoint,
refundEndpoint: fakeWechat.refundEndpoint,
refundQueryEndpoint: fakeWechat.refundEndpoint,
tradeBillEndpoint: fakeWechat.tradeBillEndpoint,
secretRef: 'app_private.tenant_secrets:payment:wechat_pay',
})],
);
await pool.query(
`
insert into public.tenant_payment_accounts (
tenant_id, provider, mode, display_name, status, config_public
)
values ($1, 'alipay', 'tenant_collect', 'Worker Alipay', 'active', $2::jsonb)
on conflict (tenant_id, provider)
do update set mode = excluded.mode,
display_name = excluded.display_name,
status = excluded.status,
config_public = excluded.config_public,
updated_at = now()
`,
[tenantId, JSON.stringify({
appId: 'ali-worker-appid',
endpoint: fakeAlipay.endpoint,
billEndpoint: fakeAlipay.endpoint,
secretRef: 'app_private.tenant_secrets:payment:alipay',
})],
);
await pool.query(
`
insert into public.orders (
@@ -234,9 +422,24 @@ async function seed(pool, fakeWechat) {
)
values
($1, $2, $3, $4, 'pending', 'svip', 'Worker 补偿月卡', 990, 'jsapi', 'wechat_pay', $5, 30, $6, '{"source":"commerce-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes'),
($7, $2, $3, $8, 'paid', 'svip', 'Worker 退款月卡', 500, 'jsapi', 'wechat_pay', $5, 30, $6, '{"source":"commerce-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes')
($7, $2, $3, $8, 'paid', 'svip', 'Worker 退款月卡', 500, 'jsapi', 'wechat_pay', $5, 30, $6, '{"source":"commerce-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes'),
($9, $2, $3, $10, 'paid', 'svip', 'Worker 微信账单月卡', 990, 'jsapi', 'wechat_pay', $5, 30, $6, '{"source":"provider-bill-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes'),
($11, $2, $3, $12, 'paid', 'svip', 'Worker 支付宝账单月卡', 1288, 'wap', 'alipay', $5, 30, $6, '{"source":"provider-bill-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes')
`,
[ids.paymentOrder, tenantId, studentUserId, orderNos.payment, planId, regionId, ids.refundOrder, orderNos.refund],
[
ids.paymentOrder,
tenantId,
studentUserId,
orderNos.payment,
planId,
regionId,
ids.refundOrder,
orderNos.refund,
ids.providerBillOrderWechat,
orderNos.providerBillWechat,
ids.providerBillOrderAlipay,
orderNos.providerBillAlipay,
],
);
await pool.query(
@@ -247,9 +450,24 @@ async function seed(pool, fakeWechat) {
)
values
($1, $2, $3, 'wechat_pay', 'jsapi', 'pending', 990, null, null, '{"source":"commerce-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes'),
($4, $2, $5, 'wechat_pay', 'jsapi', 'paid', 500, $6, now() - interval '9 minutes', '{"source":"commerce-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes')
($4, $2, $5, 'wechat_pay', 'jsapi', 'paid', 500, $6, now() - interval '9 minutes', '{"source":"commerce-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes'),
($7, $2, $8, 'wechat_pay', 'jsapi', 'paid', 990, $9, now() - interval '9 minutes', '{"source":"provider-bill-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes'),
($10, $2, $11, 'alipay', 'wap', 'paid', 1288, $12, now() - interval '9 minutes', '{"source":"provider-bill-worker-test"}'::jsonb, now() - interval '10 minutes', now() - interval '10 minutes')
`,
[ids.payment, tenantId, ids.paymentOrder, ids.refundPayment, ids.refundOrder, `wx-trade-${orderNos.refund}`],
[
ids.payment,
tenantId,
ids.paymentOrder,
ids.refundPayment,
ids.refundOrder,
`wx-trade-${orderNos.refund}`,
ids.providerBillPaymentWechat,
ids.providerBillOrderWechat,
`wx-bill-trade-${orderNos.providerBillWechat}`,
ids.providerBillPaymentAlipay,
ids.providerBillOrderAlipay,
`ali-bill-trade-${orderNos.providerBillAlipay}`,
],
);
await pool.query(
@@ -281,16 +499,52 @@ async function seed(pool, fakeWechat) {
`,
[ids.refundRequest, tenantId, ids.refundOrder, ids.refundPayment, refundNo],
);
await pool.query(
`
insert into public.commerce_bill_download_jobs (
id, tenant_id, provider, bill_date, bill_type, status,
source_name, requested_by, metadata
)
values
($1, $3, 'wechat_pay', $4::date, 'payment', 'queued', $5, null, '{"source":"commerce-worker-test"}'::jsonb),
($2, $3, 'alipay', $4::date, 'payment', 'queued', $6, null, '{"source":"commerce-worker-test"}'::jsonb)
on conflict (tenant_id, provider, bill_date, bill_type)
where status in ('queued', 'running', 'completed')
do update set status = 'queued',
source_name = excluded.source_name,
reconciliation_batch_id = null,
source_hash = null,
row_count = 0,
claimed_by = null,
claimed_at = null,
completed_at = null,
failed_at = null,
error_code = null,
error_message = null,
metadata = excluded.metadata,
updated_at = now()
`,
[
ids.providerBillJobWechat,
ids.providerBillJobAlipay,
tenantId,
providerBillDate,
`provider-bill:wechat_pay:${providerBillDate}:payment`,
`provider-bill:alipay:${providerBillDate}:payment`,
],
);
}
async function main() {
const fakeWechat = await startFakeWechatPayServer();
const fakeAlipay = await startFakeAlipayServer();
const pool = new pg.Pool({ connectionString: databaseUrl });
let seeded = false;
try {
await pool.query('begin');
await cleanup(pool);
await seed(pool, fakeWechat);
await seed(pool, fakeWechat, fakeAlipay);
await pool.query('commit');
seeded = true;
@@ -378,6 +632,70 @@ async function main() {
assert.ok(!JSON.stringify(events.rows).includes(paymentFixture.wechatApiV3Key), 'payment event payload must not leak API v3 key');
assert.ok(!JSON.stringify(events.rows).includes('PRIVATE KEY'), 'payment event payload must not leak private key');
const providerBillOutput = await runProviderBillsWorkerOnce();
assert.match(providerBillOutput, /completed=2/, 'provider bill worker should complete two provider bill jobs');
assert.ok(
fakeWechat.requests.some(item => item.method === 'GET' && item.pathname === '/v3/bill/tradebill'),
'provider bill worker should request WeChat trade bill URL',
);
assert.ok(
fakeWechat.requests.some(item => item.method === 'GET' && item.pathname === '/download/wechat-tradebill.json'),
'provider bill worker should download WeChat trade bill',
);
assert.ok(
fakeAlipay.requests.some(item => item.method === 'POST' && item.pathname === '/gateway.do' && item.body.includes('alipay.data.dataservice.bill.downloadurl.query')),
'provider bill worker should request Alipay bill download URL',
);
assert.ok(
fakeAlipay.requests.some(item => item.method === 'GET' && item.pathname === '/download/alipay-tradebill.json'),
'provider bill worker should download Alipay trade bill',
);
const providerJobs = await pool.query(
`
select provider, status, source_hash, row_count, download_url_host,
reconciliation_batch_id, error_code, error_message
from public.commerce_bill_download_jobs
where tenant_id = $1 and id in ($2::uuid, $3::uuid)
order by provider
`,
[tenantId, ids.providerBillJobWechat, ids.providerBillJobAlipay],
);
assert.equal(providerJobs.rows.length, 2, 'provider bill jobs should be persisted');
assert.ok(providerJobs.rows.every(row => row.status === 'completed'), 'provider bill jobs should be completed');
assert.ok(providerJobs.rows.every(row => row.source_hash && !row.error_code && !row.error_message), 'completed provider bill jobs should keep source hash and no errors');
assert.ok(providerJobs.rows.every(row => row.download_url_host === '127.0.0.1'), 'provider bill jobs should persist only download URL host');
const providerBatches = await pool.query(
`
select b.id, b.provider, b.source, b.source_name, b.status,
b.matched_count, b.missing_local_count, b.total_count,
count(i.id)::int as item_count
from public.commerce_reconciliation_batches b
join public.commerce_reconciliation_items i
on i.tenant_id = b.tenant_id and i.batch_id = b.id
where b.tenant_id = $1
and b.source = 'provider_download'
and b.source_name like 'provider-bill:%:2026-06-29:%'
group by b.id
order by b.provider
`,
[tenantId],
);
assert.equal(providerBatches.rows.length, 2, 'provider bill downloads should create two reconciliation batches');
assert.ok(
providerBatches.rows.some(row => row.provider === 'wechat_pay' && row.status === 'completed_with_issues' && row.matched_count >= 1 && row.missing_local_count >= 1),
'WeChat bill batch should include matched and provider-only anomaly rows',
);
assert.ok(
providerBatches.rows.some(row => row.provider === 'alipay' && ['completed', 'completed_with_issues'].includes(row.status) && row.matched_count >= 1),
'Alipay bill batch should include matched rows',
);
const jobPayload = JSON.stringify(providerJobs.rows);
assert.ok(!jobPayload.includes(paymentFixture.wechatApiV3Key), 'provider bill job rows must not leak API v3 key');
assert.ok(!jobPayload.includes('PRIVATE KEY'), 'provider bill job rows must not leak private keys');
console.log('Commerce worker integration test complete.');
} catch (error) {
await pool.query('rollback').catch(() => {});
@@ -388,6 +706,7 @@ async function main() {
}
await pool.end();
await fakeWechat.close();
await fakeAlipay.close();
}
}

View File

@@ -0,0 +1,56 @@
create table if not exists public.commerce_bill_download_jobs (
id uuid primary key default gen_random_uuid(),
tenant_id uuid not null references public.tenants(id) on delete cascade,
provider text not null
check (provider in ('wechat_pay', 'alipay')),
bill_date date not null,
bill_type text not null default 'payment'
check (bill_type in ('payment', 'refund', 'combined')),
status text not null default 'queued'
check (status in ('queued', 'running', 'completed', 'failed', 'cancelled')),
source_name text not null,
source_hash text,
row_count integer not null default 0 check (row_count >= 0),
download_hash_type text,
download_hash_value text,
download_url_host text,
reconciliation_batch_id uuid references public.commerce_reconciliation_batches(id) on delete set null,
requested_by uuid references public.platform_users(id) on delete set null,
claimed_by text,
claimed_at timestamptz,
completed_at timestamptz,
failed_at timestamptz,
error_code text,
error_message text,
metadata jsonb not null default '{}'::jsonb,
created_at timestamptz not null default now(),
updated_at timestamptz not null default now()
);
create unique index if not exists idx_commerce_bill_download_jobs_idempotency
on public.commerce_bill_download_jobs(tenant_id, provider, bill_date, bill_type)
where status in ('queued', 'running', 'completed');
create index if not exists idx_commerce_bill_download_jobs_claim
on public.commerce_bill_download_jobs(status, created_at)
where status = 'queued';
create index if not exists idx_commerce_bill_download_jobs_tenant
on public.commerce_bill_download_jobs(tenant_id, provider, bill_date desc, created_at desc);
create index if not exists idx_commerce_bill_download_jobs_batch
on public.commerce_bill_download_jobs(tenant_id, reconciliation_batch_id)
where reconciliation_batch_id is not null;
alter table public.commerce_bill_download_jobs enable row level security;
drop policy if exists tenant_isolation on public.commerce_bill_download_jobs;
create policy tenant_isolation on public.commerce_bill_download_jobs
for all
using (tenant_id = app.current_tenant_id() or app.is_platform_admin())
with check (tenant_id = app.current_tenant_id() or app.is_platform_admin());
drop trigger if exists set_updated_at on public.commerce_bill_download_jobs;
create trigger set_updated_at
before update on public.commerce_bill_download_jobs
for each row execute function app.touch_updated_at();